eLearning User Penetration Rate

I just created a script to determine the penetration rate of eLearning online Learning Management System (powered by Blackboard Learning System CE6, formly known as WebCT CE6) by comparing the enrollment data, Active Directory groups, with the actual logon users of the current semester.

For 2007 Fall semester, we have the following numbers in our main Pullman campus:

Undergraduate Students			Graduate Students			Undergraduate + Graduate Students
Total	LMS	%	Total	LMS	%	Total	LMS	%
15441	12862	83.30	1981	749	37.81	17422	13611	78.13

BbWorld 07 Conference Presentation

Blackboard acquired WebCT in year 2006. There are a lot of changes made soon after the merger. The product WebCT Campus Edition was renamed to Blackboard Learning System CE. A number of senior engineers from WebCT left. The support model was changed. But the main issue is the lack of good and timely technical support, and this have caused us a lot of struggle to support the product day to day.

But our spirits remain strong. We, the same group who presented last year, met again and co-presented "A Year Later : Looking Back and Moving Forward" in the BbWorld '07 Conference. Here are the slides from our university:

SharePoint MOSS 2007 with SSL termination on Load Balancer

We want to enable SSL in our SharePoint (MOSS 2007). Since we already have a pair of load balancers (F5 Network's BigIP load balancers) for our Blackboard Learning Management System. We would like to use them for SSL termination for SharePoint as well. The advantage is that it offloads all encryption and decryption work from our SharePoint servers on to the load balancer (which is designed to do that work, and more).

The network path is as follows:

Browser ---https---> load balancer ---http---> SharePoint servers

However, it turns out to be not an easy task. We found that the URLs embedded in http responses (such as form action link) from SharePoint are in http. Since SharePoint never knows that the traffic was originally https (as you can see from the network path above), of course it would embed URLs in http. It kind of makes sense.

I searched all over the places to see if someone had already found a solution.

One suggestion was to use the stream profile of the load balancer as workaround:

• On the BigIP load balancer, under Local Traffic | Virtual Servers | Profiles, choose Others | Stream.
• Create a Stream profile with Settings:
  Source http://sp.domain.com
  Target https://sp.domain.com

It does work. All "http://sp.domain.com" in the http responses from SharePoint would be replaced by "https://sp.domain.com". If you decide to purse this approach, you must read AskF5 knowledge base article SOL6422: Using the Stream profile with HTTP traffic may prevent the client from displaying all of the data. It documents a known issue of Stream profile, and the workaround.

But I am persistence, and kept pursuing further for the real fix in SharePoint. The following two articles had been very useful in helping me derive my own solution using BigIP load balancers.

• F5's Deployment Guide Microsft SharePoint Portal Server 2003 (BIG-IP v9.x)
• Microsoft SharePoint Team Blog What every SharePoint administrator needs to know about Alternate Access Mappings (Part 1 of 3)

It took me a day, and I think I figured it out:

• First you create a Sharepoint site in default zone, and port
  spsite port 8888
• Sharepoint will create the web application, content database accordingly.
• Then, extend this web application to a new SharePoint web site with your internal host name, port, and no SSL
  http://sp.domain.com port 80
• In the Load Balanced URL field, use https://sp.domain.com (yes, https here!).
• Put this site in Internet zone.
• Then, go to Operations | Alternate Access Mapping. You will see that the following entries:
  
  

  Internal URL	Zone	Public URL for Zone
  http://spsite:8888	Default	http://spsite:8888
  https://sp.domain.com	Internet	https://sp.domain.com

  
  
• Now, click on Add Internal URLs. Add your internal non-SSL url as Internet Zone.
  http://sp.domain.com Internet
• Then, go back to Operations | Alternate Access Mapping screen. You will see that the following entries:
  
  

  Internal URL	Zone	Public URL for Zone
  http://spsite:8888	Default	http://spsite:8888
  https://sp.domain.com	Internet	https://sp.domain.com
  http://sp.domain.com	Internet	https://sp.domain.com

Only then, SharePoint will know that the incoming URL http://sp.domain.com is associated with the Internet zone, and it should embed https://sp.domain.com inside form action link, etc when sending responses back to users.

Using Gawk for Log Analysis

Gawk is a very powerful text processing and pattern matching utility. It is the Gnu version of awk. I use it to search into the logs where grep cannot do.

For example, in our Blackboard CE/Vista 8, the webserver.log files contain the following fields:

date time time-taken c-ip x-weblogic.servlet.logging.ELFWebCTSession sc-status cs-method cs-uri-stem cs-uri-query bytes x-weblogic.servlet.logging.ELFWebCTExtras cs(User-Agent)

I can use the gawk command to easily find all http requests that had taken longer than 60 seconds to process.

gawk -F\t "$3 > 60" webserver.log

In gawk commands, the fields are preceeded by a $ sign. ie. $1 refers to date, $2 refers to time and so forth. Use the -F switch to specify the delimiter which is tab in this case.

To find all http requests that had taken a long time to process (larger than 60 seconds), but do not involve downloading of large files (smaller than 50MB):

gawk -F\t "($3 > 60) && ($10 < 52428800)" webserver.log

To find all http requests that got the error 500:

gawk -F\t "$6 == 500" webserver.log

For Windows users, you can have gawk by installing cygwin. Remember to add c:\cygwin\bin (or wherever your installation directory is) to your environment path. This way, you can run gawk directly from any command prompt or inside a script.