CWebProxy provides a channel parameter called cw_person, which is a comma-separated list of person attributes. If this parameter is set, CWebProxy is supposed to fetch the listed attributes and pass them to the back-end web application as CGI parameters. This is a potentially handy feature, because then we can develop “smart” unauthenticated apps which present content tailored to individual users. For example, in our case we’d like to develop a “campus links” channel, which presents different sets of links to different users based on their LDAP affiliations. In theory, with cw_person, that should be easy to do.

Well, I tried this out, and as usual, what sounds great in theory isn’t always great in practice. There are two issues with this feature:

1. If a user has multiple values for a given attribute, CWebProxy only passes one of them (presumably the first one returned by the LDAP query). Example: I set up a test channel with cw_person set to pass the LDAP affiliation attribute. My LDAP affiliations are “staff”, “employee”, and “alumni”, but CWebProxy only passes “staff”.
2. When the channel is refreshed (e.g. by switching to a different tab and then going back), it seems to stop passing the attribute. Not sure if this is user error, or if that’s just how it works. But if this is going to be any use to us, it needs to pass the attributes every time the channel is rendered.

These two problems will probably prevent us from using this feature to do what we want. Yeah, we can probably hack CWebProxy to make this work, but I think a better solution would be to write up a custom local connection context. That will give me complete control over what gets passed to the back-end app (and when), with the added benefit that we can use it with a CGenericXSLT type channel and aren’t limited to using a web proxy.

Incidentally, for anyone trying to set up a channel with cw_person, there’s one big “gotcha”. There is an additional channel parameter called cw_personAllow. This parameter is a list of attributes that the channel is allowed to pass to the back-end app. The default is to disallow every attribute. So if you’re like most people, you’ll set up cw_person, ignore cw_personAllow, and then wonder why it doesn’t work. To get it to pass your attributes, you can either set cw_personAllow to ‘*’ (meaning “pass any attribute”), or specify an explicit list. This can be done in the channel definition, but there’s also a global default in portal.properties called org.jasig.portal.channels.webproxy.CWebProxy.person_allow. Yes, this is all in the CWebProxy documentation, but you have to dig for it. A tutorial would probably help.

First: The facts of life.

1. UMBC has the concept of a “student ID number.” This number is used in SIS as the primary key for almost every table. All students have these IDs, as do faculty members who teach courses, etc.
2. uPortal itself does not use or require this number to do its work.
3. The legacy myUMBC portal uses this number to do its internal session management. Also, we need to know this number to look up any SIS data on behalf of the user.
4. uPortal proxies to the legacy portal to do most of its “real work” involving SIS functionality.
5. When a user logs into either portal, they provide only a username and password. We need to be able to take that username and map it to a student ID. To do this, we query our campus LDAP directory.
6. Not all users have student ID attributes in LDAP, for various reasons. In particular, if the help desk manually enters someone into the directory, they sometimes leave out the ID.
7. These users have problems accessing the portal.

Now, in the old portal, we handled these cases by prompting the user to manually enter the student ID and a 4-digit PIN (don’t go there). In uPortal, we don’t have this logic. So, it just breaks. In fact, it breaks in such a way that “bad” HTML is generated, so the channels in question don’t render at all.

Well, the rendering issue is fixed now, but it was tricker than I thought it would be. It wasn’t just a matter of tidying the HTML in the legacy Perl code. The problem was actually happening in my local connection context code. Basically in these cases, the connection context can’t generate a valid legacy portal session at all. And when that happened, it was crapping out before it added a couple of necessary URI parameters to the web proxy URL. Long story short, it was trying to bring up a legacy portal login screen, complete with navigation, decoration, and lotsa sloppy HTML to boot, and uPortal of course was refusing to render it.

Fortunately it appears that the solution to this is going to be human-engineered, meaning I will just display some appropriate wording that tells the user how to remedy the problem. So, I shouldn’t need to do anything fancier than what I’ve already done.

As an aside, I’m still working on the best way to fix these HTML-tidy issues with the web proxy channels. First trick I try, is masquerading as the user with the problem, finding the bad channel, and examining it outside the portal by passing the same arguments to the legacy myUMBC web app. Then I can view the document source, paste it into emacs, and look for problems. Once I think I have it fixed, I save it as static HTML and point another web proxy channel at it, to see if it’ll render. That usually works. Sometimes I need a Java stack trace, which can be problematic because “ordinary” users don’t have the ability to display stack traces within channels (there might be something in portal.log, but it’s often hard to sift through this and correlate log entries with individual problems). In these cases, I hack the code to force the error condition to occur with my own account. Then I can see the stack trace, find out the nature of the evil, and fix it. Yeah, it’s klunky, but it gets the job done.

So, we have this new campus portal at UMBC. And for the most part, the launch has gone pretty well. As part of this whole thing, we’re rethinking some of our old, outdated business processes and changing the way they work under the new portal. One of these is the PIN (Personal ID Number). Waaaaay back in 1996 when UMBC first launched web-based course registration, we required all students to enter a 4-digit PIN to log in. There was also a service where students could register by phone, that used the same PIN. Flash forward to 2006. We’re now using a campus-wide single signon system, the telephone registration system is gone, and we’ve done away with PINs as part of the login process. So, since students aren’t using them any more, we can just get rid of PINs altogether, right? Wrong. Problem is, we’re still using our old, crusty HP3000 mainframe as system of record for registration, so when we do online course registration, we have to play by the HP3000’s rules. The HP3000 is still running circa-1996 registration code (written in Cobol), and PINs are so deeply embedded into that code, that there’s no way we’re ever getting rid of them as long as the HP is around. We can rework things so that all the PIN stuff is handled behind-the-scenes, and users never see them or even know they exist, but on the back end, they’re still going to be there.

Now.. the HP3000 stores everybody’s PIN in a database table. But initially, that table is not populated until a user accesses the system for the first time. Then, the HP figures out an initial PIN for the user, and uses that to populate the PIN table. It then sets a flag that the user’s PIN needs to be changed. The HP will then refuse to do anything on behalf of that user, until they change their PIN. The mandatory PIN change happens when the user logs into myUMBC. With the old version of myUMBC, they would see the mandatory PIN change screen immediately after logging in, at which point they’d need to change the PIN before doing anything else in myUMBC. But again, this behavior is a relic of the days when most activities in myUMBC centered around the HP. It also prevents certain users (people who lack the appropriate data that the HP needs to generate the initial PIN) from using myUMBC at all. If we’re going to move forward, we need to get rid of this behavior.

The first step towards this goal, was to eliminate the mandatory PIN change check on initial login. Now keep in mind that we have to submit an initial PIN change request for every student, before they can do anything that involves the HP. So, I’m now doing the mandatory PIN change only when the user requests a function that uses the HP. So rather than seeing it when they first log in, they see it when they try to register for the first time. A small but significant step.

This works great, except it broke the online student parking registration app. Student Parking Registration has the distinction of being the only external (not part of the monolithic legacy myUMBC code) app that talks directly to the HP. If a student goes to this app before changing their PIN, the HP will refuse the parking registration request and the app will fail silently. Yep, this was a fun one to debug. It’s still broken, until I figure out the best way to fix it.

As expected, the whole uPortal affinity thing is really drawing folks out of the woodwork. For those of you who came in late, uPortal uses a totally different scheme from the old portal to determine who sees what content. Let’s take the “Faculty Options” channel as an example:

Portal	You see faculty options if…
old myUMBC	You are in AUTH_CLASS or TEACHER SIS tables, or have faculty LDAP affiliation
uPortal	You have faculty, staff or instructor LDAP affiliation

As I said… totally different. Now.. most folks who need faculty options will satisfy both portals’ conditions, and will see the content in both portals. However, there are always special cases, and as expected, we’ve run into a few. First, not everyone in AUTH_CLASS has one of the three magic LDAP attributes. Case in point, the education department has graduate assistants that do course authorizations for them. These people need to see the faculty options, but don’t get them in uPortal.

What we really need to do here, is rethink how we’re doing some of these affinities and who’s seeing what content. Some of this will just involve creating new PAGS groups to correspond with various affiliations. But, we may also need to create additional affiliations in LDAP to cover certain cases. Membership in AUTH_CLASS would be one of those cases. uPortal allows me to specify Person Attributes based on DB queries, so for some of these I might be able to go that route. But I’d prefer to keep this totally LDAP based.

In the meantime, I’m making things work for these people by granting them temporary staff affiliations in LDAP. Can’t have them unable to do their jobs while we’re figuring this out.

Today began with an effort to track down the infinite loop which is causing the java process to munch CPU cycles on the portal server. The problem seems to be spreading.. when I checked this morning around 6:30am, there were 4 looping threads on uportal1 and 5 on uportal2. I’ve tracked down the offending loop, and I started out by putting a counter into it and logging the number of times the loop ran after each successful completion. This gave me a pretty good idea of how many times we should go through the loop under normal circumstances (looks like only 2 or 3). Then I picked an arbitrary large number, 1000, and changed the code so it throws an exception if the counter exceeds this value. I’m hoping this will have two effects: one, stop the looping; and two, provide some logging so we can further investigate what is causing the problem. The new code is up and running, so we’ll see how it goes.

Welp, it worked. Interesting… very similar to yesterday, everything was quiet all morning and then both instances hit the infinite loop almost exactly at 1pm. Now, instead of an endlessly looping thread, I get a nice error log and stack dump. Next thing to do is try to log some additional info, to see if I can narrow this down to a particular user, activity, or whatever. If this is only affecting certain user(s), maybe someone will call the help desk and help me solve the mystery.

Other than that, the biggest issues so far have been related to permissions and affinities. Lots of people complaining that they don’t see content they used to get on the old portal. I expected this, because the uPortal groups/permissions model is quite different from what we were using with the old myUMBC. For now, I’m noting the users who are having problems, and in a week or so I’ll call a meeting to discuss how to reconcile them. In the meantime, these users can continue to use the old portal.

I also discovered today that the new portal does not work for anyone who has a “mandatory PIN change” flag set in SIS. Now first off… PINs are going away. There’s nowhere in myUMBC where a user is required to enter a PIN any more (well, there’s orientation.. but don’t go there). Given that, I made the executive decision that mandatory PIN changes are going to go away in uPortal, and tweaked the legacy code accordingly. However, it looks like the HP is checking the forced_pin_change field and disallowing registration if it is set. So it looks like I need to take this one step further, and actually submit a PIN change for these users behind the scenes. Looking into that now. Regardless, we’re definitely going to need to test how the portal behaves with “virgin” users, before the fall semester starts.

My God, I’m going through my own code that does the HP PIN stuff, and it is so bloody convoluted I want to shoot myself. From following the code, I can’t see any way for it to ever get to the HP PIN verify when accessed normally. I think what we need is a rewrite that takes all of the PIN stuff out of the normal login process, and doesn’t do anything with PINs until the user does something that accesses the HP. Then, if the user doesn’t have a PIN it can have the HP create one, then send a PIN change request to the HP so it clears out the forced_pin_change flag. Will look at that tomorrow..

First “real” test (i.e. University open for business, people hitting portal) for our re-launch of uPortal today. It went up Sunday.

Issue #1: I see that we still have the problem of the JVM going up to 100% CPU utilization occasionally. It was like that this morning when I signed on around 7:30am. Portal was still responsive. The problem went away when I bounced the Tomcat instance. I guess somewhere, a thread is going haywire or something. I learned how to get a thread dump under Tomcat: Send SIGQUIT to the JVM process, and Tomcat puts the thread dump in catalina.out. Next time it happens, I’ll see if this produces anything useful.

Well, the issue cropped up again, and I did a thread dump. It appears we’re getting hit with UP-1175. Somewhere there’s a corrupted layout with a circular reference, which is causing an infinite loop. They’ve fixed it for uPortal 2.5.x but there appears to be no fix for 2.4.x. Need to look into this a little further.. Other than that, things have gone pretty well so far today. Fingers crossed.