Fruddled Gruntbugglies

Blog

  • Bluetooth and iSync

    Bluetooth is a really neat technology. It’s got a lot of potential, but there are still a lot of issues with it. The pie-in-the-sky dream is that we will have all these different Bluetooth gadgets that all interoperate perfectly. The reality, of course, is not as rosy. There are lots of devices that speak Bluetooth, but for various reasons I won’t go into here, they don’t all work with each other, at least not without lots of fiddling. So, when something does work perfectly, it’s noteworthy. A little sad, but true.

    This week I got a new Motorola e815 phone for work. One thing I’ve always dreadded about getting a new cell phone, is trying to get all of my stored contacts from my old phone into the new phone. It’s always been a tedious, manual process. Anyhow, the e815 supports Bluetooth. My Powerbook has an app called iSync, which purports to sync info from iCal/Address Book over Bluetooth to various models of phones. Fired everything up, went to Bluetooth Options on the Mac, paired it with the new phone, ran iSync, and my entire address book was transferred to the phone. The whole process took less than 10 minutes.

    This is the way this kind of technology is supposed to work. I wish everything else was that easy. I initially tried to pair the phone with my Palm Tungsten E2, but I haven’t been able to get that to work. I’ve heard that it’s possible to get the phone to act as a modem so that you can surf the net on the Palm. However, if it doesn’t work out of the box, I’m not going to bother. It’s not a feature I’d use a whole lot, and I don’t want to waste a lot of time fiddling with it, especially if there’s a chance I won’t be successful. I’m just happy that I can transfer my address book to the phone. It’s always bugged me that I’ve got this great phone book on my Mac and Palm, but not on my cell phone, where I am actually making the calls.

    Now, if I could just export the Mac address book to Thunderbird (on the Mac and all my various Linux boxes), I’ll be even happier…

  • Stalking the infinite loop

    Today began with an effort to track down the infinite loop which is causing the java process to munch CPU cycles on the portal server. The problem seems to be spreading.. when I checked this morning around 6:30am, there were 4 looping threads on uportal1 and 5 on uportal2. I’ve tracked down the offending loop, and I started out by putting a counter into it and logging the number of times the loop ran after each successful completion. This gave me a pretty good idea of how many times we should go through the loop under normal circumstances (looks like only 2 or 3). Then I picked an arbitrary large number, 1000, and changed the code so it throws an exception if the counter exceeds this value. I’m hoping this will have two effects: one, stop the looping; and two, provide some logging so we can further investigate what is causing the problem. The new code is up and running, so we’ll see how it goes.

    Welp, it worked. Interesting… very similar to yesterday, everything was quiet all morning and then both instances hit the infinite loop almost exactly at 1pm. Now, instead of an endlessly looping thread, I get a nice error log and stack dump. Next thing to do is try to log some additional info, to see if I can narrow this down to a particular user, activity, or whatever. If this is only affecting certain user(s), maybe someone will call the help desk and help me solve the mystery.

    Other than that, the biggest issues so far have been related to permissions and affinities. Lots of people complaining that they don’t see content they used to get on the old portal. I expected this, because the uPortal groups/permissions model is quite different from what we were using with the old myUMBC. For now, I’m noting the users who are having problems, and in a week or so I’ll call a meeting to discuss how to reconcile them. In the meantime, these users can continue to use the old portal.

    I also discovered today that the new portal does not work for anyone who has a “mandatory PIN change” flag set in SIS. Now first off… PINs are going away. There’s nowhere in myUMBC where a user is required to enter a PIN any more (well, there’s orientation.. but don’t go there). Given that, I made the executive decision that mandatory PIN changes are going to go away in uPortal, and tweaked the legacy code accordingly. However, it looks like the HP is checking the forced_pin_change field and disallowing registration if it is set. So it looks like I need to take this one step further, and actually submit a PIN change for these users behind the scenes. Looking into that now. Regardless, we’re definitely going to need to test how the portal behaves with “virgin” users, before the fall semester starts.

    My God, I’m going through my own code that does the HP PIN stuff, and it is so bloody convoluted I want to shoot myself. From following the code, I can’t see any way for it to ever get to the HP PIN verify when accessed normally. I think what we need is a rewrite that takes all of the PIN stuff out of the normal login process, and doesn’t do anything with PINs until the user does something that accesses the HP. Then, if the user doesn’t have a PIN it can have the HP create one, then send a PIN change request to the HP so it clears out the forced_pin_change flag. Will look at that tomorrow..

  • Relaunch day

    First “real” test (i.e. University open for business, people hitting portal) for our re-launch of uPortal today. It went up Sunday.

    Issue #1: I see that we still have the problem of the JVM going up to 100% CPU utilization occasionally. It was like that this morning when I signed on around 7:30am. Portal was still responsive. The problem went away when I bounced the Tomcat instance. I guess somewhere, a thread is going haywire or something. I learned how to get a thread dump under Tomcat: Send SIGQUIT to the JVM process, and Tomcat puts the thread dump in catalina.out. Next time it happens, I’ll see if this produces anything useful.

    Well, the issue cropped up again, and I did a thread dump. It appears we’re getting hit with UP-1175. Somewhere there’s a corrupted layout with a circular reference, which is causing an infinite loop. They’ve fixed it for uPortal 2.5.x but there appears to be no fix for 2.4.x. Need to look into this a little further.. Other than that, things have gone pretty well so far today. Fingers crossed.

  • The never-ending basement plumbing project drags on

    I finally got back to working on my endless plumbing project today.

    A couple weeks back, I pressure tested the new branch I ran for the outside sillcock. To do this, I soldered a female adapter onto a 12″ length of pipe, screwed a quick-connect air coupling into it, and attached the other end to my branch with a compression fitting. Then I hooked it up to the compressor, cranked it up to around 30psi, and let it sit. There was a v-e-r-y slow loss of pressure, maybe 1psi or so over several hours. This was a little troubling, but it didn’t necessarily mean there was a leak (it could be the stop valve packing, the compression fitting, or even the regulator gauge). It did make me fairly confident I didn’t have any “gushers” or “blowout” type leaks.

    This morning I got the idea to use a couple compression fittings and hook the branch up to my existing plumbing. That way I can be absolutely sure that the branch holds water, and when I’m ready to make the final repair, I won’t have to worry about the branch leaking. So, that’s what I did today. Cut a short piece of copper to fit between the existing plumbing and the new branch, shut off the water, drained the plumbing, and hooked the whole mess together. When I turned the water back on, I found my pressure loss pretty quickly: it was a slow drip at the stop valve, where the valve “guts” screw into the valve body. I had taken the valve apart to sweat it, and it just needed to be tightened a bit. I also had to tighten one of the compression joints. Other than that, everything looks good, and I can finally use the sillcock I put in 6 months ago (just in time for the dead of winter. But hey — it’s frost-free)!

    With that, the only thing left is to make the original repair, which was the driving force behind this entire project. Maybe I’ll get around to that by next January or so.

  • Stupid hard-starting generator

    We have this generator, that we bought a few years back to use during power outages. It spends 99.9% of its time sitting in the garage gathering dust (much like the snowblower). Every few months, I start the thing up and run it for 20 minutes or so, just to make sure it will work if we ever really do need it. That was one of today’s chores.

    Now, the generator has always been a little hard to start when it’s been sitting for 2 or 3 months. But it’s pretty reliable: it takes 9 or 10 yanks on the cord, then it starts up. Then you have to run it partially choked for several minutes so it doesn’t miss. Then, finally, it will warm up and run at the factory-set fuel mixture. Not exactly a finely tuned machine, but at least it works.

    Today, I figured, well, it’s January, it’s cold, so the generator will probably be even more reluctant to start. So, I got the bright idea to spray some starting fluid into the carb. I did, and it fired right up on the first pull. Then, after 10 seconds or so, it died. And refused to start up again for love or money.

    A couple dozen cord pulls, several more shots of starting fluid, and countless swear words later, I checked the oil. Hmm, seems a tad low. Topped it off with some 10w30. Started on the first pull, and stayed running. I guess there was just enough oil in there to start it initially, but once it started sloshing around in there, it tripped the low oil cutoff. Moral of the story: I need to check the oil every time I start the thing up. Followup: I later checked my records, and found that this was the first time I had tried to start it after changing the oil a couple months ago. I guess I didn’t add enough.

    My theory about the hard starting problem.. I think over time, the fuel is slowly seeping out of the fuel pump/carb and causing it to lose prime. After 9 or 10 pulls, it reprimes itself and everything works again. I think Honda would do well to add a manual primer bulb to the engine. In any case, I think I’ll try the starting fluid again next time around, and see if I have better luck.

  • CGenericXSLT channels, parameters, and Local Connection Contexts

    I’m a bit strapped for time today, but I did take a quick look at this, to see if it looks doable. In a nutshell.. I’d like to use a local connection context to do legacy authentication and obtain an encr string to pass to various legacy backed services. This would allow me to create RSS-type channels that link to authenticated services, so I don’t need to use web proxy channels for everything. Initially I’d use it to connect to external services like MAP/DN, but eventually I could actually have the legacy perl code handle the rendering for stuff like registration, and just re-skin it to look like uPortal.

    I started out by seeing if I could pass an “encr” into the RSS and have it display conditionally somehow (we don’t necessarily want it appended to every link in the RSS feed). I came up with the somewhat hackish idea of using the RSS <category> element. If I give the item a category of “myumbcauth”, I can tweak the XSLT to look for that and append extra data to the link. Then, I can pass the actual encrypted string into the XSLT using a stylesheet parameter. This all works fine. The next challenge is getting the portal to set the appropriate parameter in the stylesheet. It looks like all of the channel runtime parameters are also passed in as stylesheet parameters (and in fact I was able to read one of them, baseActionURL), the question is, can I somehow add my own arbitrary param in there? Obviously this would have to be done somehow in the local connection context code. Anyhow, I got as far as that and now I have to run off and fight other fires, so I’ll have to come back to this later.

  • Update on calendar stuff

    I haven’t done much with my calendar stuff recently, for a couple of reasons; one being that work has been crazy and I haven’t had much time to hack on it, and another being that it’s all working without a hitch. I don’t even have to think about the Oracle Calendar stuff; my cron job auto-downloads it several times a day, and all I need to do is fire up iCal, refresh all calendars, and sync to the Palm. I see that Missing Sync version 5.0.3 final has been released, but beta6 is working fine for me so I’m not in a fired-up hurry to upgrade. I’m starting to make more and more use of iCal’s concept of multiple calendars; I already have a calendar with various family birthdays and anniversaries, and I’m going to do one with holidays. Holidays will be a good test of how the Palm handles repeating events, and how well Missing Sync translates repeating events from the iCalendar files to the Palm. I’ve also found myself using PHP iCalendar quite a bit when I don’t have my Mac handy. All in all, I’m really happy with how this whole thing has turned out.

    What’s next? More features, of course. I’d really like to have attendee data for some of my meetings. Because of the performance issues involved with downloading attendees, I’ll have to do two separate downloads (a large range without attendee data, and a small range with attendees) and merge the two together. This could create some problems because of the way I’m munging UIDs for repeating events. Since I’m just appending ascending numbers to create unique UIDs, there’s the chance that the same event could get a different UID depending on the date range that’s being downloaded. I think before I try to do anything fancy, I need to rethink the way I’m assigning UIDs to events. This could present an interesting challenge. Anyhow, more on that later.

    The Holy Grail of this project, of course, would be to have two-way syncing with Oracle Calendar, where changes I make on the Palm or in iCal will get back-propagated to Oracle Calendar. However, this increases the complexity of the project quite a bit, and I doubt I’ll ever go there. I don’t really miss the functionality all that much anyhow.

  • That time of year again.

    Well, it’s time once again to start working on taxes. And once again I find myself using H&R Block’s Taxcut Product. This year, they’ve gone the consumer-friendly route of including the state product with their Taxcut Deluxe package, so you don’t have to purchase it and request a rebate. Good move on their part.

    Once again, the most fun part of tax time is figuring out capital gains on all of our various stock and mutual fund sales. Or more specifically, figuring out the cost basis. And it’s even more fun, when there are splits and spinoffs involved with the stock you’re selling.

    Earlier in 2005 I put together a spreadsheet for each stock and mutual fund holding we own, that identifies each specific lot along with purchase date, amount invested, etc. That allows me to compute accurate cost basis info using either the FIFO method, or by specific identification of lots. It seems to work pretty well, and takes a lot of the tedium out of the process. However, it doesn’t handle the case of multiple purchases on the same day. It shouldn’t be hard to modify the sheet to handle this, but so far I haven’t needed to.

    At any rate, it looks like we’re due for nominal refunds from both state and fed, which is exactly what I shot for when I last filled out a W-4. Time to have another kid, so I can do that all over again!

  • Today’s database tweak..

    Well, one thing our ongoing uPortal launch has illustrated, is that contrary to popular belief, our Oracle database server does not have unlimited resources. To that end, a lot of my recent efforts have been geared towards making our installation more “database friendly”. The centerpiece of this is the connection pooling we set up on Monday. Of course, once you’ve got a nice, manageable connection pooling setup, you want to use it whenever possible. And until today, there was one big piece of the portal that still wasn’t using the pool: the “glue” that interfaces the uPortal web proxy channels to the legacy portal’s authentication scheme. uPortal calls this a local connection context, and ours goes by org.jasig.portal.security.UmbcLegacyLocalConnectionContext. The legacy portal’s session information is all database driven, so this code needs to connect to the database and create a valid legacy portal session for the user, so the web proxy channels will work and the kiddies can see their schedules and drop all their classes. This code was doing an explicit connect to the ‘myumbc’ user in the UMBC instance. Each channel needs to do it, and some of our portal tabs contain several of this type of channel. I’m not sure exactly how many times this code was getting invoked, or how many connections it was generating, etc. because I didn’t do any profiling. But it definitely had an impact.

    Anyhow, I’ve modified the code so that it pulls a connection from the pool (using RDBMServices.getConnection) and uses that instead. I needed to modify the LegacyPortalSession code a bit to support this. Also, since our connection pool uses the ‘uportal’ user (not ‘myumbc’), I needed to get our DBA to do a couple of grants so that ‘uportal’ would have access to the tables it needs.

    For better or for worse, it’s in production now, so we’ll see how it goes.

    The plan for tomorrow: Fix all of the missing or broken links that people have reported. Create a new channel exclusively for DN/MAP. And, look into local connection context usage with CGenericXSLT type channels. I recently discovered that this type of channel can use a local connection context. Depending on how it works, I may be able to use it to eliminate a couple more web proxy channels and replace them with RSS type channels. We’ll see.

  • Legacy myUMBC ACLs as PAGS Groups

    I think I’ve found a way (two ways, actually) to import program ACLs (from the BRCTL.PROG_USER_XREF SIS table) into uPortal as PAGS groups, so that we can publish uPortal channels with the exact same access lists as the respective areas in the legacy myUMBC. This would be a big win, particularly for an app like Degree Navigation/MAP. In the old portal, we control access to DN/MAP using a big, looong list of individual usernames. If the user isn’t on the list, they don’t even see a link to DN/MAP. However, with uPortal, we currently don’t have access to this list, so we have to present the DN/MAP link to a much larger set of users (basically anyone who is faculty or staff), or we’re faced with totally replicating the access list in uPortal, and maintaining two lists. Not what we want.

    Fortunately, we designed the old portal with a bit of forward thinking, and made its ACL mechanism totally database driven. That is, all ACL info is stored in the Oracle database, so some future portal could theoretically extract that data and use it down the road. The challenge, then, is to figure out how to get uPortal to do that.

    uPortal provides a very nice groups manager called PAGS, which allows us to create arbitrary groups based on what uPortal calls Person Attributes. It can extract Person Attributes directly from LDAP, as well as extracting them from the results of an arbitrary RDBM query. It then presents this group of attributes as a seamless collection, regardless of the actual backend datasource for each individual attribute. It’s really very nice.

    My first thought, then, was to just have uPortal query the legacy myUMBC ACL table to get a list of each app a particular user can access, and map the results to “Person Attributes”. I tested this and it works just fine, but there’s one problem: The legacy ACL table is indexed by UMBC username, but the way we have uPortal configured, it’s currently using the LDAP GUID to do its queries. So, to do this the right way (that is, without hacking the uPortal code), we’d need a table that maps the GUID to the username, so that we could do a join against it to get our results. Currently, we don’t have LDAP GUID data anywhere in our Oracle database. Now, I don’t think getting it there would be a huge issue (we’re already doing nightly loads of usernames from LDAP to Oracle), but it still needs to happen before we could use this method.

    The second method would be to import the user’s legacy ACL data into the LDAP database as an additional attribute. Then I could just pull the data directly out of LDAP, without having to worry about an RDBM query at all. This seems like a simpler solution, if it’s possible. More later..

    Note: Configuration of Person Attributes is done in the file /properties/PersonDirs.xml. When specifying an RDBM attributes query, the SQL statement must include a bind variable reference, or the code will crap out. I learned this when I tried to remove the bind variable and hardcode my own username.. no dice. To test this stuff out, subscribe to the “Person Attributes” channel, which is under the “Development” group. Then look for the attributes you defined in the config file. If they’re there, it worked. If not, not.