Fruddled Gruntbugglies

Enthralling readers since 2005

Author: lpaulriddle

  • More on VNC

    Well, it’s been a little over a month since I first played around with VNC, and now I wonder how I got along without it. Mostly, I use it to access my Linux box in the basement office. For awhile I grabbed its main X11 display using the XFree86 VNC module. This works OK, but it’s still slow. Then, I tried starting a dedicated VNC server on the Linux box (it opens a new virtual X11 screen on localhost:1), and the performance difference is like night and day. It’s so fast that sometimes I forget that I’m working on a remote desktop. It’s even fast when I access it from the Mac over the wireless.

    The only issue with this setup is access from the Linux desktop itself. I have to open up a VNC client to access the desktop running on the same machine, and I have to remember to start apps that I’ll be accessing remotely (gnucash, etc) inside the VNC server. This seems a little inefficient/redundant, but it’s a worthwhile tradeoff considering the performance is so much better. Plus, I can re-enable direct rendering on the native X desktop now if I want.

  • The hassle^H^H^H^H^H^Hlegacy of PINs at UMBC

    So, we have this new campus portal at UMBC. And for the most part, the launch has gone pretty well. As part of this whole thing, we’re rethinking some of our old, outdated business processes and changing the way they work under the new portal. One of these is the PIN (Personal ID Number). Waaaaay back in 1996 when UMBC first launched web-based course registration, we required all students to enter a 4-digit PIN to log in. There was also a service where students could register by phone, that used the same PIN. Flash forward to 2006. We’re now using a campus-wide single signon system, the telephone registration system is gone, and we’ve done away with PINs as part of the login process. So, since students aren’t using them any more, we can just get rid of PINs altogether, right? Wrong. Problem is, we’re still using our old, crusty HP3000 mainframe as system of record for registration, so when we do online course registration, we have to play by the HP3000’s rules. The HP3000 is still running circa-1996 registration code (written in Cobol), and PINs are so deeply embedded into that code, that there’s no way we’re ever getting rid of them as long as the HP is around. We can rework things so that all the PIN stuff is handled behind-the-scenes, and users never see them or even know they exist, but on the back end, they’re still going to be there.

    Now.. the HP3000 stores everybody’s PIN in a database table. But initially, that table is not populated until a user accesses the system for the first time. Then, the HP figures out an initial PIN for the user, and uses that to populate the PIN table. It then sets a flag that the user’s PIN needs to be changed. The HP will then refuse to do anything on behalf of that user, until they change their PIN. The mandatory PIN change happens when the user logs into myUMBC. With the old version of myUMBC, they would see the mandatory PIN change screen immediately after logging in, at which point they’d need to change the PIN before doing anything else in myUMBC. But again, this behavior is a relic of the days when most activities in myUMBC centered around the HP. It also prevents certain users (people who lack the appropriate data that the HP needs to generate the initial PIN) from using myUMBC at all. If we’re going to move forward, we need to get rid of this behavior.

    The first step towards this goal, was to eliminate the mandatory PIN change check on initial login. Now keep in mind that we have to submit an initial PIN change request for every student, before they can do anything that involves the HP. So, I’m now doing the mandatory PIN change only when the user requests a function that uses the HP. So rather than seeing it when they first log in, they see it when they try to register for the first time. A small but significant step.

    This works great, except it broke the online student parking registration app. Student Parking Registration has the distinction of being the only external (not part of the monolithic legacy myUMBC code) app that talks directly to the HP. If a student goes to this app before changing their PIN, the HP will refuse the parking registration request and the app will fail silently. Yep, this was a fun one to debug. It’s still broken, until I figure out the best way to fix it.

  • Lessons learned about check deposits..

    I have this credit union account, which I keep around solely because there’s an ATM I can walk to from my office. I get the convenience of using the ATM for check deposits and cash withdrawals, and in return, the credit union gets to keep a nominal amount of my money and pay me practically zero interest on it. Any balance over and above the aforementioned nominal amount, gets transferred into my brokerage account (where it earns much better interest) as soon as the check clears. In general, this works out pretty well. But, there are certain rules which must be followed, which brings us to today’s tale of woe.

    Monday, January 23, 2006: I went to the ATM and deposited a check for a few thousand dollars. As usual, the ATM receipt showed that $100 of the deposit was available immediately, and the rest was on hold until the check clears. Now, ever since the new Check 21 system was implemented, the credit union has been clearing my checks much more quickly, a lot of times by the next business day. Sure enough…
    Tuesday, January 24, 2006: I checked my account balance online at the credit union, and it showed that the check had cleared and the entire amount was available for withdrawal. Going by this, I scheduled three transfers into various different brokerage accounts, to be completed on Thursday the 26th.
    Wednesday, January 25, 2006: I checked the credit union again. Oops.. seems that my funds were mysteriously no longer available! The same day I get a snail-mail letter (handwritten, btw) stating that a “special hold” had been placed on the deposit due to the large amount. The hold was for 4 business days, meaning the funds wouldn’t be available until the 27th. It’s too late to cancel my transfers, because they’ve already gone to ACH for processing. Bummer.
    Thursday, January 26, 2006: The transfers all get bounced back, and I get charged $81 in overdraft fees.
    Friday, January 27, 2006: The funds become available at the credit union, and I start making phone calls to get this mess straightened out.
    Monday, January 30, 2006: After two calls and about 30 minutes on hold with the credit union, my $81 is refunded. Thanks guys.
    Today: I reschedule the transfers to the brokerage accounts and cross my fingers.

    So, what have I learned? #1, there’s something to be said for going to the bank and making the deposit with a teller, because they probably would have told me about the special hold then and there. #2, when making a large deposit at an ATM, don’t touch the money for a week, and don’t trust the online banking system to give accurate info WRT funds availability during that period.

    Can I scream now?

  • Affinities affinities affinities…

    As expected, the whole uPortal affinity thing is really drawing folks out of the woodwork. For those of you who came in late, uPortal uses a totally different scheme from the old portal to determine who sees what content. Let’s take the “Faculty Options” channel as an example:

    Portal You see faculty options if…
    old myUMBC You are in AUTH_CLASS or TEACHER SIS tables, or have faculty LDAP affiliation
    uPortal You have faculty, staff or instructor LDAP affiliation

    As I said… totally different. Now.. most folks who need faculty options will satisfy both portals’ conditions, and will see the content in both portals. However, there are always special cases, and as expected, we’ve run into a few. First, not everyone in AUTH_CLASS has one of the three magic LDAP attributes. Case in point, the education department has graduate assistants that do course authorizations for them. These people need to see the faculty options, but don’t get them in uPortal.

    What we really need to do here, is rethink how we’re doing some of these affinities and who’s seeing what content. Some of this will just involve creating new PAGS groups to correspond with various affiliations. But, we may also need to create additional affiliations in LDAP to cover certain cases. Membership in AUTH_CLASS would be one of those cases. uPortal allows me to specify Person Attributes based on DB queries, so for some of these I might be able to go that route. But I’d prefer to keep this totally LDAP based.

    In the meantime, I’m making things work for these people by granting them temporary staff affiliations in LDAP. Can’t have them unable to do their jobs while we’re figuring this out.

  • Sharing address books

    Now that I’ve got my calendar stuff all figured out, it’d be nice to do the same with my address book and contacts. Right now, the “reference” version of my address book is stored in the Mac Address Book application. From there, I use Missing Sync to synchronize it with my Palm, and Apple’s iSync to send it to my bluetooth-enabled cell phone. This all works pretty nicely. The missing piece is making the email directory available to my mail client, Thunderbird. Now, this would be relatively easy if I only ran Thunderbird on my Mac. There are a number of tools out there that will take exported Mac Address Book data and import it into Thunderbird on the Mac. The problem is, I also run Thunderbird on various different Linux desktops at work and at home. It’d be great to have the same address book available on every mail client I use.

    The other day I ran across this site which describes how to build a shared address book using a private LDAP server. This is a great idea that hadn’t occurred to me. Thunderbird has built-in Support for accessing remote address books via LDAP. If I use OpenLDAP (which is conveniently available as a Debian package), I can even run slurpd and keep replicated servers at home and work. This should be fun to try out.

    Once the server is in place, the issue then becomes getting the Mac Address Book to synchronize with the LDAP server. That is probably going to be the biggest challenge of the project.

    The drawback to this approach is that there’s no way to automatically update the LDAP database from Thunderbird. It essentially becomes a read-only data source. However, there are a few free web-based LDAP admin tools floating around, so I could probably adapt one of those to suit my needs. It’s not a perfect solution, but it might work.

  • Mortgage escrow analysis demystified

    Just found out what my mortgage payment is going to be for the next year, and the escrow amount jumped a bit more than I had expected. So, being the accounting buff that I am, I decided to try and figure out the formula the mortgage company is using to determine the monthly escrow amount. In the past, this has always seemed like some kind of arcane art shrouded in mystery. I would get my escrow analysis statement in the mail, give it a blank stare or two, note whether I was getting a refund check, write down the new mortgage payment, and move on. Well, it turns out that it’s actually pretty simple:

    1. Take the total of last year’s disbursements and divide by 12, to get the base monthly escrow amount.
    2. Multiply by two to get the minimum escrow balance required (two months worth).
    3. Starting with the escrow balance on the anniversary date, figure out the projected escrow balance for each of the next 12 months. To figure the balance for a given month, take the balance for the previous month, add the base monthly amount (from step 1) and then subtract any disbursements that were made in the same month last year.
    4. Find the month where the projected escrow balance is the lowest. If that number is less than the minimum balance from step 2, subtract the two to get the shortage. Most mortgage companies will let you either pay this amount as a lump sum, or spread it out over the next 12 months. In the latter case, divide the shortage by 12 and add the result to the base escrow amount (step 1), to get the final escrow amount.

    I broke out my favorite spreadsheet and crunched the numbers for my own mortgage, and came up with exactly the same numbers as the mortgage company. So at least I know they’re not cheating me. From analyzing this method, it seems that it is definitely in one’s advantage to have most of the disbursements happen towards the end of the “mortgage year”. If the payments are skewed towards the beginning of the period, you end up with a higher payment (to satisfy the minimum balance requirement) and a larger balance at the end. If you end up with an overage (projected low balance greater than minimum required balance), the mortgage company is required to refund this to you, but you’re still giving them an interest-free loan (at least in most states).

    Then, there’s the question of whether to pay the escrow shortage as a lump sum, or spread it out over 12 months. The answer is pretty simple: crunch the numbers both ways, figure out which method would result in the lowest average monthly escrow account balance, and go with that. In my case, paying the shortage over 12 months was the clear winner.

    The nice thing about the escrow analysis process is that it’s very predictable, everything’s based on the disbursements made in the previous year (there’s no hand-waving or estimating of future bills going on). In my case, as soon as I get my July property tax bill, I can figure out next year’s mortgage payment. I’d just as soon not have an escrow account at all (I’m perfectly capable of earmarking the money myself and earning interest on it to boot), but at least I understand the process now.

  • Bluetooth and iSync

    Bluetooth is a really neat technology. It’s got a lot of potential, but there are still a lot of issues with it. The pie-in-the-sky dream is that we will have all these different Bluetooth gadgets that all interoperate perfectly. The reality, of course, is not as rosy. There are lots of devices that speak Bluetooth, but for various reasons I won’t go into here, they don’t all work with each other, at least not without lots of fiddling. So, when something does work perfectly, it’s noteworthy. A little sad, but true.

    This week I got a new Motorola e815 phone for work. One thing I’ve always dreadded about getting a new cell phone, is trying to get all of my stored contacts from my old phone into the new phone. It’s always been a tedious, manual process. Anyhow, the e815 supports Bluetooth. My Powerbook has an app called iSync, which purports to sync info from iCal/Address Book over Bluetooth to various models of phones. Fired everything up, went to Bluetooth Options on the Mac, paired it with the new phone, ran iSync, and my entire address book was transferred to the phone. The whole process took less than 10 minutes.

    This is the way this kind of technology is supposed to work. I wish everything else was that easy. I initially tried to pair the phone with my Palm Tungsten E2, but I haven’t been able to get that to work. I’ve heard that it’s possible to get the phone to act as a modem so that you can surf the net on the Palm. However, if it doesn’t work out of the box, I’m not going to bother. It’s not a feature I’d use a whole lot, and I don’t want to waste a lot of time fiddling with it, especially if there’s a chance I won’t be successful. I’m just happy that I can transfer my address book to the phone. It’s always bugged me that I’ve got this great phone book on my Mac and Palm, but not on my cell phone, where I am actually making the calls.

    Now, if I could just export the Mac address book to Thunderbird (on the Mac and all my various Linux boxes), I’ll be even happier…

  • Stalking the infinite loop

    Today began with an effort to track down the infinite loop which is causing the java process to munch CPU cycles on the portal server. The problem seems to be spreading.. when I checked this morning around 6:30am, there were 4 looping threads on uportal1 and 5 on uportal2. I’ve tracked down the offending loop, and I started out by putting a counter into it and logging the number of times the loop ran after each successful completion. This gave me a pretty good idea of how many times we should go through the loop under normal circumstances (looks like only 2 or 3). Then I picked an arbitrary large number, 1000, and changed the code so it throws an exception if the counter exceeds this value. I’m hoping this will have two effects: one, stop the looping; and two, provide some logging so we can further investigate what is causing the problem. The new code is up and running, so we’ll see how it goes.

    Welp, it worked. Interesting… very similar to yesterday, everything was quiet all morning and then both instances hit the infinite loop almost exactly at 1pm. Now, instead of an endlessly looping thread, I get a nice error log and stack dump. Next thing to do is try to log some additional info, to see if I can narrow this down to a particular user, activity, or whatever. If this is only affecting certain user(s), maybe someone will call the help desk and help me solve the mystery.

    Other than that, the biggest issues so far have been related to permissions and affinities. Lots of people complaining that they don’t see content they used to get on the old portal. I expected this, because the uPortal groups/permissions model is quite different from what we were using with the old myUMBC. For now, I’m noting the users who are having problems, and in a week or so I’ll call a meeting to discuss how to reconcile them. In the meantime, these users can continue to use the old portal.

    I also discovered today that the new portal does not work for anyone who has a “mandatory PIN change” flag set in SIS. Now first off… PINs are going away. There’s nowhere in myUMBC where a user is required to enter a PIN any more (well, there’s orientation.. but don’t go there). Given that, I made the executive decision that mandatory PIN changes are going to go away in uPortal, and tweaked the legacy code accordingly. However, it looks like the HP is checking the forced_pin_change field and disallowing registration if it is set. So it looks like I need to take this one step further, and actually submit a PIN change for these users behind the scenes. Looking into that now. Regardless, we’re definitely going to need to test how the portal behaves with “virgin” users, before the fall semester starts.

    My God, I’m going through my own code that does the HP PIN stuff, and it is so bloody convoluted I want to shoot myself. From following the code, I can’t see any way for it to ever get to the HP PIN verify when accessed normally. I think what we need is a rewrite that takes all of the PIN stuff out of the normal login process, and doesn’t do anything with PINs until the user does something that accesses the HP. Then, if the user doesn’t have a PIN it can have the HP create one, then send a PIN change request to the HP so it clears out the forced_pin_change flag. Will look at that tomorrow..

  • Relaunch day

    First “real” test (i.e. University open for business, people hitting portal) for our re-launch of uPortal today. It went up Sunday.

    Issue #1: I see that we still have the problem of the JVM going up to 100% CPU utilization occasionally. It was like that this morning when I signed on around 7:30am. Portal was still responsive. The problem went away when I bounced the Tomcat instance. I guess somewhere, a thread is going haywire or something. I learned how to get a thread dump under Tomcat: Send SIGQUIT to the JVM process, and Tomcat puts the thread dump in catalina.out. Next time it happens, I’ll see if this produces anything useful.

    Well, the issue cropped up again, and I did a thread dump. It appears we’re getting hit with UP-1175. Somewhere there’s a corrupted layout with a circular reference, which is causing an infinite loop. They’ve fixed it for uPortal 2.5.x but there appears to be no fix for 2.4.x. Need to look into this a little further.. Other than that, things have gone pretty well so far today. Fingers crossed.

  • The never-ending basement plumbing project drags on

    I finally got back to working on my endless plumbing project today.

    A couple weeks back, I pressure tested the new branch I ran for the outside sillcock. To do this, I soldered a female adapter onto a 12″ length of pipe, screwed a quick-connect air coupling into it, and attached the other end to my branch with a compression fitting. Then I hooked it up to the compressor, cranked it up to around 30psi, and let it sit. There was a v-e-r-y slow loss of pressure, maybe 1psi or so over several hours. This was a little troubling, but it didn’t necessarily mean there was a leak (it could be the stop valve packing, the compression fitting, or even the regulator gauge). It did make me fairly confident I didn’t have any “gushers” or “blowout” type leaks.

    This morning I got the idea to use a couple compression fittings and hook the branch up to my existing plumbing. That way I can be absolutely sure that the branch holds water, and when I’m ready to make the final repair, I won’t have to worry about the branch leaking. So, that’s what I did today. Cut a short piece of copper to fit between the existing plumbing and the new branch, shut off the water, drained the plumbing, and hooked the whole mess together. When I turned the water back on, I found my pressure loss pretty quickly: it was a slow drip at the stop valve, where the valve “guts” screw into the valve body. I had taken the valve apart to sweat it, and it just needed to be tightened a bit. I also had to tighten one of the compression joints. Other than that, everything looks good, and I can finally use the sillcock I put in 6 months ago (just in time for the dead of winter. But hey — it’s frost-free)!

    With that, the only thing left is to make the original repair, which was the driving force behind this entire project. Maybe I’ll get around to that by next January or so.