iCal is the calendaring app that Apple includes with MacOS X. It comes bundled with another app, iSync, which handles synchronization with the Palm. On MacOS 10.4 (Tiger), iCal’s app data lives under $HOME/Library/Application Support/iCal. It appears that the calendar data itself is stored in iCal format (.ics), while the metadata is stored in XML format. Separate calendar categories are stored in separate files, unlike the single monolithic binary file Palm Desktop uses. All data files appear to be flat ASCII. That means that, if I wanted to, I could easily maniuplate the files using Perl or somesuch. However, not knowing the interrelationships between the various files, I feel like I should probably avoid this. So, let’s look at the app and see what kind of import features it offers.

iCal calls calendar categories “Calendars”. When I go to import a vCal file, it asks me to select a destination calendar. I can pick an existing calendar, or tell it to create a new one. In the latter case, iCal creates a calendar with the same name as the vCal file (minus extension). It appears to ignore the category data in the vCal file, which for my purposes is good.

To delete a calendar, I just select it in the left pane and press the “Delete” key. It warns me that all entries under that calendar will be deleted, then does it. Everything is undoable.

This seems to work great. The import process is smooth. It creates a new calendar category for me automatically. Deleting the data (prior to a subsequent import) requires only one keystroke. It appears to import locations and attendees too. The only drawback is the handling of “daily notes” and “day events” from Oracle Calendar. However, this problem is on the export side. Ideally I’d like these to come in as “all day” events rather than 12:00am-12:01am meetings. If I can’t configure things this way on the export side, I should be able to write a perl script to modify the vCal import file.

iCal also includes sharing features. I can subscribe to a calendar on the internet, and also publish my calendar to a WebDAV server. I haven’t researched this much, but I hopefully I can use this to access my calendar from anywhere on the net (read: my desktop machine at work).

Palm Desktop is, of course, the app that Palm produces to do desktop integration with its handhelds. All in all it’s not a bad little app suite, and you certainly can’t beat it for integration with the Palm. It supports all of the Palm’s features and synchronization (via the bundled HotSync manager) is flawless.

Palm Desktop is not server based, so all of your data lives on the machine where you have it installed (which, by extension, is the same place you synchronize the handheld). There doesn’t appear to be a way to share the calendar data on the internet. That means I’ll need either my laptop or my Palm handy to view my calendar. A drawback, but not a show stopper.

On the Mac, the Palm Desktop calendar data appears to live in a single file, $HOME/Documents/Palm/Users/Paul Riddle/User Data. This file contains binary data presumably in some database format that Palm Desktop can read. The ‘file’ utility reports it as a “DBase 3 data file”. If I run ‘strings’ on it I can see entries from both my calendar and my address book. This doesn’t look like something I want to muck with outside of the Palm Desktop app. So, we need to see if Palm Desktop can do bulk deletion of imported entries.

Looks like we’re out of luck with the bulk deletion. In an older version of Palm Desktop (on my Windows machine), it works. I can create a custom category, create some entries within that category, then delete the category. The software gives me the option to delete all entries within the category, or move them to the “Unfiled” category. However, this doesn’t work with the newer version of Palm Desktop on the Mac. It looks like they’ve added the ability to have multiple categories assigned to an event. If I delete a category, the events in that category stick around, even if no other categories are assigned to the event in question. Kind of sucks that they would cripple that functionality. But it pretty much means that I can’t use Palm Desktop for what I want to do. Nevertheless, I’ll continue to evaluate it a bit and see what I find.

Palm Desktop also seems to have problems with “day events” and “daily notes” under Oracle Calendar. These events are actually untimed, so they should be imported as “untimed” (or “all day”) events. However, Palm Desktop imports them as timed events and sets the start/end times to 12:00am and 12:01am. I’m not sure if this is the fault of Palm Desktop or the Oracle Calendar export.

OK, the “day event” handling appears to be a “feature” of the Oracle Calendar export. The 12:00am-12:01am time range is in the vCal import file, so Palm Desktop is just recording it as it sees it. The vCal file also includes Category information, and Palm Desktop is reading that and creating new categories as it sees them. The Oracle Calendar entries show up with categories of “APPOINTMENT”, “HOLIDAY”, “MISCELLANEOUS”, etc. If a bulk delete were possible, I’d need to delete all of these categories before re-importing; a bit of a pain. Alternatively, I could massage the vCal import file to put everything into a single category.

To summarize: Palm Desktop uses a (possibly proprietary) binary file format to store entries on the host computer. There’s no out-of-box way to share calendars among multiple computers (although apparently there are third-party products that will do this). Bulk deletion of entries does not appear to be easily possible. Even if it was possible, the import process would require multiple steps to recategorize imported entries into a single category.

Given all this, it looks like Palm Desktop is not going to be the answer for me. A shame, because I’m sure I won’t find anything else as tightly integrated with the Palm. Next up, I’ll check out iCal.

Earlier I mentioned iCal and Palm Desktop as two possible solutions for my desktop calendar. To integrate with Oracle Calendar, both of these apps would require me to manually import the Oracle Calendar data in vCal format. Oracle Calendar’s export function gives me several options for choosing what data gets exported. I can export entries within a specific date range (from 1/31/2004 through 12/31/2006 for example), I can export a relative date range (1 year in the past through 2 years in the future), or I can export the entire enchilada (data that dates back to early 1999, in my case). In the past, I’ve gone with a relative date range starting 1 year in the past and going 2 years in the future. Reason being, exporting the whole thing is kind of slow and there’s no bounds on the size of the data set being exported, so it will keep getting slower and slower the more data I accumulate in Oracle Calendar.

Now, with a 3-year date window, it’s a given that I’ll be exporting a lot of the same data over and over, and importing the duplicate data into iCal or Palm Desktop. Neither of these apps has what I would call an “intelligent” import feature. That is, if I import data more than once, it will show up multiple times in the application. This isn’t a show stopper, but duplicates are undesirable and we need a way to prevent this from happening.

One way to do this would be to just import the data, then cobble something up (in Perl, perhaps) to strip out the duplicate entries. This would require the ability to manipulate the calendar app’s data outside of the calendar app itself, so presumably it would rely on the app using some kind of standard, documented format to store its data. My surfing reveals that someone has done this with iCal, so this may be possible if iCal is my solution. Dunno about Palm Desktop, but intend to find out.

Another way to do it would be to simply delete all of the previously imported data out of the calendar, than re-import. If I went this route, I would need a way to differentiate imported data from other data inside the calendar, so I can delete just the imported data and nothing else. Then, I’d need the ability to do a bulk delete, ether from within the app’s GUI, or (once again) by manipulating the app’s data with an external script.

Both of these methods will require a relatively robust Palm sync app, so that deleted entries are picked up and properly deleted off the Palm. Otherwise we’ll still end up with duplicate entries on the Palm.

Next I’ll look at Palm Desktop and iCal more closely, to see if either will work as part of my evil plan.

Since I need to interoperate with Oracle Calendar, and I want to sync with my Mac, and Oracle’s Mac Calendar sync client will not work with my E2, it follows that I need to come up with a way to get the Oracle Calendar data onto the Palm without using the sync client.

The Oracle Calendar client includes an “Export” feature which will export data into a vCal file. I can export data for any specified date range, then import this data into any app that supports vCal. Palm Desktop supports this, as does Apple’s iCal app that comes bundled with MacOS X. So, a simple solution would be to just export the Oracle Calendar data regularly, import it into one of those apps, and then sync the data with the Palm.

The obvious drawback to this is that it’s a one-way operation; this will pull data out of Oracle Calendar, but it will not take new entries created on the Palm and back-populate them into Oracle Calendar. If I specifically need something (a meeting for work, etc.) to show up in Oracle Calendar, I’ll need to use the Oracle Calendar client to create it. I can live with this, but it will mean changing my calendar habits, using Oracle Calendar only for work stuff like meetings etc. and using Palm Desktop or iCal as my “main” calendar. I won’t be able to use the Palm to schedule group meetings at work, but I never used it for that anyhow, because the sync client doesn’t support the concept of multiple attendees. So, no great loss there.

Another issue is the export/import process itself. It’s cumbersome and time consuming, but certainly doable in the immediate term. Oracle does provide an API which I could potentially use to automate things on the export side. I’ve managed to track this API down. It’s part of the Supplemental package supplied with the Oracle Collaboration Suite product. Oracle offers a free download. They provide a C++ interface and a Java/Web Services interface. However it appears they are phasing out the C++ interface. The current stuff from Oracle only has C++ libraries for the Mac (happily, that’s what I’d be using). It’s also unclear if either of these interfaces will work with UMBC’s slightly-out-of-date version of the calendar server.

I may give this a shot at some point, but it will take some time (I will also need to get up to speed with developing stuff on the Mac). In the meantime, I can do it manually.

Here is a link to some sample code that uses the C++ API to export Oracle Calendar data.

There’s also a package out there called OraCal2iCal which purports to automatically download Oracle Calendar data into Apple’s iCal app. From reading about it, it sounds like it works by actually driving the GUIs for the Oracle Calendar client and iCal to do the work for you. A cute idea, but a bit hackish for my tastes. Still, it might be worth a look at some point, particularly if I’m not able to work out a solution that uses Oracle’s API.

Let’s make a laundry list of stuff I want to be able to do with my calendar.

1. View and update calendar from my Palm, my laptop, and all my desktop machines, be they Linux, XP or OSX based.
2. View and update calendar from the web.
3. Interoperate somehow with Oracle Calendar, because that’s where all my work stuff lives.
4. Work with the Palm’s PIM apps without crippling features like “To Do” list categories, etc.
5. Avoid snafus inherent to calendar synchronization, i.e. duplication of events
6. Work with a minimum of hassle
7. Allows me to sync the Palm on my Mac.

That’s a tall order, and I don’t think I’ll be able to find a solution that does all of this perfectly. The challenge, then, is to find the best compromise of functionality, reliability and convenience.

So, Oracle Calendar sync worked more-or-less OK for me under Windows with my old Palm. So let’s try it out on the Mac, with the E2. Should work fine, right? Wrong. Everything installed correctly. Configured stuff, without a hitch. Attempted to hotsync. Barfage. Some sort of “database full” error or somesuch. Google. Turns out the MacOS Oracle Calendar sync stuff won’t work with the E2’s flash memory. Oracle promises an update “real soon now”. ETA on the update is second quarter 2005. It’s now November 2005. Guess I shouldn’t hold my breath, huh?

I subsequently spent a couple hours in broken-conduit-reinstall-palm-desktop hell trying to eradicate the non-functional Oracle Calendar conduits from the Mac. For future reference: After clean install of Palm Desktop, go to /Library/Application Support/Palm HotSync/Conduits. Back up all conduits. Restore them by copying the backups back into this directory. If HotSync complains about “duplicate conduits”, follow the prompts and HotSync again, repeat as necessary until the problem goes away. Once I figured that out, I could restore my conduits whenever I wanted to without having to reinstall Palm Desktop.

So it looks like I’m going to have to actually switch on the brain here and figure out a new calendaring solution. Or, I could wait for Oracle to release a Mac Sync client that works with the E2. In the meantime I could use my Windows box to sync. But, I don’t want to sync with my Windows box, I want to use my laptop. And truth be told, if I can find an acceptable solution that doesn’t rely on the Oracle calendar sync client, I’d be more than happy. Seems like I need to step back, decide just what it is I want to accomplish, and then figure out the best way to do it.

OK, so I’ve got this shiny new Palm and shiny new Powerbook. Right now my entire life schedule lives on UMBC’s Oracle Calendar server (barf). In a perfect world, I would like to be able to view and interact with the same calendar on my Palm, my Mac, my Linux desktop at work, my wife’s doze box at home, etc. etc. So, how do I do it?

Oracle Calendar, obfuscated beast that it is, actually handles this task pretty well for everything except the Palm, as long as I’m connected to the internet. That’s because everything in Oracle Calendar lives on a central server at UMBC, and I can pull the info up with an Oracle Calendar client, and there are clients available for every desktop OS I use (XP, OSX and Linux). There’s even a web interface for places where I don’t have access to the calendar client. All in all, it works pretty well despite being a big, archaic, bloated, closed-source monster. And I’m stuck with using it in some capacity anyhow, as long as it’s UMBC’s enterprise calendar of choice, which figures to be the case for the next couple years at least.

Enter…… the Palm. This is my first Palm in a year or so (when I punted my old hand-me-down M105). I used to synchronize the old M105 with Oracle Calendar using Oracle’s suppled Palm-sync client on a Windows box at work. That worked pretty well, with one major annoyance: It screwed up my “To Do” list categories on a regular basis. But, it provided me with a synchronized, up-to-date calendar on my Palm, so I put up with it.

I hated the M105, though, because the battery gauge was inaccurate and I was constantly losing data when the batteries ran down unexpectedly. I think it had a bad capacitor too. When I pulled the batteries out, it instantly erased all my data. All in all, it was a bloody pain.

The new Palm is a Tungsten E2. It has flash memory. It supposedly won’t lose its mind if the battery runs down. Which is going to take me awhile to test, because the battery seems to last forever. But anyhow, it seems pretty cool so far. Honestly I wouldn’t have bought another Palm if they hadn’t done something about the memory-loss issue. Of course, as I soon learned, nothing is perfect……..