Data Swarms
By Mike Nygard, Mon Feb 12 00:00:00 GMT 2001

Today, and even more so in the future, your personal data (profile, communications, calendar) resides "out there somewhere," on a remote host. How do we effectively manage our information in this de-centralized model?


When I'm not pontificating in print, I work as a consultant in the Twin Cities. Let me tell you, my PDA is all that keeps me going. I've never had the greatest memory (perhaps due to a bad childhood habit of "bonking" my head?) My office has several dusty Daytimer-style organizers. Each one has a high density of entries at the beginning that decays exponentially toward zero after a month or two. I just can't stay with them.

My PDA, on the other hand, stays with me. I've even used it to take notes while shopping for furniture. (The hidden cost of relying on an external device is that the weak memory I had has now completely atrophied. I'm lucky to remember my own address without looking it up.)

As long as I keep all of my appointments and business cards in the PDA, everything is fine. Recently, however, I was working with three clients at the same time. Two of them were using Exchange. The third uses Netscape Calendar, as does my own company. I spent a lot of time entering meetings in multiple calendars. True, I could have synchronized with one of the four calendars, but that doesn't help with the others. Even setting up conduits to each of the four calendars is not enough.

An attemp to unify


Assuming they all adopt the fledgling SyncML (which is not a given), multi-calendar syncing is subject to race conditions. Between syncs, two calendars can schedule the same block of time. The conflict will only be detected at the next sync. SyncML ensures that the conflicting appointments will both faithfully appear in my PDA's calendar. There really is no way to resolve the conflict at this point. The conflict has to be caught earlier.

The bigger problem with multi-calendar sync is confidentiality. Client A should not see the details of meetings I'm having with Client B! All Client A should see is that the time is unavailable. And I don't really want any of them knowing about my dental appointment. Or the PTA meeting. Or that my family is meeting at a local restaurant this weekend.

I make my entries sufficiently obscure to hide their meaning, but then I probably won't remember what they mean either. I cannot designate visibility by group. Part of the problem here is that I have touch points with many "enterprises:" My employer, my clients, my family, local government, various service businesses, and so on. None of them is an authoritative source of time information about me. None of them can act as my agent when negotiating time and schedule.

For instance, my calendar should know that I take lunch at 11:30 and that I generally work from 9AM to 6PM. It should take other appointments into account, as well as transit time from one location to another. It should also do all of this without violating confidentiality-mine or the clients.

"Enterprise" software is about the enterprise, not about people. It keeps track of information about my relationship with the enterprise. It stores the intersection of the set of information about me and the set of information about the enterprise. In that light, it's no surprise that my company's enterprise directory server doesn't have my dentist's FAX number. That isn't part of the intersection. (Unless I work for a dental clinic, I suppose.)

A possible solution


One way to fix this problem is to invert the relationship. Instead of making my PDA a client of the enterprise calendar server, make the PDA a server. Let the enterprise calendar server contact my PDA to request availability or schedule meetings. This way, the calendar server on the PDA becomes the system of record. Enterprise servers like Exchange now act like brokers, communicating with multiple calendar servers to negotiate time information. I could also set visibility on my entries to show only as much information as I want to reveal. Now the real server is my PDA. It contains the set of information about me. All of the other enterprise servers can intersect their information with what my PDA chooses to reveal to them.

Of course, this has some pretty obvious technical hurdles. The PDA has to receive wireless connections. I don't want to think about what my battery life would be like with an "always-on" connection to my PDA! Somehow, the idea of running a server on my PDA just seems wrong. Cool, but wrong. Maybe when my PDA can remain on continuously for days, and wireless data rates are unlimited, this will be more palatable.

A better approach is to create a calendar server "out there somewhere" that will be my calendar. It would be the only authoritative source of information about my time. All requests for my time would be negotiated with this server. From here, the next logical extension is to call the PDA one channel of many that can access the calendar. Sometimes, a desktop client would be more appropriate. Sometimes, my cell phone would be a better access mechanism.

Simon Phipps, Sun's Chief Software Evangelist, calls this model "swarms". Your digital persona will be represented by a swarm of services on the network. The various devices you use will merely be channels between you and your swarm. At various times, a service in the swarm might contact you through the best available channel. Likewise, you can contact a service in the swarm through whatever device best suits your needs at the time.

Obviously, a host of protocols and implementations have to be updated before swarms can become a reality. The seeds are already here, however. I can contact services for email, an address book, a calendar, storage space, investment, banking, bill payment, and so on. Some of these even allow multi-channel access--HTML and WAP, sometimes voice.

What is lacking thus far is any notion of me. Just as the enterprise calendar server is about serving the enterprise, not the individuals, these services are all about the service provider, not about the service consumer.

Michael Nygard is Chief Scientist at Javelin Solutions, where he analyzes emerging trends in the network economy. His experience runs the gamut, covering scientific, military, financial, educational, banking, and manufacturing applications. Michael is focusing on true integration of wireless devices in the enterprise.