OSGi from Here to There

In the last installment I talked about programs built on the OSGi runtime, how they consist of discrete bundles of code and resources loosely coupled with each other, using a service registry to communicate, much the way discrete applications work together in an SOA environment. I tried to draw a picture of a Java application not as a main class entry point into a soup of jar files and classes all piled onto a single tower known as the class path, but rather as a set of these OSGi bundles, each declaring their own dependencies and exports, with a runtime that wires everything up as a network to satisfy all the dependencies. I tried to describe how applications broken up this way are actually easier to understand and manage. You can look at any individual Java package and make strong assertions about who has access to it and how, and understand what will break if it is modified or removed.

If my picture wasn’t very clear, or you still don’t fully grasp the motivation for this, I’ll describe a specific example that many Java developers should relate to. But before I do, it might be a good idea to make a more detailed presentation of OSGi and Equinox, the implementation used by Eclipse. Rather than take up space here though, I think I’ll just include some references to articles which I think give the best overview:

This presentation by Jeff McAffer covers some OSGi basics as well as some of the really interesting areas being explored, such as dynamic provisioning of bundles (think of Java Webstart) and API tooling.

• 50 minute presentation by some Equinox implementers introducing OSGi and Equinox. Sample Code available for download.
• Equinox Documentation Page, pretty much all you need.
• Peter Kriens’ OSGi blog (thanks Abhijat!). This covers OSGi in more detail as well as a lot of the politics going on around competing component models.

On a recent project I was working on we ran into a sort of crisis. We had reached a point where development was no longer scaling. Incremental changes to the architecture required a tedious littany of changes to configuration files, build files, test scripts, library paths, IDE configurations, and installer scripts. These changes all had to be carefully coordinated and missed steps often didn’t show up until further downstream; something as simple as adding a new class could result in a ClassNotFoundError in testing if one of the many dependent configurations had not been updated with the new class. So much effort and diligence was required for even small changes that we could not keep up with the kinds of improvements necessary to remain competitive.

We decided to migrate our application runtime to the OSGi Equinox runtime. As it turned out our motivation had nothing to do with the issues above but with other business issues. What we discovered was that after making the investment in structuring our application into bundles, we were able to breathe new life into it. We could again make big changes to the “plumbing” with much less effort and could make stronger assertions about the different parts of the application–what could be safely refactored and what had fragile dependencies.

The improvements included but were not limited to:

• Going from 25,000 lines of ant code required for a full build down to about 200 lines of boilerplate configuration, plus about 200 lines of custom callbacks,
• We eliminated about seventy class file catalogs used to ensure extra classes were not inadvertently shipped in the wrong jar file or duplicated unnecessarily,
• Reducing the size of application distros by eliminating unused dependencies
• Surfacing previously unknown bugs based on dangling references to missing classes and libraries,
• Eliminating a large body of code devoted to managing extensions with segregated class spaces using custom class loaders.
• Going from managing four different runtime configurations for each application–the IDE classpaths, the IDE launchers, the runtime script classpaths, the build script classpaths–down to a single feature descriptor listing the OSGi bundles comprised by each application.

And probably the biggest improvement was in the introduction of the extension mechanism provided in Equinox to extend applications with a sort of dependency injection. This allowed us a new lease on life as a platform for custom configuration and specialized implementations instead of a product trying to be all things to all customers.

The investment required wasn’t trivial but it turned out to be much easier and simpler than I expected. I worked on the conversion myself over a period of weeks. We were able to pull the trigger on the new configuration after about two weeks (implemented over a slow holiday period) and have it fully baked with about a four week effort spread over time. Much of this effort consisted on eliminating the old build system, verifying that everything we did before was being done the same way using the new build system. Developers didn’t really have much downtime but it was very disruptive for them. I had to send out emails every few days or few weeks with instructions on how to do things differently. Most notes were joyous announcements that “you no longer have to” do some tedious routine task that you had to do before. For instance, developers no longer had to kick off an ant build before testing an app. They could launch it directly. They no longer had to update build scripts. We no longer had to wait for the CI server to inform us that a build broke because a dependency was missing. Now Eclipse would show us the missing dependencies. In fact, it did this before but because the configuration of the IDE was separate from the build it wouldn’t really mean anything.

To this day team members still come up to me occasionally to thank me for introducing OSGi, often after being reminded what things were like by having to go back to an old release build. I wish I could take credit but the truth is I never anticipated most of these benefits until I started the conversion.

Now clearly most of the benefit derived had to do with the painfully wrongheaded buildsystem we already had in place by the previous generation of developers. In their defense, it was developed when the only real build tool available was an early version of ant, before they were even using a modern refactoring IDE. And there are a lot of good choices for modern build tools which solve some of these problems (Maven and Buckminster are two that I would look at). But OSGi isn’t really a buildsystem or a configuration management system. It’s simply a runtime that enables solutions to these problems with a much more coherent, consistent and simplified approach.

Our buildsystem, our unit test runner, the IDE, the extensible platform, the launchers, the branding, dependency management–all of these mechanisms are now based on the same set of build artifacts, the Equinox configuration files (bundle manifests, feature and product descriptors), files which are managed with a very nice GUI.

I was expecting to start this entry in early December but I’m learning it’s hard to get a few hours of quiet time at the computer now with the baby and two year old. I’m adapting slowly! I hope to continue the OSGi discussion with the actual procedure for converting applications to OSGi and how the PDE headless build works.

Share and Enjoy