So what does it take to migrate a large, multi-tier product to the OSGi Equinox runtime?
We started out with a large, over-frameworked set of applications built with a creaky buildsystem cobbled together from perl scripts and ant build files. We had all the code crammed into two projects, one for core libraries and theother for application code. The applications were all part of a stack that had a lot of shared libraries and resources and the only thing that kept parts separated were the build scripts which extracted selected packages from the projects and built jars out of them. The jars would then be combined into the classpath for the corresponding application. So there was some attention to keeping components decoupled but nothing that actually enforced dependency management other than the build files. You couldn’t see any real structure emerge by browsing the two projects but after you ran the build you could browse the dozen or so different jar files in the targets directory to identify the common components, the application modules, client libraries for server communication, and the shared open source libraries.
Don’t bother reading that last paragraph again. It’s not worth it. It’s enough to know that things were a mess. We were a long way from a system based on a component architecture with every functional component of the system assigned to a standalone project with it’s dependencies and exports plainly elaborated in the IDE. And even if we did manage to completely reorganize everything, how could it possibly build? Our buildsystem would be useless, and at that moment there were about 25,000 lines of it to contend with.I spent about three weeks doing nothing but analyzing the situation. I didn’t change anything but played around with Equinox. I hacked our jarfiles to make them look like bundles. I wrote some PDE builders to see how they worked. I considered all possible options for breaking apart our two mammoth projects and putting everything back together again in. I read documentation, wikis, blogs and watched a number of EclipseCon presentations. I was already halfway through my alloted time and I had not made one change. Finally I decided to take the plunge. I sent out e-mails to the dev team and told them for the next few weeks (which included the Christmas Holiday) I would be moving code around, breaking the build often, and changing the way they did everything. Then I went to work on a sort of recursive process to break apart the projects and re-constitute them as OSGi bundles.
Here’s how it went:
It’s not hard to make any Java application run in an OSGi runtime. All you have to do is zip everything up into a single file–class files, jar files, resources–and add the appropriate manifest headers. You need to write a few lines of java that implements a bundle Activator by invoking your main() routine. Then you start up your OSGi runtime (Equinox) and have the configuration reference your zip file, which is now a bundle. No application code needs to be changed.
So that was my starting point. I created four different projects for the different applications that made up our product. These projects are OSGi Bundles. I went back to my build output folder from the old ant build and grabbed the jar files needed for each of the applications and copied them into my new bundles. Then I created an Activator class for each one that just called the main() method for the application. I was stunned when it actually worked. It only took a little while but of course I wasn’t actually compiling code. It was just a re-packaging of the binaries.
To get to the point of having all the source in OSGi bundles I was going to have to break down the big bundles into smaller ones and move the source over one at a time. This turned out to be an iterative process that looked something like this:
- Pick one of the bundle projects I had just created.
- Go through the jar files inside of it one at a time.For each one, I would create a new bundle corresponding only to that jar file. Let me useutilities.jar as an example. The bundle I created was calledcom.acme.utilities.
- I would take all the classes that appeared in utilities.jar, find the java source files in the old projects and move them into my newcom.acme.utilities bundle source folder.
- Once I had that working, I would go back through all the other bundles I had already created and find any others that were using the utilities.jar. In those bundles I would delete thejar file and add the dependency on com.acme.utilities.
- Now my new bundle project had no more jar libraries in it. It also had no source code. I had broken it up into a bunch of bundles likecom.acme.utilities so now I could just delete it.
The trick was to work up the food chain starting from the bottom. Pick out the jar files which did not depend on anything else that wasn’t already a bundle. I started with 3rd party libraries and simply converted those jar files into bundles. It only takes a minute to do this in Eclipse or with the excellent BND utility from Peter Kriens. You add the 3rd party bundles to what’s known in Eclipse as the target platform for your application.
After this phase, I had gone from two huge projects to a dozen or so smaller ones, each more clearly identified with it’s function. Instead of one giant “libs” project I had com.acme.utilites.core, com.acme.utilities.swing, andcom.acme.utilities.html, etc.
Along the way I discovered problems in our application, like circular dependencies. These weren’t necessarily bugs but were isolated cases where someone had specifically broken a rule like accessing some view code from the model. The IDE was catching problems like this.
One lesson I learned was to let go of our classloaders. We were using classloaders in a number of places to try to scope visibility of jar files or look up classes dynamically by name. OSGi now managed all of the issues we were trying to solve with classloaders. Invariably when I encountered places in the code that were passing classloaders as arguments I could pretty much just remove them completely. OSGi gives you the proper class visibility no matter where you are.When it was all finished I could look at our code and see real structure. Bundles represented groups of related classes that often corresponded to some layer in the application, like persistence, ui, domain code, messaging, utilities, etc. Not only could I see the dependencies but I could make strong assertions about them because they were actually being enforced. I could find exactly who had visibility into classes down to the package level. The public module APIs were clearly delineated from the implementation classes. Not only was there loose coupling but there was also good cohesion.
Other unanticipated benefits to developers were immediately apparent:
- It was much easier to understand how things were being used and easier to change them.
- We have an environment that supports different Java VMs in different tiers, but we had to write code to the least common denominator. Now we could move the major parts of the server to Java5 and utilize things like generics without worrying about breaking other tiers because we had separated out the bundles that were not going into the Java 1.3 applications.
- We didn’t have a 20 minute ant build anymore on the developer desktop. It was part of the headless build only. Because the same build artifacts were used to launch apps in the IDE and drive the headless build, they were pretty much in sync with each other so developers didn’t need to do a headless build. Just edit code, launch application.
- We could create extension points for add-ons, rather than implement complicated hooks with classloaders and convoluted configuration files. This is part of Equinox, not OSGi per se. It’s the same kind of benefit you would get with Spring.
- Eclipse was much more responsive. Apparently having lots of small projects instead of one giant one is more efficient for many tasks. It didn’t improve Eclipse build times but it did help with other things like auto-complete and refactoring.
There were many, many other nice benefits yielded from this exercise, most of them unanticipated.
Would anyone yield these kinds of benefits from converting to OSGi? I don’t think so. We saw such a dramatic improvement because our existing infrastructure was so dilapidated. Our code was old and the buildsystem brittle. It’s a common scenario for many Java developers–dealing with the legacy of a system that became massively bloated over time. We all look at systems like that and wistfully imagine rewriting layers with Hibernate, Spring, Struts, JSF–whatever. Many times the benefit doesn’t justify the cost, or the rewrite turns into a classic Second System. From the outset this didn’t seem much different. Early on I worried this was going to be a very ambitious undertaking, and I had doubts about whether it would spin out of control.
It didn’t turn out like that at all. In fact, this was my first experience giving a facelift to such an old application with relative ease. I didn’t have to rewrite any code (other than fixing a few bugs). I just moved classes around. I did rewrite the buildsystem, but since the new build artifacts were a tiny fraction of the size of the old buildsystem, this effort paid for itself quickly. It only took a long time because I was working alone and was being extremely cautious, keeping the system stable continuously throughout the process, rather than just break everything for two weeks and put it all back together again all at once.
Engineers understand the value of this kind of improvement immediately. You can almost get Product Managers on board with the “SOA in a JVM” story. But Sales and Marketing won’t bother to stop typing on their Blackberries once you start talking about OSGi. For me this was mostly about extending the life of a very useful piece of software. The legacy of older applications isn’t just the code, but all the energy, innovation and hard work that goes into them, regardless of how you feel about the end result. We owe it to the original authors to make as much as we can of their work. We’d want nothing less for our legacy.