Monday, June 11, 2007

Will JSR-277 take us out of the Dependency-Hell?

Back in time before .Net, on Windows there was something called DLL-hell. In Java, we have a, somewhat mildly version, of that problem, which we could call Jar-hell but which is actually more generally a dependency hell.

Take a common technology stack for a JEE application, it could contain these major dependencies:
  • Tapestry
  • Spring
  • Hibernate
Each of these dependencies depends on other libraries (jar files), which again depends on other.... More often than not, there will be conflicts. Tapestry might depend on commons-collections-3.0 and hibernate might depend on v2.0. Only one version can be the one loaded on a given classloader, so one of them have to work with a version it was not tested with.

These dependencies easily comes out of hand. The most common solution I see applied out there, is something in the lines of, taking the most recent version of the jars in conflict and hoping it works.

Another problem is that the dependencies between jars are not explicit. There is no standard way for a jar file to express a dependency on another jar file in a given version. Today, we need each individual project to document their dependencies. More often than not, this is not done and jar file dependencies distributed with a library, often have file names without versions.

.Net solved this
In the .Net platform, assemblies are the jars of .Net and all assemblies contain both their own version and the names and versions of other assemblies, that it depends upon. The CLR is responsible for using this information at execution-/load-time to locate the depended upon assemblies, in correct versions, and link them together.

To be able to do this, there are some important aspects of the .Net platform itself:
  • A standard, system-wide place to deploy assemblies (the Global Assembly Cache, GAC)
  • Version of assembly as a part of the assembly file format itself
  • Version and names of depended upon assemblies as part of the assembly file format itself
  • Code in the CLR (the VM of .Net) that makes use of this information
In addition to this, each application can use an application configuration file to further specify the assembly linking information. For example, if the need arises to link against something specific. Here is an example of such an application configuration file:
<?xml version ="1.0"?>
<requiredRuntime version="v2.0.50727"/>
<supportedRuntime version="v2.0.50727"/>
<assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
<assemblyIdentity name="foo-assembly" culture="" publicKeyToken="83d83231a2fe9534"/>
<bindingRedirect oldVersion="" newVersion=""/>

This file will force the CLR version used to v2.0.50727 (and fail execution of not available), and redirect the dependency on v2.1.0.0 of foo-assembly to v2.2.0.0 of same assembly.

JSR-277 might come to rescue
Finally, someone has seen the need for this on the Java platform too. JSR-277: "Java Module System", seems to aim at solving this stuff. What JSR-277 will provide is:
  • A new distribution format called a Java Module, which will contain metadata, both about what the module itself contains, but also about which dependencies code in the module has.
  • A standardized versioning scheme
  • A repository for storing, discovering and retrieving modules
  • Runtime support in application launcher and class loader for using the module metadata
Java Modules will simply be jar files which contain a METADATA.module file in a MODULE-INF directory.

It looks like the JSR has taken a lot of its ideas from OSGi. For instance the idea of explicitly exporting which parts of the module, that can be depended upon from other modules. Actually, this opens up for a nice feature, as the JVM can hide internal dependencies from other modules, in effect making different versions of the same jar file possible in the same application, as long as they are non-exported dependencies.

All in all, JSR-277 seems like a huge bunch of features and functionality, that has the possibility of a major impact on the platform as we know it.

The JSR-277 seems to be scheduled for Dolphin and I hope it will make it into that. But I also hope for early adoption of its ideas by the community. The success of such a feature depends upon framework and library authors to embrace it. Just look at what maven has done for a common repository to locate dependencies from and version information in the names of jar files. There was a long time with maven1, where far from all projects embraced its use.

So, what do you think, will JSR-277 take us out of the Dependency-Hell? I sure do hope so!


Peter said...

The interesting question is, what will JSR 277 bring to the table in 2009 that OSGi/JSR 291 not already provides today? And more, like dynamic loading and unloading ...

Why would you like to wait 2 years for something that has a lot less ambition?

Kind regards,

Peter Kriens

Kocka said...

I will check the specification, but I think maven gives a good enough solution for this problem, also there is an interesting classloader called POMStrap that loads the classes with based on the info from the POM.

Mihai Campean said...

I agree with Peter, we have the possibility to get rid of the jar hell right now. OSGi, offers a lot more than just dependency management and modularization, dynamic library loading and some other standard services for various types of applications are just the surface.

murphee said...

Use OSGi - it's a solution that is proven. Eclipse is based on it (ever since Eclipse 3.0, released in 2004). More and more companies such as IBM (their J2EE server) and BEA started building on OSGi, alongside many more smaller companies.

And keep in mind: JSR 277 will only be usable from Java 7, which will be out in early 2009, and will probably have an adoption rate similar to Java 5, due to its many changes. JSR 277 is unlikely to be backported, because it requires significant changes in the JVM Spec to even make it possible - the Spec team says this was necessary, so I guess there's no way to get the same behavior/speed/stability on older (non Java 7) versions.

In short: Forget about JSR 277 in this decade - you won't be able to use it.

Also: OSGi is here _now_ and available on nearly everything that runs Java - also J2ME and ancient Java versions (I believe the base even runs on Java 1.1).

For more info, see this for instance:

Per Olesen said...

Hi, you all. And thanks for the comments. I've read up on the stuff you mention, and I must agree.

We should go for OSGi now and kill JSR-277.

I for one was a bit disappointed about the deep level of changes needed in the VM and classloader to make JSR-277 work.

Alexander Shvets said...

Hi, Peter:

Agree with you completely. With lots of jars involved it becomes unmanageable. And nobody wants 2 years to have it implemented...

I have a project which tries to answer this question. it's not that wise and intelligent as Java Modules, but it's good enough for today.

I use classworlds library for dynamic building of CLASSPATH on one side and maven dependencies file on another side. It does not mean that it's based on maven at all.

Currently I use it for running various scripting languages for JVM. But it also could be used for any other Java project.

If you are interesed, you can look at details here:
Alexander Shvets.