Dependency management - the gotchas

Love it or hate it Maven and its ilk have reached a level of use in mainstream Java projects that it is generally simpler to set up your project dependencies using pom.xml file(s) rather than manually obtaining the appropriate jars for the libraries that you need to make use of.

To bring in a simple standalone library that is self-contained (i.e. a single jar with no dependencies) this can be a simple matter of including a few lines of XML.

If your project is doing anything slightly non-trivial then there is a reasonable chance that it will, in turn, be making use of non-trivial dependencies - which depend on other third party libraries/jars. These are termed as transitive dependencies.

If you are not paying attention to the way transitive dependencies are defined in third party POM files, you can experience what a few people I generally respect for their software development experience refer to the phenomenon of "maven downloading the Internet".

Sonatype has circulated a blog post on that subject:

http://www.sonatype.com/people/2011/04/how-not-to-download-the-internet/

I would like to highlight a couple of gotchas, surprising things that can slip into your project if you don't pay enough attention.

Take for example the popular logging system log4j. If you simply specify in your POM file that you want log4j included in your project would you expect javax.mail to be brought along with it?

javax.mail then has a dependency on the activation framework - which could be another jar.

This is a relatively basic example of a system specifying dependencies for functionality that you might not actually make use of. In this case log4j can allow you to specify and SMTPAppender to send out email messages when significant events are logged.

I first tried to use the SMTPAppender several years ago on a project that wasn't making use of a depency management system, so I ended up scratching my head wondering why the application was silently failing every time that something should have been logged using the SMTPAppender. Any guesses what the problem was? - I hadn't deployed javax.mail or the activation framework jar.

The log4j example above might be considered as an example of how dependency management can stop you from making silly mistakes, so the example that follows will show how the opposite can also be true.

On a fairly recent project I have been making use of Jersey to expose RESTful web services with the JAX-RS standard.

Partway through the project I was following some posts on the user group and realised that the latest version would allow us to have our JSON objects represented in a much more efficient manner, so I simply updated the version number for the relevant dependencies in my project pom.xml file and set maven off to download and assemble a fresh artifact.

Everything went fine until I attempted to deploy the generated WAR file. Then, much to my surprise, I began to see a warning about the inclusion of a jar file containing the servlet API - something which Tomcat is thankfully clever enough to exclude from the classpath.

Using the m2eclipse Maven plugin for eclipse I was soon able to determine that the Jersey 1.6 server artifact included a dependency upon the servlet 3.0 API.

I received a prompt response to a post on the Jersey user group advising that the error would be corrected in the next nightly build, but I am reluctant to deploy with a nightly build.

I don't know about you, but I really don't like seeing warnings in my application logs. So after a few days of tolerating the warning and checking on the status of version 1.7 of Jersey I decided to eliminate the problem by overriding the dependency using an alternative scope in a dependencyManagement section of my project's POM file. Not ideal, but effective.

So, the main thing to take away from these cases is that you should take the time to examine what your project dependency tree looks like whenever you bring in a new dependency or change version of an existing dependency.

Maven and friends can only get you so far

I think that one of the best things about software development is that it is always changing - unfortunately this ever changing environment doesn't often allow for things to change as simply as dropping in the next version.

Deploying applications to some servers recently I discovered that some of the components used different versions of some low level libraries. This could have made me appreciate more what projects developed with Maven or Ivy get to see while in progress, but in this case it was not some component of our code that had jumped ahead of some other component - it was actually a container application on the server.

Fortunately enough the versions of the Apache commons jars are backwards compatible, so it was a relatively painless upgrade process.

Maven - more than meets the eye

Earlier this evening I attended a presentation by John Ferguson Smart, a fellow kiwi who is in London running his Java Power Tools Bootcamp at SkillsMatter.

The title of the session was, "Getting Serious About Build Automation: Using Maven in the Real World". Based on his recent JavaOne presentation of the same name (slides available for download).

Before attending the talk I was probably a lot like most of the readers of this blog (both of you? :-) ) in that I had used Maven to get some jars/artifacts into projects that I have worked on, set up some internal project dependencies, groaned aloud in frustration at the maintainers of other external projects for not having their config in place as part of their upgrade process..., but haven't really taken the time to see where to go from there.

Here is a brief summary of some aspects of Maven that I will be looking into further because of the talk:
- the m2eclipse plugin for Eclipse, including its visualization of dependency conflicts (I've seen these conflicts in a project I worked on, but didn't know quite what they meant or how to resolve them by exclusions or specific declarations)
- the ability to standardize project settings using inheritance, including the option to specify versions of artifacts in such a way that they will only be applied to child projects if the child project goes to introduce the dependency ( block)
- using multiple modules to reduce the build time
- the Nexus Maven Repository Manager - refered to as an enterprise repository for caching external dependencies and holding copies of locally built artifacts
- the use of patches to manipulate differences in artifacts being built for different platforms, rather than having to introduce runtime configuration through external dependencies

I find John's blog is a good source of up-to-date info about tools that Java developers should be using to make their lives easier, and their projects progress more smoothly.

Grails gotchas / showstoppers effecting uptake

I was somewhat relieved when I came across this blog post which highlights the fact that other developers have also faced a loss of productivity when working with Grails.

The "convention over configuration" approach speeds up development and keeps the learning curve fairly easy for most developers, but the list of open issues on the Jira suggests that there are still some aspects that need tuning - or detuning in some cases (e.g. domain objects not having GORM functionality due to something missing in lazy decoration/injection).

I've been wondering what sort of risks are involved when you use a software product that is essentially just providing a layer of abstraction over several loosely related other products.

For example, what can you do if you want to upgrade to a later version of one of the underlying technologies (for a fixed bug, or to use a new feature), when the layer above that hasn't been tested or made configurable for that new release? At present, I would expect that you would be stuck with whatever version is compatible with your version of Grails. This could change once OSGi becomes a bit more mainstream, allowing for the use of multiple versions of the same libraries within a single application. I wouldn't want to rely on that being the saviour of any application of mine though.

Grails upgrade from 1.1 to 1.1.1, attempt 2

Today I came to realise an important reason for upgrading from Grails 1.1 to 1.1.1, the staging sites running on Tomcat were not allowing updates to data via GORM due to this issue.

My most recent contributions to the project worked fine on my local system, but that was just a stand alone Jetty running courtesy of mvn grails:run-app and not much use to the QA team.

So, I figured that it was worth going back to check whether the suggestions provided on the grails user mailing list for the maven plugin version issue would work.

Updating the plugins on my local system was fairly straight-forward, so I checked my config changes into version control and fired up a build and deploy through the continuous integration / build server.

The build required a little manual intervention to clean out old versions of plugins, which I was easily able to detect having seen the same error messages on my local workstation.

Deploying the built application onto a Tomcat system that is hosting several related web apps involved a couple of areas of uncertainty, but it did take a while to discover what was preventing the application from starting up. It turned out to be due to there being multiple versions of an underlying jar in the WAR because of a an in-direct dependency pulled in by maven at build time.

I contemplated upgrading the other application to use the same version of the jar, but decided to just set up an exclusion in the Grails-based application's pom.xml for the time being. All seems to be playing together nicely now.

QA can move my task cards across to "completed" - hoorah!

Not quite the bleeding edge of technology, but

The upgrade to Grails 1.1.1 wasn't quite as painless as I had been hoping it would be.

It was versioned as a minor release and had a fair number of Jira issues fixed - including several that had been hindering progress and could ultimately lead to Grails being pushed aside as the platform of choice in the next week or so - so I figured it was worth being an early adopter (upgrading within an hour of the release announcement).

Twitter came in handy for the first issue - a Windows batch script had an out of date version number embedded in it and other had struck the same issue.

We're now stuck on a version mismatch issue somewhere in the mirky mists of Maven and potentially out of date plugins.

My post to the mailing list has gotten some responses from other users in the same situation, so I wouldn't be surprised to find a solution in my inbox when I get to work in the morning (or am I being the optimistic software developer?).

I'm still unsure whether I want to stick with Grails, or go back to Java and Spring. Using Groovy has reminded me of my years of developing ASP pages in Perl - not much care for type safety (if you don't want it).