Thursday 30 July 2009

Senior developer now

My new job has been running quite smoothly so far, although the seniority aspect is a little different to what I was expecting as most of my teammates are recent graduates or still studying.

Today I got that "hmm, maybe I am getting old" feeling when one of my colleagues mentioned that he would only have been five or six years old around the timeframe I was describing. It started off as a, "what's this WSDL file for?" question that lead me to explain a bit about what it is and how it is used etc. I then dug a bit deeper and mentioned how a similar concept called CORBA had been around since the mid-nineties but was largely considered as a legacy approach these days.

I wonder what will be the equivalent of punch cards when younger people hear about my historical experiences of using computers? Loading programs onto the ZX Spectrum by casette tape is about as historic as I get (and I still have a working 48k Spectrum in storage back in New Zealand).

Eclipse plugins playing catch-up

Since upgrading to Eclipse Galileo (a.k.a. 3.5) I've noticed that there has been a definite lag in plugin developers supporting the latest version.

A couple of examples that have resulted in me keeping a parallel Ganymede (3.4) version installed include:
- Google App Engine
- Spring Tool Suite (well, not really, as I uninstalled this once I saw the recommended configuration tweaks to get the plugins to play nicely with OSGi)

Today I noticed that SpringSource have blogged and tweeted about an impending milestone 1 release of Eclipse Groovy Tools, still tied to 3.4 but with the assurance that 3.5 support will be available soon.

Thursday 23 July 2009

Android phone, to buy or not to buy?

This evening I attended a meeting of the London Android group hoping to glean enough interesting information to tempt me into upgrading from my current mobile phone (see image).

So far my market research online has been interesting. Some mobile phone deals come with special offers - I don't think I need hair straighteners thanks anyway.

A "deal" that almost sucked me in turned out to be for a longer contract term, and included "optional" insurance (free for the first month, then you have to opt out or be charged).

Vodafone almost had me ordering online - but unfortunately I have only been in the UK for the last 10 months or so, so I can't give them valid UK addresses for the past 3 years!

I have been thinking for a while that a decent phone could save me money - I wouldn't have to buy guide books or maps for long weekends out of London etc., but £30 times 18, + roaming data rates could pay for a lot of guide Lonely Planet books!

I'll just have to expand the cost rationalisation out to include how much I already spend on my pay-as-you-go mobile, then see if I could connect my laptop to use the unlimited internet access on the phone - saving me what I currently pay for pay-as-you-go mobile broadband...

Tuesday 21 July 2009

JBoss following SpringSource's leadership, by emphasizing separation between free and commercial offerings?

I've recently started working at a new job, which has involved installing and getting up to speed with some JBoss software.

Based on my previous experience, I thought that it would simply be a matter of heading to the JBoss website and downloading a zip containing the binaries. Alas that was not to be, as the website only appears to offer 30 day evaluation downloads and emphasizes the commercially supported versions.

I thought this might not be a new thing, as it had been almost a year since I last dabbled in JBoss application servers and portal servers. When I checked what tweets had been sent on Twitter today I found that John Smart (author of Java Power Tools) had noticed a similar situation.

So, I had another look and found a single page mentioning the availability of community versions of the software and linking to - even then the most obvious download links on the community site appear to direct users to the evaluation downloads. and seem to be the main pages to find your way to the non-evaluation implementations.

This reminded me of Rod Johnson's blog post entitled, "Red Hat Reacts to SpringSource's Leadership", as it seemed that JBoss / RedHat were now making it more difficult to get at their "open source middleware" much like how SpringSource still require registration before allowing people to attempt to download their free Spring Tool Suite.

If you navigate through the site you could even be forgiven for thinking that Hibernate might only be available as part of a commercial product download. It's fair enough for them to be economical with the truth when they're trying to marker their commercial products.

From my limited experience, it would appear that these significant providers of enterprise Java software have somehow gotten the idea that by making the process of acquiring the community / free version of their software, they will somehow encourage people to look into their commercial offerings. I, for one, am not convinced that the approach will have a positive effect on sales or subscriptions.

If and when Oracle buys Sun Microsystems, they could make some tweaks to the MySQL download process - which currently only has optional registration - to try to introduce a negative initial user experience.

Saturday 18 July 2009

What you don't know that you don't know

Software development is an interesting process

Once you've identified what needs to be developed there's typically some aspect that you haven't dealt with before. Here are some examples:
- a data type in the persistence layer (e.g. GIS Shapefile data)
- an unfamiliar MVC framework (e.g. moving from Struts to Spring MVC)
- a different JDBC connection pooling implementation
- Grails and Groovy instead of java with Spring and Hibernate

The decision to use the unfamiliar has been made with good reasons, it's your job to get stuck in and make it all work.

The approach that I like to take in these types of situations is to isolate each unfamiliar component and have a scratch project where I can try things out quickly and without effecting my development codebase.

With a testing environment in place, I can try out some distinctive characteristics that the application requires.

An example: Loading in some GIS data

This is the sort of thing that you expect to be trivial, after all the product has extensive documentation and is in use in hundreds, if not thousands, of organisations worldwide.

Underestimating the importance of this aspect of a project cost me a weekend researching and installing GIS libraries into PostgreSQL a few years back. Our team developed on Windows workstations, and intended to deploy the finished product onto a Linux server (Debian, in case you're interested). Sure enough, when I tried deploying to a Linux staging server the core of the system that we had been working on for several weeks failed to even initialise.

This was a case of what I like to call, "You don't know what you don't know". We knew what we wanted to produce, we knew what API to use to produce it, but we didn't know that the implementation we had based our GIS manipulation on would not work under Linux.

Fortunately the project schedule had been planned (by yours truly) to get the GIS code in place early on, so apart from my lost weekend the development continued and delivered on time and on budget.

The client subsequently commissioned me to use the same technology for another project, which was an interesting exercise in reusing code originally intended as a "one off". Both sites are still up and running.

Thursday 16 July 2009

Refactoring of persistent classes in Java

This morning on my way home from the latest job interview, I got to thinking about the costs and benefits of applying different approaches to persisting the domain objects of an application.

The company that I had the interview with have two development teams, working on two major projects. One of the projects is using iBatis, which I took as a sign that this company isn't just blindly following the majority of the industry, who seem to be using Hibernate. The other project is using JPA. So I had to ask why they have gone with a different approach, to which the response was that it was quicker and more flexible while the product is still being fleshed out.

So, what happens when you need to make some changes to the structure of your domain, and, correspondingly, your database structure? Is there a best practice, or does each organisation have their own approach?

Are there tools out there that recognise the side effects of the changes that you are making to your Java code, and can produce SQL DML update scripts for you to apply?

This situation reminds me of the early 2000s when UML diagrams were trendy and tools such as TogetherJ attempted to keep class diagrams and source code in sync with eachother, except this situation has an additional dimension to it - in addition to the mappings between your classes and database tables there is existing data which needs to be transformed to fit the new structure.

I see this as a scenario where you really need to keep a close eye on what the ORM system you have chosen is doing. As far as I am aware if you make changes that effect where data is being stored, you are going to have to involve some human intervention for transforming your old database structure and transferring its data before you can safely put your modified code and its ORM mappings into place.

Of course all of this is nothing new, but I'm a little curious as to whether tools can make the implications of changes visible, or even automate the generation of database update scripts. The alternative would seem to require identifying the changes and following the approaches outlined in Refactoring Databases: Evolutionary Database Design.

It looks like someone has made an attempt at automating some of the updates required for updating a schema to match an updated version:

Another PostgreSQL Diff Tool

A quick Google search reveals there are a few commercial products for MS SQL Server as well.

A spontaneous Twitter thought

I wonder how long it will take for world records to start related to Twitter?

e.g. the most retweeted tweet.

I'm mildly curious about seeing some visualisations of patterns of activity in the twitterverse. So far I'd expect Michael Jackson's death to have been the most significant world event since Twitter has been in the mainstream.

Wednesday 15 July 2009

Is London shrinking?

The old expression that "it's a small world" rally rang true for me today.

On my way home from a job interview this afternoon I thought that I recognised an old friend from New Zealand queuing up to use an ATM. Sure enough it was! After nine and a half months I have finally bumped into someone that I know from back home.

As if that wasn't enough, when I attended the London Spring User Group meeting this evening I ended up chatting with one of the 2 people who had interviewed me for a different job yesterday afternoon! Wait for it, later in the meeting I recognised the name of one of the attendees who had won a book in the feedback survey prize draw. Down at the pub for the obligatory socialising and networking afterwards I inquired as to whether he was a recruiter - sure enough it was the agent that I have been dealing with for the job the same job. SMALL WORLD.

This month's Spring user group session was focussed on Spring Batch and what is coming up in Spring 3.0.

I found that I appreciated the details of Spring Batch much more this time around, as opposed to when I first saw a presentation about it - after lunch at the Spring in Finance seminar day back in September 2008.

Seeing the list of what will be removed or deprecated in version 3.0 of Spring made me feel better about not knowing "which controllers" I had used in my experience of Spring MVC - a question that came up in a recent job interview by phone. The hierarchy of form controllers are being ditched in favour of the annotation based approach. So, I have only been using the latest APIs - still doesn't hurt to read about the old way of doing things (even if the MVC chapter in my reference book is 100 pages long).

It's a numbers game - Java developer interviews

Today I heard that I was definitely the second choice, out of the final four candidates that were interviewed for a Java developer role last week. I hadn't been pressuring the agent for feedback, and he was quite keen on emphasizing that the company will be looking to grow further in the coming months - hopefully I won't still be available then.

On Wednesday morning I have my first face-to-face interview with another company who interviewed me by phone last Monday. After the phone interview they had another eight or nine candidates to speak to, so I've already made it to the final three.

Unrelated to the job interviews, I've started noticing that some of the people that are following me on Twitter don't appear to be interested in the topics that I am tweating about. If I see that they are following more than three times the number of people who are following them, then it reinforces my theory that they are just trying to increase their popularity for no good reason - so I block them.

Monday 13 July 2009

Java language specification makes for interesting reading

As a different approach to preparing for the latest round of interviews, I've been reading snippets of the Java Language Specification. After a decade of programming in Java, I was surprised to come across a keyword that I didn't recognise, strictfp.

Evidently it's been there since version 1.2, but I guess back then I was more interested in developing Swing components and troubleshooting garbage collection issues from not tidying up those components' listeners etc.

The old expression is still valid, you do learn something new every day.

Wednesday 8 July 2009

Hibernate and JPA

I've been posting responses to various queries on the Hibernate Forums lately.

There seem to be a few recurring queries, so I'm looking to put together a white paper of sorts.

Here are my thoughts of what this might include.

JPA versus Hibernate:
- Standards compliant versus proprietary
- EntityManager versus Session

JPA does not require EJB3
- using a JPA implementation does not necessarily require an EJB3 container

JPA implementations:
- Hibernate
- DataNucleus (formerly known as JPox)
- Eclipselink
- OpenJPA
- Toplink
- ...?

Things to consider when evaluating products for your product or project:
- Licensing
- Stability
- Documentation and support
- Performance
- Benchmarking that matches your system's likely use cases
- Cost

JDBC Connection pooling
- Avoiding stale connections
- Implementations:
- c3p0
- Spring / Tomcat
- ...?

Lazy initialisation
- collections


XML versus Annotation based configuration

- Annotations for declaring Transactional behaviour of methods
- Which methods in which classes to mark with transactional annotation

- Second level cache
- Query cache
- Session / EntityManager

Performance issues
- N+1 queries
- batch size

Coding considerations:
- lazy initialisation exceptions
- avoid use of instanceof operator when proxies may be in place

Monday 6 July 2009

Maven - more than meets the eye

Earlier this evening I attended a presentation by John Ferguson Smart, a fellow kiwi who is in London running his Java Power Tools Bootcamp at SkillsMatter.

The title of the session was, "Getting Serious About Build Automation: Using Maven in the Real World". Based on his recent JavaOne presentation of the same name (slides available for download).

Before attending the talk I was probably a lot like most of the readers of this blog (both of you? :-) ) in that I had used Maven to get some jars/artifacts into projects that I have worked on, set up some internal project dependencies, groaned aloud in frustration at the maintainers of other external projects for not having their config in place as part of their upgrade process..., but haven't really taken the time to see where to go from there.

Here is a brief summary of some aspects of Maven that I will be looking into further because of the talk:
- the m2eclipse plugin for Eclipse, including its visualization of dependency conflicts (I've seen these conflicts in a project I worked on, but didn't know quite what they meant or how to resolve them by exclusions or specific declarations)
- the ability to standardize project settings using inheritance, including the option to specify versions of artifacts in such a way that they will only be applied to child projects if the child project goes to introduce the dependency ( block)
- using multiple modules to reduce the build time
- the Nexus Maven Repository Manager - refered to as an enterprise repository for caching external dependencies and holding copies of locally built artifacts
- the use of patches to manipulate differences in artifacts being built for different platforms, rather than having to introduce runtime configuration through external dependencies

I find John's blog is a good source of up-to-date info about tools that Java developers should be using to make their lives easier, and their projects progress more smoothly.

Saturday 4 July 2009

What a difference a name makes

I've started working on an application to allow people to manage their Facebook groups from outside of Facebook, as a hobby project to keep my skills fresh while I am between roles.

This morning as I was writing some tests and expanding the application to the point where it would be ready to start interacting with Facebook, I found myself feeling uneasy about the structure of a Service.

The method needed to know about a few fields that I have already encapsulated elsewhere. I found myself wondering, do I pass in the entire object or should the object actually be responsible for calling the method and passing its member fields as parameters?

After a few minutes of umming and aahing, the penny dropped and I renamed my "Service" class to be a Gateway. The way that the call gets processed should have made it stick out like a flashing neon sign that this is a gateway.

Having an organisation object make calls on the gateway, rather than being passed as a parameter into the gateway seems cleaner, but I expect another aspect of the domain that I have yet to uncover will move the responsibility out of the organisation object. Of course day 1 of the project is probably a bit early on to get concerned about this sort of detail.

This is yet another reason why Martin Fowler's Patterns of Enterprise Application Architecture lives on my table, rather than my bookshelf.

Friday 3 July 2009

Professional Experience - A brief summary of commercial software development projects that I have been involved in

This is going to be a long post, something I am considering as an appendix to my CV so that interested parties can get a more detailed view of my areas of expertise.

- Developing RESTful web services using JSON
- Converting stand-alone SOAP services to be integrated into a web application
- Liaising with client project managers and their suppliers
- Consuming XML web services

- Mentoring junior developers
- Retail E-Commerce development using Hybris 4.0, Spring, MySQL, Tomcat 6
- Sole responsibility for implementing payment gateway integration with Commidea XML Web Service v4.

- E-Commerce development using Hybris 3.1
- Configuration of load-balanced production environment on hosted servers
- Developing a back office system using Grails and Groovy
Prototype reporting system for an online casino application with Grails 1.1 and MySQL 5.1.
Used GORM and GSQL (abstraction over Spring and Hibernate) for data access
Added HttpInvokerProxyFactoryBean instances to an existing Spring-based application to expose functionality

- Enhancements to existing applications
Updated database tables, views, stored procedures
Added config for re-structured Spring MVC controllers
Added formatting logic to Velocity templates
Configured TeamCity continuous integration server to check out from subversion and run tests
Set up Javascript and associated cross site scripting code for Facebook functionality

- Agile development in team of 8
Java Persistence API (JPA) (equivalent to Hibernate) for Object-Relational Mapping;
Java Server Faces (JSF) with extensive use of AJAX capabilities provided by the JBoss RichFaces components;
Use of Ivy for managing jar dependencies;
Ant build script;
Hudson with various plugins for continuous integration and monitoring of code test coverage;
Selenium for automated running of acceptance tests;
MS SQL Server 2005 and PostgreSQL databases;
Eclipse as IDE and CVS client;
Prototyping of secure Web services interaction with Ministry of Education;

- Harcourts Grenadier - Real Estate sales management system for a commercial property developer
Project manager / Programmer / Analyst and client liaison responsibilities;
Based on a similar previous project, but with different business rules, data format, and using Java Server Faces and Facelets;

- Infinity Investment Group - Peninsula Bay Property development marketing site
Struts 1.3 and Tiles for MVC architecture and template-based layout
Role-based access control using Struts and a custom authentication system;
Java version of GeoTools and Java Advanced Imaging APIs for generation of map images;
Postgis data source for GIS data;
SQL Server 2005 for registration and bid data;
JPox JDO (now known as DataNucleus and used by Google App Engine) for Object-Relational Mapping;
Javascript and dynamic image maps for presentation of up to date details on map images

Merging of existing contact details database with the new online system;

Provision of back office administration interface to allow staff to quickly view the current status of individual properties and prospective purchasers.

- Airways Collaborative Arrivals Manager
Agile development in a floating team of 4 developers, with a business analyst providing requirements;
Contributing to high level architecture discussions;
Extensive pair programming;
MySQL database;
High Availability Linux cluster;
Use of new java.util.concurrent capabilities. Initially tried Executor, later changed to use Futures;
Some use of GWT for prototyping the required AJAX functionality;
AJAX Javascript with JSON (JavaScript Object Notation);

- Infinity Investment Group - Pegasus Town
Primary developer with support from 2 others for prototyping GIS and modelling the domain;
Project management and client liaison;

Initial project was a website with a novel approach to displaying availability of property. This expanded to include display of live data on projectors in the sales centre, provision of a live auction system for the sale day.

Avoided considerable expense and risk of failure by strongly pushing for the sale day to be operated on the sale site, as opposed to setting up expensive and complex short term Internet connectivity from the site to the server hosting facility.

Success of this project lead to the client returning to us to apply the same underlying technologies for another property development project.

- Windows Mobile / Pocket PC application development using C#, Visual Studio, .net Compact Framework

2005 - 2007
Alchemy were initially approached to provide text messaging capabilities to the existing software, but were then invited to re-architect the system to make it more scalable and able to be maintained without system administrator intervention.

I was the primary developer, responsible for client liaison, acquiring requirements, designing, developing, testing, and deploying all changes.

The first stage involved converting classic ASP VBScript pages backed by multiple Access databases into JSP pages backed by a shared SQL Server database. Struts, Tiles and JDO were used for isolating the various layers of the application.

I suggested to the client that they could expand their potential revenue by adding students to as well as parents to the people that could be contacted by the system.

Enhancement projects included:
- addition of SMS text message sending capacility by sending requests to a SOAP web service
- set up of online stationery ordering using WestPac bank e-commerce services, and providing a simple back office system for schools to monitor orders
- creation of Early Notification System web services for Ministry of Education to allow schools to contact parents by email and SMS text message, with robust status reporting capabilities

- MainPower Content management system based website
Plone, customisation of templates and search functionality
Inquiry forms processed by Python scripts

- LocalEye - Portal website for local government promoting Christchurch and Canterbury New Zealand

Enhancements to Zope-based portal site;
Python scripts for manipulating search results from multiple indexes;
Upgrade of Swish-e search engine;
Evaluation of upgrade to later version of Zope

-NewJobZ - Website for a business based on assisting prospective immigrants to find work in New Zealand

Struts and tiles for MVC and templated look and feel;
TJDO for object-relational mapping;
SQL Server 2000 RDBMS;
Tomcat 5;
Processing of payments online using PayPal;

- RealJourneys Tourism Reservation System: J2EE server with Rich Java Swing GUI client
Java 1.4;
Implemented printing capabilities using XSLT and apache FoP;
Set up fax server interface using HylaFax and Java APIs;
Installed Linux operating system and fax serving software on client hardware.

1999 - 2007
- Harcourts Group
The largest real estate company in New Zealand had a number of projects that I worked on.
- Development of Swing UI components for trust accounting system (Java 1.2).
- Maintaining ASP intranet - including an enhancement that prevented cancelled long running queries from tying up processing resources on the web server.
- Production of Perl scripts to export listing data in various XML formats to several major partner websites.

1999 - 2002 (approximate)

- New Zealand Immgration Service
Maintained public website, processed change requests, added and maintained CGI scripts backed by a PostgreSQL database

Set up Swish-e search engine for indexing content


Conversion of classic ASP based website to a scalable J2EE based solution
A selection of high profile pages within the website were chosen for conversion from classic ASP to a more scalable Java EE based system.
Java Server Pages (JSP) were set up with the same HTML as the existing site (for consistency and a smooth transition);
Existing SQL queries were analysed and brought across to the new JDBC data access objects;
SQL Server 2000 was the existing underlying RDBMS;
Custom tag libraries were developed to provide content to the pages in a way that would work easily for graphic designers using DreamWeaver;
Stored procedures for a custom built online advertising module were tuned so that deadlocks would not occur;
Tomcat was configured as the server to host the new site;
Server side caching of data from the database and the filesystem (camera image file details) were implemented using a group of configurable time-based cache entry maps;
IIS was configured to redirect users from the old ASP URLs to the new site

Snow reports and webcam pages which had been pulling information from a database and a filesystem on every page request were now only updating their data periodically, significantly improving the site's ability to cope under heavy load.

E-commerce - Online shop for purchase of mountain passes - processing credit card payments in real time.

Snow reports, web cam pages, email newsletters.

- Colliers Jardine
Commercial property website with listing management backoffice interface
Team member for initial development, primary maintainer for ongoing changes.
Developed using SQL Server 7, IIS 4, classic ASP, Perlscript, modules developed in house as well as source from CPAN
Responsible for applying enhancements once they had been filtered through the project manager - e.g. addition of one-off campaigns where properties would be featured on the homepage and displayed more prominently with that property type's section of the site.

I know there's a heatwave on, but where's my cloud gone?

Thursday was not a good day to be deploying updates to my hobby application on Google's App Engine.

Early in the day I had been making some good progress with refactoring my client UI, and adding some intelligence to the server side for allowing game rules to start to take effect, but by mid-afternoon I noticed that I was unable to redeploy my application.

Secure in the knowledge that nothing of signicance had changed at my end, I continued to work offline - on paper even - for some design considerations that needed to be addressed soon anyway.

All going well, I should have something fully functional by the weekend.

My original plan of using Javascript for the client didn't feel right, as it didn't look particularly pretty even after a fair amount of fiddling around with CSS. So, I have gone down the Java Applet path with a Swing UI. Next version may even use JavaFX.