It's a bit late in the evening for me to give a detailed account of what exactly I did, why, and what I found - so here's the short version.
Grails 1.1 was misbehaving whenever we tried to update an existing persisted object.
Grails version 1.1.1 was shown as fixing that particular issue.
I volunteered to be the guinea pig to be the first in the office to try upgrading.
Hurdle 1 - Original Grails 1.1.1 download included an install script with a Windows specific issue
Hurdle 2 - Maven plugin for Grails also would not play nicely (something about a dependency introduced by one of the minor new features added in Grails 1.1.1)
...
It works - even the updates to existing objects make it to the database without throwing exceptions etc. Hoorah!
A couple of days later we struck an issue where Groovy was unable to determine that an object was supposed to have GORM methods injected. The symptom being that it was unable to find a property (actually method) called save.
Mutter mutter mutter....
The workaround for that case has been to add some code to the BootStrap.groovy file to force Grails/GORM to be aware of that type of object - preventing us from relying on lazy initialisation.
Sunday, 24 May 2009
Wednesday, 20 May 2009
Grails upgrade from 1.1 to 1.1.1, attempt 2
Today I came to realise an important reason for upgrading from Grails 1.1 to 1.1.1, the staging sites running on Tomcat were not allowing updates to data via GORM due to this issue.
My most recent contributions to the project worked fine on my local system, but that was just a stand alone Jetty running courtesy of mvn grails:run-app and not much use to the QA team.
So, I figured that it was worth going back to check whether the suggestions provided on the grails user mailing list for the maven plugin version issue would work.
Updating the plugins on my local system was fairly straight-forward, so I checked my config changes into version control and fired up a build and deploy through the continuous integration / build server.
The build required a little manual intervention to clean out old versions of plugins, which I was easily able to detect having seen the same error messages on my local workstation.
Deploying the built application onto a Tomcat system that is hosting several related web apps involved a couple of areas of uncertainty, but it did take a while to discover what was preventing the application from starting up. It turned out to be due to there being multiple versions of an underlying jar in the WAR because of a an in-direct dependency pulled in by maven at build time.
I contemplated upgrading the other application to use the same version of the jar, but decided to just set up an exclusion in the Grails-based application's pom.xml for the time being. All seems to be playing together nicely now.
QA can move my task cards across to "completed" - hoorah!
My most recent contributions to the project worked fine on my local system, but that was just a stand alone Jetty running courtesy of mvn grails:run-app and not much use to the QA team.
So, I figured that it was worth going back to check whether the suggestions provided on the grails user mailing list for the maven plugin version issue would work.
Updating the plugins on my local system was fairly straight-forward, so I checked my config changes into version control and fired up a build and deploy through the continuous integration / build server.
The build required a little manual intervention to clean out old versions of plugins, which I was easily able to detect having seen the same error messages on my local workstation.
Deploying the built application onto a Tomcat system that is hosting several related web apps involved a couple of areas of uncertainty, but it did take a while to discover what was preventing the application from starting up. It turned out to be due to there being multiple versions of an underlying jar in the WAR because of a an in-direct dependency pulled in by maven at build time.
I contemplated upgrading the other application to use the same version of the jar, but decided to just set up an exclusion in the Grails-based application's pom.xml for the time being. All seems to be playing together nicely now.
QA can move my task cards across to "completed" - hoorah!
Making adjustments to a moving target
Late last week I found myself in a mini spiral of database updates for a reporting system - removing an outer join in the morning, only to add another outer join in the afternoon.
Sure enough, this week I am taking a look at the bigger picture - where is the data coming from, and why is it so ssslllloooowwwww to come out. Late yesterday afternoon / early evening I looked into how the ETL tables are being populated and realised that the data in the new table that I have started joining against could easily be included in the denormalised structure that we have set up especially for reporting. I also came to appreciate why some other data had been included a couple of weeks back (before I had the bright idea of excluding it and adding a join to a later query).
I'm fairly certain that the changes I have applied today will improve performance considerably, and should also make the data access code more readable for future reporting requirements. I was unsure whether it would make sense to split out the ETL into 2 tables, as that could be considered as normalisation - but in this case it made sense, as the queries will be simpler because they will not have to filter out duplicate rows for the data that is common for a collection of entities that are related for a set period of time.
The "moving target" aspect of this exercise came in the form of some database structure updates being added by a colleague at around the same time. Essentially it has meant that I have no historic data to verify that the most complex report generated from this data is still working. In theory that should be a trivial obstacle to overcome, but it's the sort of surprise that can turn a morning's work into a couple of days' work if you're not careful.
Sure enough, this week I am taking a look at the bigger picture - where is the data coming from, and why is it so ssslllloooowwwww to come out. Late yesterday afternoon / early evening I looked into how the ETL tables are being populated and realised that the data in the new table that I have started joining against could easily be included in the denormalised structure that we have set up especially for reporting. I also came to appreciate why some other data had been included a couple of weeks back (before I had the bright idea of excluding it and adding a join to a later query).
I'm fairly certain that the changes I have applied today will improve performance considerably, and should also make the data access code more readable for future reporting requirements. I was unsure whether it would make sense to split out the ETL into 2 tables, as that could be considered as normalisation - but in this case it made sense, as the queries will be simpler because they will not have to filter out duplicate rows for the data that is common for a collection of entities that are related for a set period of time.
The "moving target" aspect of this exercise came in the form of some database structure updates being added by a colleague at around the same time. Essentially it has meant that I have no historic data to verify that the most complex report generated from this data is still working. In theory that should be a trivial obstacle to overcome, but it's the sort of surprise that can turn a morning's work into a couple of days' work if you're not careful.
Friday, 15 May 2009
Not quite the bleeding edge of technology, but
The upgrade to Grails 1.1.1 wasn't quite as painless as I had been hoping it would be.
It was versioned as a minor release and had a fair number of Jira issues fixed - including several that had been hindering progress and could ultimately lead to Grails being pushed aside as the platform of choice in the next week or so - so I figured it was worth being an early adopter (upgrading within an hour of the release announcement).
Twitter came in handy for the first issue - a Windows batch script had an out of date version number embedded in it and other had struck the same issue.
We're now stuck on a version mismatch issue somewhere in the mirky mists of Maven and potentially out of date plugins.
My post to the mailing list has gotten some responses from other users in the same situation, so I wouldn't be surprised to find a solution in my inbox when I get to work in the morning (or am I being the optimistic software developer?).
I'm still unsure whether I want to stick with Grails, or go back to Java and Spring. Using Groovy has reminded me of my years of developing ASP pages in Perl - not much care for type safety (if you don't want it).
It was versioned as a minor release and had a fair number of Jira issues fixed - including several that had been hindering progress and could ultimately lead to Grails being pushed aside as the platform of choice in the next week or so - so I figured it was worth being an early adopter (upgrading within an hour of the release announcement).
Twitter came in handy for the first issue - a Windows batch script had an out of date version number embedded in it and other had struck the same issue.
We're now stuck on a version mismatch issue somewhere in the mirky mists of Maven and potentially out of date plugins.
My post to the mailing list has gotten some responses from other users in the same situation, so I wouldn't be surprised to find a solution in my inbox when I get to work in the morning (or am I being the optimistic software developer?).
I'm still unsure whether I want to stick with Grails, or go back to Java and Spring. Using Groovy has reminded me of my years of developing ASP pages in Perl - not much care for type safety (if you don't want it).
Wednesday, 13 May 2009
JDBC Connection pooling
On a development system that I have been working on lately, the MySql database connections have been dropping out due to a lack of activity overnight (> 8 hours - a default timeout).
There doesn't seem to be a suitable corresponding timeout setting for the default connection pooling system for the corresponding Grails datasource (DBCP), so I've gone looking at what the alternative pooling systems have to offer, and how they could be configured to replace the default Grails DataSource.
On the surface of things development for dbcp and c3p0 seem quite inactive. Although dbcp shows a bunch of updates coming in their next release, it has now been over a year since their last release.
The Hibernate guys appear to have stopped supporting use for Hibernate with DBCP
C3P0 seems to use SourceForge to allow downloads, but keeps the documentation elsewhere.
Proxool looks like it could have potential - pity the user mailing lists are full of spam, so the likelihood of getting community involvement is remote.
C3P0 will do for today.
Update: I just attended a SpringSource talk on tcServer in London, during which Mark Thomas mentioned that the Tomcat developers have been working on their own connection pooling implementation, and that he is in the process of contributing some fixes to DBCP. (No mention of c3p0).
There doesn't seem to be a suitable corresponding timeout setting for the default connection pooling system for the corresponding Grails datasource (DBCP), so I've gone looking at what the alternative pooling systems have to offer, and how they could be configured to replace the default Grails DataSource.
On the surface of things development for dbcp and c3p0 seem quite inactive. Although dbcp shows a bunch of updates coming in their next release, it has now been over a year since their last release.
The Hibernate guys appear to have stopped supporting use for Hibernate with DBCP
C3P0 seems to use SourceForge to allow downloads, but keeps the documentation elsewhere.
Proxool looks like it could have potential - pity the user mailing lists are full of spam, so the likelihood of getting community involvement is remote.
C3P0 will do for today.
Update: I just attended a SpringSource talk on tcServer in London, during which Mark Thomas mentioned that the Tomcat developers have been working on their own connection pooling implementation, and that he is in the process of contributing some fixes to DBCP. (No mention of c3p0).
Subscribe to:
Posts (Atom)