Connection pooling in AWS Lambdas

A few months back I posted my findings from troubleshooting a resource leak in a Java application that had been lifted and shifted to become a lambda rather than a long-running app on AWS.

The resource leak turned out to be the setup of a fresh pool of connections to the back-end cache on each invocation of the lambda, without any corresponding clean up call to close those connections.

Yesterday I attended the Redis Day conference in London during which a presentation happened to include some code for a much simpler use case involving connecting to a similar back end cache - Redis - but with a different approach to initialisation.

The lambda from the presentation was extremely lightweight, so there was no bloated microservice framework involved.  Just a single method to perform a single task. The initialisation of the connection to the Redis system was performed in a static block and had no corresponding call to close it. I believe that this approach would ensure that the Redis client is reused between invocations of the lambda, and appears to be a recommended pattern for setting up connection pools for other resources such as database connections - relying on the receiving end to clean up resources if / when the lambda's container is shut down terminating the connection.

The static initialisation approach reduces the startup time for all but the first invocation of the lambda.

Hibernate and JPA

I've been posting responses to various queries on the Hibernate Forums lately.

There seem to be a few recurring queries, so I'm looking to put together a white paper of sorts.

Here are my thoughts of what this might include.

JPA versus Hibernate:
- Standards compliant versus proprietary
- EntityManager versus Session

JPA does not require EJB3
- using a JPA implementation does not necessarily require an EJB3 container

JPA implementations:
- Hibernate
- DataNucleus (formerly known as JPox)
- Eclipselink
- OpenJPA
- Toplink
- ...?

Things to consider when evaluating products for your product or project:
- Licensing
- Stability
- Documentation and support
- Performance
- Benchmarking that matches your system's likely use cases
- Cost

JDBC Connection pooling
- Avoiding stale connections
- Implementations:
- c3p0
- DBCP
- Spring / Tomcat
- ...?

Lazy initialisation
- collections

Proxies


XML versus Annotation based configuration


Transactions
- Annotations for declaring Transactional behaviour of methods
- Which methods in which classes to mark with transactional annotation

Caching
- Second level cache
- Query cache
- Session / EntityManager

Performance issues
- N+1 queries
- batch size

Coding considerations:
- lazy initialisation exceptions
- avoid use of instanceof operator when proxies may be in place

JDBC Connection pooling

On a development system that I have been working on lately, the MySql database connections have been dropping out due to a lack of activity overnight (> 8 hours - a default timeout).

There doesn't seem to be a suitable corresponding timeout setting for the default connection pooling system for the corresponding Grails datasource (DBCP), so I've gone looking at what the alternative pooling systems have to offer, and how they could be configured to replace the default Grails DataSource.

On the surface of things development for dbcp and c3p0 seem quite inactive. Although dbcp shows a bunch of updates coming in their next release, it has now been over a year since their last release.

The Hibernate guys appear to have stopped supporting use for Hibernate with DBCP

C3P0 seems to use SourceForge to allow downloads, but keeps the documentation elsewhere.

Proxool looks like it could have potential - pity the user mailing lists are full of spam, so the likelihood of getting community involvement is remote.

C3P0 will do for today.

Update: I just attended a SpringSource talk on tcServer in London, during which Mark Thomas mentioned that the Tomcat developers have been working on their own connection pooling implementation, and that he is in the process of contributing some fixes to DBCP. (No mention of c3p0).