Hibernate, In-clause, OutOfMemory

Using Hibernate 4.2 and MySQL in a project with an in-clause query such as:

select t from Thing t where t.id in (?)

Hibernate caches these parsed HQL queries.
Specifically the Hibernate SessionFactoryImpl has QueryPlanCache with queryPlanCache and parameterMetadataCache.
But this proved to be a problem when the number of parameters for the in-clause is large and varies.

These caches grow for every distinct query.

So this query with 6000 parameters is not the same as 6001.

The in-clause query is expanded to the number of parameters in the collection.

Metadata is included in the query plan for each parameter in the query, 

including a generated name like x10_, x11_ , etc.

Imagine 4000 different variations in the number of in-clause parameter counts, each of these with an average of 4000 parameters.

The query metadata for each parameter quickly adds up in memory, filling up the heap, since it can't be garbage collected.

This continues until all different variations in the query parameter count is cached or the JVM runs out of heap memory and starts throwing java.lang.OutOfMemoryError: Java heap space.

Avoiding in-clauses is an option, as well as using a fixed collection size for the parameter (or at least a smaller size).

UPDATE:

Check out this great post with more info.

For configuring the query plan cache max size, see the property 'hibernate.query.plan_cache_max_size', defaulting to 2048 (easily too large for queries with many parameters)

Code Warfare and Seam

pinned down after a short skirmish the previous day
it was too quiet, spirits were dampened
earlier morning brought some hope, help would be coming
which eroded throughout the day, it was not enough
deep in the trenches, we waited still, scounts had not returned
our only option was to retreat and go around
all were hesitant, so i made the move...

summary: i hate jboss seam
or rather we all do at this point

for the past 2 days, we (the entire team including even other teams) have been researching seam issues.
including class conflict issues, instances not matching expected classes, not being able to cast, for seam core classes.
the errors had been occurring for the past few months but not caused any observable issues... until now.
and were threatening to hold up deadlines.

today we found what appears to be the cause: having two seam ears in one jboss 4 server
this apparently causes the apps to randomly step on each other,
including loading/creating core seam components from wrong classloader, from other ear.

eventually, i gave up on trying to find out why, as we all slowly did.
it didn't make any sense, all ear isolation settings had been set and no obvious causes.
many fixes/attempts were made, many possible solutions, but none had any effect.

in the end, i went forward with removing seam from one of them.
since it had been added recently for one and thankfully used very little.
so it was a viable option, and better than setting up a new jboss to run each ear in its own.

... and so the battle was won,
but the war will continue
(haha war, get it? ear-war, ok, over and out).

JavaOne in October

As of yesterday, I am signed up to attend JavaOne 2011.

It will be from October 2-6 in San Francisco, California!

This will be my second time on the west coast and first time in California.

Normally I wouldn't attend such an expensive ($2k to register) conference but work is paying.

And yesterday was still during the early bird pricing.

I'm registered but still need to book flights and a hotel.

I can't really be excited about it. I don't like flying and I'm not sure what I'll be doing there.

These talks do sound interesting.

And at least the registration info said it could cater food for vegans.

Coincidentally Java7 became available yesterday.

I don't really care. I mean it is nice to see movement on Java. But I'll be more interested after the updates come out and it becomes stable.

Not that i'm even working on the latest Java6, so who knows when I'll try out the new features.

Open Source Contributions

So I've been contributing to open source projects for the last couple years, especially the OpenMRS project. The majority of my work has been around converting the project from Ant to Maven. The trunk code base has already moved to Maven and will soon be releasing this first version since the change.
Most recently I've continued work on a bash script to convert OpenMRS modules from Ant to Maven and started a Maven branch using this script for one of the modules I've worked with, Idgen.

Here's a list of issues I've either actively worked on or been involved in discussions (ordered roughly by time spent):
TRUNK-241 Add Maven Integration
TRUNK-1686 Create openmrs module maven conversion script
TRUNK-1690 Add coding conventions to maven
TRUNK-1712 Rename modified dependencies in maven repository and project
TRUNK-1682 Add maven module for checkstyle
TRUNK-1705 Add distribution section to pom for deploying to maven repository
TRUNK-1713 Fix dependencies in maven project for move to Spring 3
TRUNK-1830 Duplicate dependencies in poms
TRUNK-1747 Fix maven build to not fail if .svn folders not present
TRUNK-1704 jetty resource filtering prevents running mvn eclipse:eclipse
TRUNK-1703 jetty:run locks static resources on windows
TRUNK-1726 Allow jetty context to be configured
ITSM-450 Install a Nexus Server for Maven Repo
ITSM-463 Update CI plans to new maven trunk
TRUNK-12 Fix @Should tags in Generated Javadoc
TRUNK-434 Multi Level Database Caching
TRUNK-1734 Logic is not included in openmrs trunk anymore after maven switch

Another project I became involved with is the Maven Java Code Formatter project on GoogleCode.

For the OpenMRS project, I was looking for a way to format Java source code with a Maven plugin using either checkstyle or Eclipse formatter config files. There are a lot of developers on the OpenMRS project using different IDEs (Eclipse, NetBeans, Intellij IDEA) and also different OS, and this can result in ugly code (spacing, newlines).

This GoogleCode project fit what I was looking for and I was granted access as a committer. I cleaned up the project, added new features, added site docs, and released the new 0.2 version to the Sonatype and Maven central repositories (all documented in the issues and on the Features Wiki).