Recently in Groovy on Grails Category

With the release of Grails 1.3.1 we decided that it was finally time to stop our ‘copy all the jars to lib’ process and use the awesome dependency manager.

As with any move from static lib directory to a smarter dependency manager we ran into some puzzling issues. One engineer set it all up and it worked great on our build system, but then about 50% of our engineers ran into some strange issues including this one which prevented any grails scripts from running:

...
SLF4J: This version of SLF4J requires log4j version 1.2.12 or later. See also http://www.slf4j.org/codes.html#log4j_version
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/Users/john_doe/.ivy2/cache/org.slf4j/slf4j-log4j12/jars/slf4j-log4j12-1.5.8.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/Users/john_doe/.ivy2/cache/org.slf4j/nlog4j/jars/nlog4j-1.2.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
...

Hacking around randomly seemed to fix it in every case. Here’s the crude process we ended up following to resolve these strange issues:

  1. Purge the local caches by deleting ~/.ivy and ~/.grails
  2. Reboot (Yes this fixed the problem for someone… go figure)
  3. Run grails upgrade over and over until it started working

What is the root cause? I have no idea. Perhaps our dependency on the public repositories is to blame: one should never trust a free third party service.

Groovy Deadlocks

Today I managed to set up my grails environment in such a way that it triggered deadlocks quite easily when stressed with a simple jmeter script. While this was unintentional, it did give me an opportunity to diffuse some ticking time bombs.

A few of the deadlocks were caused by some of the creative ways that my project uses the Groovy on Grails APIs for rendering pages from GSP code stored in a database, and with some tweaks most of those were resolved.

Some other deadlocks, though, seem to be the fault of Groovy itself. A quick google search turned up over 300 hits relating to deadlocks in the Groovy bug tracker:

http://www.google.com/search?hl=en&q=site:jira.codehaus.org+%2Bgroovy+%2Bdeadlock

This is when I had a sad.

I know that as an open source project if a bug really bothers me, I should submit a patch, which I am likely to do, but even if I find fixes for some of the deadlocks I’m facing, I’ll either have to wait for the changes to make it into the Groovy codebase, Groovy to release, grails to pick up the changes, etc. or I’ll have to hack the patch into my application and hope for the best.

Hopefully I’ll find some magic work around that does not tickle these bugs :)

Update: Someone must have seen this blog entry, or maybe they’re just cool. Anyway, they fixed this problem wicked fast: http://jira.codehaus.org/browse/GRAILS-6398 Grails continues to rock as a result.

Grails Override Configuration with Properties File

Sometimes you will need to switch up configuration in a hurry. For example, you may need to switch the database connection URL in order to use your warm standby database. This can be a pain in Grails because the convient-during-development Config.groovy is compiled both by grails run-app and grails war.

The simplest way to change configuration without rebuilding and redeploying your Grails application is to make use of an externalized configuration file. Although would appear to be very straight forward based on the docs, you will need to jump through a couple undocumented hoops to get it working in all environments.

  1. Create your override configuration file in the grails-app/conf folder. I called mine override.conf. For now you can leave it empty.
  2. Enable this external configuration file by pasting this into the top of your grails-app/conf/Config.groovy file:
grails.config.locations = [ "classpath:override.properties"  ]
This is where the docs end. When you start up grails you’ll see the following error message:
Unable to load specified config location classpath:override.properties : class path resource [override.properties] cannot be opened because it does not exist
1. Grails uses different paths in run-app mode and when being run as a war. We’re going to need to add a step post compilation to ensure our configuration file is available at all times. We’ll do this by modifying _Events.groovy (or Events.groovy if you’ve been migrating your grails app since v1.1). Open scripts/_Events.groovy, or create it if it does not exist. 1. Add the following to _Events.groovy (or merge it in if you already have an eventCompileEnd defined):
eventCompileEnd = {
    ant.copy(todir:classesDirPath) {
      fileset(file:"${basedir}/grails-app/conf/override.properties")
    }
}
  1. Try setting a new property and fire up grails as usual. For example, if you want to set the database URL, you’d add this to your grails-app/conf/override.properties file:
dataSource.url=jdbc:postgresql://example.host/somedb:shutdown=true

Source: Some Grails - user mailing list post by Ian Roberts

Grails restart loop

The Problem

The other day I was doing some bulk file operations on a grails project. I was moving some classes in bulk from a plug-in into an application. After I made the move I ran into a puzzling issue: grails kept restarting in a loop. Each time the plugin scanner ran (every 5 seconds after startup), grails would complain that it needed to compile 1 file. This went on for a few minutes until my app ran out of PermGen and crashed. To make matters worse, grails would not tell me which file it was recompiling even with logging set to trace!

This seems to be a classic grails issue that is rare, but frustrating.

The Cause

The likely cause is a groovy file outside the correct package or with a file name that does not match the case name. The latter can happen when tinkering around between case sensitive and case insensitive file systems as is common when developing with OS X.

The Fix

To fix it you need to find the messed up class and fix it. The below method is crude, but works.

  1. If your app crashes quickly due to PermGen issues, provide more PermGen space. You’ll need at least 2 or 3 minutes of run time for this to work. Use the JVM argument -XX:MaxPermSize=1024m if you have to.
  2. Let your app run for a few minutes. Let it recompile that file a few times.
  3. If your app has not already crashed, stop it
  4. Now to find the compiled class file. Go to the directory where grails claims to be compiling the file. In my case it was in ~/.grails/1.2.0/projects/fooApp/classes
  5. Run this shell command to see when most of the files were updated: find ./ | xargs ls -al The vast majority of the listed files should all have an updated time within the same minute. In my case it was 15:17 Lets fileter those out and see what’s left behind.
  6. Run another find command and see the newer files: find ./ | xargs ls -al | grep -v 15:17. I saw a few files and headers that had to be ignored, but in the middle I found the class name which was compiled from a .groovy file of a different name.
  7. Go back to your app code and resolve the conflicted state of your groovy file(s)
  8. Quit cursing under your breath and go have a beer if you need one

Maybe you just upgraded Groovy on Grails, or maybe you’re doing a fresh install. Either way you may be confronted with this curious error message:

$ grails test
Exception in thread "main" java.lang.NoClassDefFoundError: org/codehaus/groovy/grails/cli/support/GrailsStarter

What’s wrong? You have a grails mismatch. Your GRAILS_HOME environment variable is not pointing to the same place as the grails in your PATH. Make them agree and try your grails command again.

Rendering Groovy GSP Code From A String

Ever used WordPress? It has that cool feature that allows users to heavily customize their blogs by editing templates. If you’re trying to do something like that in Groovy on Grails, you’ve found the right blog entry. It’s quite easy as of Grails v1.1.1, although it does have some quirks and caveats.

First, we’ll need to import a few classes for this to work. Add these imports to your controller:

import org.codehaus.groovy.grails.web.pages.GroovyPagesTemplateEngine
import org.springframework.web.context.request.RequestContextHolder
import org.codehaus.groovy.grails.web.servlet.mvc.GrailsWebRequest

Next, in our controller we’ll need the template engine. Luckily just like our grails services the framework will inject this in for us. To let it know what to inject, add this to the top of your controller:

    GroovyPagesTemplateEngine groovyPagesTemplateEngine // allows us to call the Groovy page-rendering stuff within the controller

Now for the fun part. To render a template from a string just add this private method to your controller:

    private void renderTemplate(String content, Map model) {
        groovy.text.Template template = groovyPagesTemplateEngine.createTemplate(content, groovyPagesTemplateEngine.getCurrentRequestUri(request))
        def renderedTemplate = template.make(model)
        GrailsWebRequest webRequest = (GrailsWebRequest) RequestContextHolder.currentRequestAttributes()

        /**
         * There are extra characters rendered into my page:
         * ServletWebRequest: uri=/app_name/grails/controller/actionname.dispatch;client=127.0.0.1;session=xgjq4afedjry
         * this is a hack to keep that stuff from being rendered into the page
         */
        def crapAtTheBeginningLength = webRequest.out.toString().length()
        renderedTemplate.writeTo(webRequest.out)
        def stringToRender = webRequest.out.toString()
        render stringToRender.substring(crapAtTheBeginningLength)
    }

Simply call this function from any action like you’d call render, except pass in the String version of the template as the first argument:

    def index = {
       String templateString = "..."; //get the template string somehow
       //... code to build out the model to be passed into the template ...
        renderTemplate(templateString, [foo: "bar", stuff: listOfStuff])
    }

This should work pretty well. Now for the caveats:

  • I haven’t done any load testing yet, but there may be performance implications due to the way the GSP caching works. This technique appears to have circumvented some parts of it and may have significant negative performance impact.
  • GSP code can never be made completely safe. Never render GSP code that was created by untrusted people. Although you are passing a model in, the GSP can run any Groovy code that it wants to run and could result in a serious security breech.

Also, no I didn’t come up with this code on my own. Credit goes to a bunch of grails mailing list threads that I’ve lost links to, help from people in #grails on irc.freenode.net, and this other blog entry on the same topic and yet another related blog entry.

Update: Here’s another example that’s a bit more refined.

Grails ConfigSlurper and its Permgen Apatite

This is an older one. I’ve only verified it with Grails v1.1, but it may still exist. It’s frustrating enough that it warrants a post, though.

Is your app slowing to a crawl due to an amazing Permgen leak? I mean big like 400+ MB of Permgen usage and 50k loaded classes. Well if you happen to be abusing ConfigSlurper, I may have a solution for you.

Are you doing something like this?

for (int i = 0; i < 50000; i++) { 
    ConfigObject config = new ConfigSlurper().parse("foo = bar"); 
    print " ${i}" 
} 

Well stop doing it. You’re leaking classes. Anything that you need to update at runtime that often should not be in the configuration file. Use JMX MBeans or a rest web service to allow updates instead.

Thanks to everyone on this email chain for resolving this issue.

Since Grails v1.1.1 has come out, there has been a strange issue. It seems to sneak up on me every time something important happens, like a production release. Grails throws an exception and I see an error page on my web browser. The exception always looks something like this:

2009-07-16 21:10:16,923 ERROR   org.codehaus.groovy.grails.web.errors.GrailsExceptionResolver:60        groovy.lang.MissingPropertyException: No such property: save for class: foo.Bar
org.codehaus.groovy.runtime.InvokerInvocationException: groovy.lang.MissingPropertyException: No such property: save for class: foo.Bar
        at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:92)
       ...
Caused by: groovy.lang.MissingPropertyException: No such property: save for class: foo.Bar
        at foo.BarController$_closure8.doCall(BarController.groovy:105)
        at foo.BarController$_closure8.doCall(BarController.groovy)

Research has revealed that other people face this issue as well. Some solutions that people have proposed:

  • Wait a few minutes and try again
  • Restart your application server (this seems to work well for me)
  • run MyDomainClass.count in your Bootstrap.groovy
  • run MyDomainClass.get(-1) before you attempt save

These all seem like black magic to me, though. I wanted to actually understand what was going on. I found the related JIRA (GRAILS-4580) and even the commit that Graeme made to fix it (which will be available in grails 1.2).

After everything I’ve read it seems like it’s another problem with Hibernate proxy objects not getting initialized correctly. This, however, does not explain why it’s been fixed with an application server restart for me each time, or if there’s anything I can do to avoid it while I’m still using Grails v1.1.1.

My plan going forward: Do not change my approach. When I see this issue in the future I’ll continue to attempt black magic. If I have a show stopper issue, I’ll apply the fix to grails v1.1.1.

Mocking configuration in Grails for unit testing

Configuration can be a bit hairy to test in grails unit tests. While there is an official technique that I found in a JIRA that looks like this:

mockConfig '''
john.doe.name = "bill"
'''

I’ve never gotten it to work. I’ve fallen back to using the Groovy provided meta-programming techniques. This means you need to mock the configuration differently depending on how you access your configuration.

No matter what, start by creating a ConfigObject()

def mockedConfig = new ConfigObject()
mockedConfig.john.doe.name = "bill"

The next step depends on how you’re accessing config in the code you’re testing. If you’re using ConfiguratoinHolder, try this

ConfigurationHolder.config = mockedConfig

On the other hand, if you’re using grailsApplication, this will work better.

GrailsApplication.metaClass.getConfig = {-> mockedConfig }

This is yet another reason to be consistent in your coding. If you’re not consistent you may end up mocking both.

Easy Groovy on Grails Debugging

Maybe you’re experiencing trouble with your IDE debugger, or you’re just a hard core VI kind of guy. Either way sometimes you need something a little more than println. Lucky for you Groovy has a pretty powerful yet undocumented debugging tool built right in called the Object Browser. It’s far from a full blown debugger, but it’s a great object inspector.

To use this inspector just add this anywhere in your Groovy program where someObject is the object that you’d like to inspect:

groovy.inspect.swingui.ObjectBrowser.inspect(someObject)

You’ve probably already noticed the swingui part of it. This brings up a handy dandy window for interactive inspection. It looks something like this:

groovy_inspect.jpg

But wait, there’s more. If you’re working with closures, as is the case almost everywhere in Groovy on Grails, try inspecting the object owner. This will inspect the container for the closure you’re currently running, which provides a ton vital information.