Friday, April 17, 2009

Running Spring on Google App Engine

In case you've been living under a rock the last couple of weeks, Google recently announced the addition of Java support to its App Engine. I have written a small sample application that leverages Spring and each of the Google infrastructure services that are exposed either as proprietary APIs or serves as backend to standard APIs.

The application itself, Feeling Lucky Pictures, is very simple: you login using your Google account, and you import images from URLs into your personal gallery. You can also send an email with your pictures attatched.

Integration with the Google infrastructure is as follows:
  • Authentication is of course done with the Google Accounts API.

  • Image import uses a regular java.net.URL input stream, but on the GAE runtime that class is backed by the URL Fetch API.

  • Storage is implemented using JDO, which is backed by a BigTable data store.

  • Reading objects from the data store uses the JSR 107 javax.cache API, which is backed by Memcache.

  • Imported images are enhanced with the I'm feeling lucky filter, part of the Images API.

  • Email is sent with the standard JavaMail API, which also is backed by a custom mail service on GAE.
As I said, the application is built on Spring as I wanted to see what problems, if any, I would run into if I were to run a regular Spring application on GAE. Most things worked out fine, but there were a handful of issues that I discovered and resolved, which is the real value of this application.
  • I decided to make as heavy use of the annotation configuration option as possible, including automatic classpath scanning. However, Spring will then attempt to scan for JPA annotations if certain conditions are met, which in turn will cause the javax.naming.NamingException class to load.

    This class is not on the GAE class whitelist, and working around the missing class by providing a JNDI API jar or a dummy class won't work either. My solution was to roll a JDO-only version of the spring-orm jar. This problem is also discussed in this thread, with alternative solutions.

  • When configuring the LocalPersistenceManagerFactoryBean, don't specify a configuration file property (i.e. "classpath:META-INF/jdoconfig.xml"). Instead, set the persistenceManagerFactoryName property to "transactions-optional", which is the name used in the default configuration file provided by the Eclipse GAE plugin.

  • I wanted to avoid using the various Google API static factories programmatically in my MVC controllers, instead injecting service interfaces like ImageService to improve testability. This worked fine for the most part, simply using the factory-method attribute in the bean definition, with the exception of the Cache interface.

    Cache extends java.util.Map, and for some reason the @Autowire mechanism requires generic key and value types on constructor parameters of Map type. Unfortunately the Cache interface is not possible to parameterize, so my workaround was to embed it in a very simple CacheHolder one-property class, which is produced by a factory bean and injected.

  • I'm caching picture ids per email address, like this: "foo@bar.com":[1,5,7]. In a regular in-memory HashMap cache you can add elements to a collection map value, but on GAE you need to overwrite the old entry with a new collection that contains the old elements plus the new one. See JdoRepository for details.

  • Spring provides a set of abstractions on top of JavaMail, which I wanted to keep. In particular, the JavaMailSender interface and MimeMessageHelper class, for sending emails with attachments. This turned out to be a bit cumbersome, since the GAE mail backend is wired into the implementation of the JavaMail API in an unusual way. There's no SMTP transport, for example.

    What I had to do was to override Spring's implementation of JavaMailSender and align creation and usage of javax.mail.Session and javax.mail.Transport exactly with the howto instructions for the GAE mail service. Basically you can't let the Transport connect and then send the message on the established connection, you need to do it all in one pass using the static Transport.send() method.

    Oh, and don't leave the body of an email empty, or you'll get a really poor error message.

Other than that it's business as usual. The complete source code is available on Google Code, and it's MIT licensed so you can do whatever you want with it. If it helped you out, or if you've found a better or simpler solution to any of the problems above, please drop a comment on this article.

16 comments:

Anonymous said...

Is there a reason why you took JDO instead of JPA? I assume because JPA would not work with Spring? Or is there an other reason?

Peter Backlund said...

Good question. Yes, the main reason was that I ran into problems with JPA annotation processing, and also that I wanted to see what it was like working with JDO.

Anonymous said...

Your answer corresponds to the consensus in this thread: http://groups.google.com/group/google-appengine-java/browse_thread/thread/d14e8b11f7b78a19#

...I have asked whether you should use JDO or JPA when you starting a new GAE project. The thread is quite insightful. So I think JDO is the way to go :)

Uday Bhaskar said...

Can I use google app engine for commercial purpose?

Peter Backlund said...

Uday, you'll have to consult the information that Google provides about App Engine. I'm not in a position to determine that.

Anonymous said...

Thanks again for your great proof of concept Spring@GAE app! It saved me lot of time. Just one question: My colleague uses JSF instead of Velocity. As far as I understand, both are a kind of "template engine". Is there a reason you have chosen Velocity and not JSF? Is I18n possible with Velocity? Or should I use JSF for *.properties-based I18n implementations and is there an out-of-the-box I18n implementation for Velocity?

Btw I am working on a project that tries to incorporate Spring@GAE with GWT.

Peter Backlund said...

Velocity has a much smaller scope than JSF, so they are not immediately comparable. Velocity is purely a template engine, and roughly matches JSF EL (i.e. #{foo.bar}). The combination of Spring MVC and Velocity together is comparable to JSF. It's perfectly possible to do i18n with Spring MVC + Velocity using the bundled #springMessage Velocity macros and the usual Spring i18n subsystem.

More info here:

http://static.springframework.org/spring/docs/2.5.x/reference/view.html#views-form-macros

Nadathur Srinivasan Ajay Kumar said...
This comment has been removed by the author.
Nadathur Srinivasan Ajay Kumar said...

I am sorry, but the code.google.com website does not seem to allow or have any download links. Can you help me view your source ?

Peter Backlund said...

You can checkout the code from the Subversion repository: http://code.google.com/p/feeling-lucky-pictures/source/checkout

Ivan said...

Hi Peter, thanks for this example, I am trying to figure out few things:
1) how is called PersistenceManager.close()? I understood that google advices use one PersistenceManager per request, but neither spring configuration seems like that (scope etc..)
2) I wanted to put service loayer in between controller and repository which would handle transactions etc.. You know in spring so easy @Transactional on method, so GAE implementation of service would have to inslude explicite tx.start / tx.commit how you think?

Peter Backlund said...

Hello Ivan,

JDO resource management, including closing the PersistenceManager, is handled by the Spring JdoTemplate much like all the Spring templates (Jdbc, Hibernate, iBatis, Jndi, Ldap etc).

AFAIK the usual Spring transaction handling is available, by using @Transactional + <tx:annotation-driven/> + a JdoTransactionManager bean.

http://static.springframework.org/spring/docs/2.5.6/api/org/springframework/orm/jdo/JdoTransactionManager.html

http://static.springframework.org/spring/docs/2.5.6/api/org/springframework/orm/jdo/JdoTemplate.html

Farhad said...

When I run your code on my local I get this exception, did you host your code on appengine to ensure it works properly?

java.lang.RuntimeException: Unable to replace JPACallbackHandler.
at org.datanucleus.store.appengine.DatastorePluginRegistry.getExtensionPoint(DatastorePluginRegistry.java:66)

Chris Marx said...

hi,
when i try to run the project now, i get this error:

Nested in org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'deletePictureController'
defined in file [C:\Java\workspaces\general\picol\war\WEB-INF\classes\rma\flp\DeletePictureController.class]: Unsatisfied dependency expres
sed through constructor argument with index 0 of type [rma.flp.PictureRepository]: : Error creating bean with name 'jdoPictureRepository' de
fined in file [C:\Java\workspaces\general\picol\war\WEB-INF\classes\rma\flp\JdoPictureRepository.class]: Unsatisfied dependency expressed th
rough constructor argument with index 1 of type [rma.flp.CacheHolder]: : Error creating bean with name 'rma.flp.CacheHolderFactoryBean#0': F
actoryBean threw exception on object creation; nested exception is java.lang.NoSuchMethodError: com.google.apphosting.api.ApiProxy$Environme
nt.getDefaultNamespace()Ljava/lang/String;;


has something in the app engine changed?

Unknown said...

first of all thanks for your project, it saved me a lot of time figuring out Spring and Velocity.

I managed to add spring valitation [springBind fix necessary] but still experience some problems wchich might be showstopping.

First of all, the caching mechanism doesn't work for me!

the method findByOwnerEmail returns null... actually null comes out from ownerCache.get(ownerEmail);

Every time I log in [get a new JVM instance?], the picture list is empty.... if I add some stuff, it'll be displayed OK, but only the ones added during my current session.

I've checked and entities are being persisted, and are being put into the cache with fillCache, but when I try to retreive them they disappear.

Replacing Cache with HashMaps makes the thing work, so the issue must be the way Caches are being used.

anil said...

Here is the link to use app engine and spring, this is based on many suggestions made on web and also suggestions from this blog.
https://docs.google.com/document/d/1QHD2T4nqBdSaer_rTpbyUOUODyFR
6sXnnGO9rxVcg7o/edit?pli=1#