Groovy 2.0 Performance compared to Java

Also published on java.dzone.com, see link.

End of July 2012 Groovy 2.0 was released with support for static type checking and some performance improvements through the use of JDK7 invokedynamic and type inference as a result of type information now available through static typing.

I was interested in seeing some estimate as to how significant the performance improvements in Groovy 2.0 have turned out and how Groovy 2.0 would now compare to Java in terms of performance. In case the performance gap meanwhile had become minor or at least acceptable it would certainly be time to take a serious look at Groovy. Groovy is ready for production for a long time. Let's see whether it can compare with Java in terms of performance.

The only performance measurement I could find on the Internet was this little benchmark measurment on jlabgroovy. The measurement only consists of calculating Fibonacci numbers with and without the @CompileStatic annotation. That's it. Certainly not very meaningful to get an overall impression, but I was only interested in obtaining some rough estimate how Groovy now compares to Java concerning performance.

Java performance measurement included

Alas, no measurement was included in this little benchmark how much time Java takes to calculate Fibonacci numbers. So I "ported" the Groovy code to Java (here it is) and repeated the measurements. All measurements were done on an Intel Core2 Duo CPU E8400 3.00 GHz using JDK7u6 running on Windows 7 with Service Pack 1. I used Eclipse Juno with the Groovy plugin using the Groovy compiler version 2.0.0.xx-20120703-1400-e42-RELEASE. These are the figures I obtained without having a warm-up phase:

	Groovy 2.0 without @CompileStatic	Groovy/Java performance factor	Groovy 2.0 with @CompileStatic	Groovy/Java performance factor	Kotlin 0.1.2580	Java
static ternary	4352ms	4.7	926ms	1.0	1005ms	924ms
static if	4267ms	4.7	911ms	0.9	1828ms	917ms
instance ternary	4577ms	2.7	1681ms	1.8	994ms	917ms
instance if	4592ms	2.9	1604ms	1.7	1611ms	969ms

I also did measurements with a warm-up phase of various length with the conclusion that there is no benefit for neither language with @CompileStatic or without. Since the Fibonacci algorithm is that recursive the warm-up phase seems to be "included" for any Fibonacci number that is not very small.

We can see that the performance improvements due to static typing has made quite a difference. This little comparison does not make up for an only little ambitious performance comparison. But to me the impression that static typing in Groovy in conjunction with type inference has led to significat performance improvements, the same way as with Groovy++, has become very strong. With @CompileStatic the performance of Groovy is about 1-2 times slower than Java and without about 3-5 times slower. Unhappily, the measurements for "instance ternary" and "instance if" are the slowest. Unless we want to create master pieces in programming with static functions, the measurements for "static ternary" and "static if" are not that relevant for most of the code with the ambition to be object-oriented (based on instances).

Conclusion

The times where Groovy was somewhat 10-20 times slower than Java (see benchmark table almost at the end of this article) are definitely over whether @CompileStatic is used or not. This means to me that Groovy is ready for applications where performance has to be somewhat comparable to Java. Earlier, Groovy (or Ruby, Closure, etc.) could only serve as a plus on your CV, because of the performance impediment (at least here in Europe).

New JVM kid on the block: Kotlin

I added the figures for Kotlin as well (here is the code). Kotlin is a relatively new statically typed JVM-based Java-compatible programming language. Kotlin is more concise than Java by supporting variable type inference, higher-order functions (closures), extension functions, mixins and first-class delegation, etc. Contrary to Groovy, it is more geared towards Scala, but also integrates well with Java. Kotlin is still under development and not officially released, yet. So the figures have to be taken with caution as the guys at JetBrains are still working on the code optimization (see KT-2687). Ideally, Kotlin should be as fast as Java (see this post). The measurements were done with the current "official" release 0.1.2580.

And what about future performance improvements?

At the time when JDK1.3 was the most recent JDK I still earned my pay with Smalltalk development. At that time the performance of VisualWorks Smalltalk (now Cincom Smalltalk) and IBM VA for Smalltalk (now owned by Instantiations) was very well comparable to Java. And Smalltalk is a dynamically typed language like pre-Groovy 2.0 and Ruby, where the compiler cannot make use of type inference to do optimizations. Because of this, it always appeared strange to me that Groovy, Ruby and other JVM-based dynamic languages had such a big performance penalty compared to Java when Smalltalk had not. Well, coming to think about it: Hot Spot runtime optimization in Java was taken from Smalltalk, anyway (see this article). Nothing beats the arrogance of a Smalltalk developer, even not a Mac enthusiast... Seems like creating a JVM-based language is easier than optimizing it's byte code. From that point of view I think that there is still room for Groovy performance improvements beyond @CompileStatic.