Also published on java.dzone.com, see link.
End of July 2012 Groovy 2.0 was released with support for static type checking and some performance improvements through the use of JDK7 invokedynamic and type inference as a result of type information now available through static typing.
End of July 2012 Groovy 2.0 was released with support for static type checking and some performance improvements through the use of JDK7 invokedynamic and type inference as a result of type information now available through static typing.
I was interested in seeing some estimate as to how significant the
performance improvements in Groovy 2.0 have turned out and how Groovy
2.0 would now compare to Java in terms of performance. In case the
performance gap meanwhile had become minor or at least acceptable it
would certainly be time to take a serious look at Groovy. Groovy is
ready for production for a long time. Let's see whether it can compare
with Java in terms of performance.
The only performance measurement I could find on the Internet was this little benchmark measurment on jlabgroovy. The measurement only consists of calculating Fibonacci numbers with and without the @CompileStatic annotation. That's it. Certainly not very meaningful to get an overall impression, but I was only interested in obtaining some rough estimate how Groovy now compares to Java concerning performance.
The only performance measurement I could find on the Internet was this little benchmark measurment on jlabgroovy. The measurement only consists of calculating Fibonacci numbers with and without the @CompileStatic annotation. That's it. Certainly not very meaningful to get an overall impression, but I was only interested in obtaining some rough estimate how Groovy now compares to Java concerning performance.
Java performance measurement included
Alas, no measurement was included in this little benchmark how much
time Java takes to calculate Fibonacci numbers. So I "ported" the
Groovy code to Java (here it is) and repeated the measurements. All
measurements were done on an Intel Core2 Duo CPU E8400 3.00 GHz using
JDK7u6 running on Windows 7 with Service Pack 1. I used Eclipse Juno
with the Groovy plugin using the Groovy compiler version
2.0.0.xx-20120703-1400-e42-RELEASE. These are the figures I obtained without having a warm-up phase:
| Groovy 2.0 without @CompileStatic |
Groovy/Java performance factor |
Groovy 2.0 with @CompileStatic |
Groovy/Java performance factor |
Kotlin 0.1.2580 |
Java | |
| static ternary | 4352ms | 4.7 | 926ms | 1.0 | 1005ms | 924ms |
| static if | 4267ms | 4.7 | 911ms | 0.9 | 1828ms | 917ms |
| instance ternary | 4577ms | 2.7 | 1681ms | 1.8 | 994ms | 917ms |
| instance if | 4592ms | 2.9 | 1604ms | 1.7 | 1611ms | 969ms |
I also did measurements with a warm-up phase of various length with the conclusion that there is no benefit for neither language with @CompileStatic or without. Since the Fibonacci algorithm is that recursive the warm-up phase seems to be "included" for any Fibonacci number that is not very small.
We can see that the performance improvements due to static typing has made quite a difference. This little comparison does not make up for an only little ambitious performance comparison. But to me the impression that static typing in Groovy in conjunction with type inference has led to significat performance improvements, the same way as with Groovy++, has become very strong. With @CompileStatic the performance of Groovy is about 1-2 times slower than Java and without about 3-5 times slower. Unhappily, the measurements for "instance ternary" and "instance if" are the slowest. Unless we want to create master pieces in programming with static functions, the measurements for "static ternary" and "static if" are not that relevant for most of the code with the ambition to be object-oriented (based on instances).
Conclusion
The times where Groovy was somewhat 10-20 times slower than Java (see benchmark table almost at the end of this article) are definitely over whether @CompileStatic is used or not. This means to me that Groovy is ready for applications where performance has to be somewhat comparable to Java. Earlier, Groovy (or Ruby, Closure, etc.) could only serve as a plus on your CV, because of the performance impediment (at least here in Europe).
New JVM kid on the block: Kotlin
I added the figures for Kotlin as well (here is the code). Kotlin is a relatively new statically typed JVM-based Java-compatible programming language. Kotlin is more concise than Java by supporting variable type inference, higher-order functions (closures), extension functions, mixins and first-class delegation, etc. Contrary to Groovy, it is more geared towards Scala, but also integrates well with Java. Kotlin is still under development and not officially released, yet. So the figures have to be taken with caution as the guys at JetBrains are still working on the code optimization (see KT-2687). Ideally, Kotlin should be as fast as Java (see this post). The measurements were done with the current "official" release 0.1.2580.
And what about future performance improvements?
At the time when JDK1.3 was the most recent JDK I still earned my pay with Smalltalk development. At that time the performance of VisualWorks Smalltalk (now Cincom Smalltalk) and IBM VA for Smalltalk (now owned by Instantiations) was very well comparable to Java. And Smalltalk is a dynamically typed language like pre-Groovy 2.0 and Ruby, where the compiler cannot make use of type inference to do optimizations. Because of this, it always appeared strange to me that Groovy, Ruby and other JVM-based dynamic languages had such a big performance penalty compared to Java when Smalltalk had not. Well, coming to think about it: Hot Spot runtime optimization in Java was taken from Smalltalk, anyway (see this article). Nothing beats the arrogance of a Smalltalk developer, even not a Mac enthusiast... Seems like creating a JVM-based language is easier than optimizing it's byte code. From that point of view I think that there is still room for Groovy performance improvements beyond @CompileStatic.
We can see that the performance improvements due to static typing has made quite a difference. This little comparison does not make up for an only little ambitious performance comparison. But to me the impression that static typing in Groovy in conjunction with type inference has led to significat performance improvements, the same way as with Groovy++, has become very strong. With @CompileStatic the performance of Groovy is about 1-2 times slower than Java and without about 3-5 times slower. Unhappily, the measurements for "instance ternary" and "instance if" are the slowest. Unless we want to create master pieces in programming with static functions, the measurements for "static ternary" and "static if" are not that relevant for most of the code with the ambition to be object-oriented (based on instances).
Conclusion
The times where Groovy was somewhat 10-20 times slower than Java (see benchmark table almost at the end of this article) are definitely over whether @CompileStatic is used or not. This means to me that Groovy is ready for applications where performance has to be somewhat comparable to Java. Earlier, Groovy (or Ruby, Closure, etc.) could only serve as a plus on your CV, because of the performance impediment (at least here in Europe).
New JVM kid on the block: Kotlin
I added the figures for Kotlin as well (here is the code). Kotlin is a relatively new statically typed JVM-based Java-compatible programming language. Kotlin is more concise than Java by supporting variable type inference, higher-order functions (closures), extension functions, mixins and first-class delegation, etc. Contrary to Groovy, it is more geared towards Scala, but also integrates well with Java. Kotlin is still under development and not officially released, yet. So the figures have to be taken with caution as the guys at JetBrains are still working on the code optimization (see KT-2687). Ideally, Kotlin should be as fast as Java (see this post). The measurements were done with the current "official" release 0.1.2580.
And what about future performance improvements?
At the time when JDK1.3 was the most recent JDK I still earned my pay with Smalltalk development. At that time the performance of VisualWorks Smalltalk (now Cincom Smalltalk) and IBM VA for Smalltalk (now owned by Instantiations) was very well comparable to Java. And Smalltalk is a dynamically typed language like pre-Groovy 2.0 and Ruby, where the compiler cannot make use of type inference to do optimizations. Because of this, it always appeared strange to me that Groovy, Ruby and other JVM-based dynamic languages had such a big performance penalty compared to Java when Smalltalk had not. Well, coming to think about it: Hot Spot runtime optimization in Java was taken from Smalltalk, anyway (see this article). Nothing beats the arrogance of a Smalltalk developer, even not a Mac enthusiast... Seems like creating a JVM-based language is easier than optimizing it's byte code. From that point of view I think that there is still room for Groovy performance improvements beyond @CompileStatic.
