Implicits in Scala: Conversion on Steroids

Also published on java.dzone.com, see link. 

With the use of implicits in Scala you can define custom conversions that are applied implicitly by the Scala compiler. Other languages also provide support for conversion, e.g. C++ provides the conversion operator (). Implicits in Scala go beyond what the C++ conversion operator makes possible. At least I don't know of any other language where implicit conversion goes that far as in Scala. Let's have a look at some sample Scala code to demonstrate this:

class Foo {      def foo {         print("foo")     } } class Bar {     def bar {         print("bar")     } } object Test {     implicit def fooToBarConverter(foo: Foo) = {         print("before ")         foo.foo         print(" after ")         new Bar     }                 def main(args: Array[String]) {         val foo = new Foo         foo.bar      } }

Running Test.main will print to the console: "before foo after bar". What is happening here? When Test.main is run the method bar is invoked on foo, which is an instance of class Foo. However, there is no such method bar defined in class Foo (and in no superclass). So the compiler looks for any implicit conversion where Foo is converted to some other type. It finds the implicit fooToBarConverter and applies it. Then it tries again to invoke bar, but this time on an instance of class Bar. As class Bar defines some method named bar the problem is resolved and compilation continues. For a more detailed description about implicits compilation rules see this article by Martin Odersky, Lex Spoon, and Bill Venners. Note that no part of the conversion code from Foo to Bar is defined neither in class Foo nor in class Bar. This is what makes Scala implicits so powerful in the given sample (and also a bit dangerous as we shall see in the following).

If we tried to get something similar accomplished in C++ we would end up with something like this (C++ code courtesy to Paavo Helde, some helpful soul on comp.lang.c++):

#include <iostream> class Foo { public:     void foo() {         std::cout << "foo\n";     } }; class Bar { public:         Bar(Foo foo) {                 std::cout << "before\n";                 foo.foo();                 std::cout << "after\n";         }     void bar() {         std::cout << "bar\n";     } }; void bar(Bar bar) {     bar.bar(); } int main() {       Foo foo;       bar(foo); }

There are also "to" conversion operators defined by the syntax like:

class Foo {             operator Bar() const {return Bar(...);} };

Note that such conversions are often considered "too automatic" for robust C++ code, and thus commonly the "operator Bar()" style conversion operators are just avoided and the single-argument constructors like Bar(Foo foo) are marked with the 'explicit' keyword so the code must explicitly mention Bar in its invocation, e.g. bar(Bar(foo)).

The C++ code including the comment in the paragraph above is courtesy to Paavo Helde. As it can be seen it is not possible in C++ to achieve the same result as with implicits in Scala: There is no way to move the conversion code completely out of both class Foo and Bar and getting things to compile. So conversion in C++ is less powerful than in Scala on one hand. On the other hand it is also less scary than implicits in Scala where it might get difficult to maintain a large Scala code base over time if implicits are not handled with care.

Looking for a matching implicit to resolve some compilation error can keep the compiler busy if it repeatedly has to look through a large code base. This is also why the compiler only tries the first matching implicit conversion it can find and aborts compilation if applying the found implicit won't resolve the issue. Also, if implicits are overused you can run into a situation where you need to step through your code with the debugger to figure out the conversion that results in a different output being created than expected. This is an issue that made the guys developing Kotlin drop implicits from their Scala-like language (see reference). The problem that you can shoot yourself into your foot when overusing implicits is well known in the Scala community, for instance in "Programming in Scala" [1] it says on page 189: "Implicits can be perilously close to "magic". When used excessively, they obfuscate the code's behavior. (...) In general, implicits can cause mysterious behavior that is hard to debug! (...) Use implicits sparingly and cautiously.".

What remains on the positive side is a powerful language feature that has often proven to be very useful if applied with care. For instance Scala implicits do a great job in glueing together disparate APIs transparently or in achieving genericity. This article only dealt with one specific aspect of implicits. Scala implicits have many other applications, see f.ex. this article by Martin Odersky, Lex Spoon, and Bill Venners.

[1] "Programming in Scala", Dean Wampler & Alex Payne, O'Reilly, September 2009, 1st Edition.

Go-style Goroutines in Java and Scala using HawtDispatch

Also published on java.dzone.com, see link. 

In Google Go any method or closure can be run asynchronously when prefixed with the keyword go. According to the documentation "(...) they're called goroutines because the existing terms—threads, coroutines, processes, and so on—convey inaccurate connotations" (see reference). For that reason I also stick to the term goroutine. Goroutines are the recommended method for concurrent programming in Go. They are lightweight and you can easily create thousands of them. To make this efficient goroutines are multiplexed onto multiple OS threads. Networking and concurrency is really what Go is about.

This blog describes how to make use of HawtDispatch to achieve a very similar result in Java. HawtDispatch is a thread pooling and NIO event notification framework, which does the thread multiplexing that in Go is built into the language. There is also a Scala version of HawtDispatch. So the approach described here for Java can be applied in the same way in Scala. The code shown in this blog can be downloaded here from GitHub (includes a maven pom.xml to get HawtDispatch installed). Go provides channels as a means for goroutines to exchange information. We can model channels in Java through JDK5 BlockingQueues.

Let's have a look at some Go code that makes use of goroutines and channels (the sample code is shamelessly stolen from this article, see the chapter named "Channels"):

ch := make(chan int) go func() { result := 0 for i := 0; i < 100000000; i++ { result = result + i } ch <- result }() /* Do something for a while */ sum := <-ch // This will block if the calculation is not done yet fmt.Println("The sum is: ", sum)

Making use of JDK8 default methods we can define in our Java world something like a keyword go. For that purpose I created one named async (using pre-JDK8 we would have to stick to little less elegant static methods):

public interface AsyncUtils {     default public void async(Runnable runnable) { Dispatch.getGlobalQueue().execute(runnable); } }

The async method will execute Runnables on a random thread of a fixed size thread pool. If you wanted to implement something like actors using HawtDispatch you would use serial dispatch queues. Here is a simplistic actor implemented using HawtDispatch (with queueing being serial through the use of the queue class DispatchQueue):

public class HelloWorldActor { private DispatchQueue queue = Dispatch.createQueue()     public void sayHello() {         queue.execute(()->{ System.out.println("hello world!"); });     }     public static void main(String[] args) {         HelloWorldActor actor = new HelloWorldActor();         actor.sayHello(); // asynchronously prints "hello world"     } }

To be precise the HelloWorldActor in the snippet above is more of an active object as functions are scheduled rather than messages as with actors. This little actor sample was shown to demonstrate that you can do much more with HawtDispatch than just running methods asynchronously. Now it is getting time to implement the sample in Go in Java with what we have built up so far. Here we go:

public class GoroutineTest implements AsyncUtils {         @Test     public void sumAsync() throws InterruptedException     {         BlockingQueue<Integer> channel = new LinkedBlockingQueue<>();         async(()->         {             int result = 0;             for(int i = 0; i < 100000000; i++) {                 result = result + i;             }             channel.add(result);         }); /* Do something for a while */         int sum = channel.take();         System.out.println("The sum is: " + sum);     }         @After     public void tearDown() throws InterruptedException {         DispatcherConfig.getDefaultDispatcher().shutdown();     } }

The code presented here would also work with pre-JDK8 since JDK8 is not a requirement for HawtDispatch. I just preferred to make use of JDK8 lambdas and defender methods to get the sample code more compact.