Testing Groovy Actors

One thing I did not quite get when I was looking at multithreading technologies was how to unit test them.

I have not looked at Akka for a while, but a few years ago it seemed like the consensus was to use a logger like Log4J, and then parse the logging messages. That is kind of unwieldy and feels wrong. (Akka seems to have more ways to test now.)

I emailed the GPars list about putting private fields in an actor, changing them in the onMessage method, and then checking them afterwards; something like this:

You could do that. There is nothing wrong with having private fields in an actor. You might want to keep some data between requests (like an encryption key). But making a field just for testing seems wrong.

Then I had an idea: Have your actor call another class that does the actual work, and unit test the worker class:

I suppose that will not help with functional testing. It might violate some SOLID principle, or DRY, or GRASP or the Law of Demeter or Principle of Least Astonishment or Poe’s Law, but it seems like a pretty good idea to me.

You’re welcome.

Image from the “Imperial Menologian“, an 11th-century Byzantine manuscript, not to be confused with the Menologion of Basil II. Manuscript  housed at The Walters Art Museum, manuscript information here, image from World Document Library, image assumed allowed under Fair Use.

Groovy Concurrency With Closure Locks

I have played around a bit more with closures and locks in Groovy.

In one of Venkat S’s talks, he says that the java.util.concurrent.locks package solves a lot of the issues with the synchronized keyword: You have to have try/catch/finally blocks to get the lock and release it. I made a class that will hold a Lock object, and can take code as a closure and run it. It seems to work pretty well. At least I think so.

But I realized there is one possible issue: The ClosureLock class only imports from java.util.concurrent.locks. What if some of the code in the closure is from another package that is not imported in the ClosureLock class?

I made a class that imports groovy.time.Duration, and creates a couple of objects and invokes methods in a closure. And it ran without blowing up. I also made a class called DebugClosureLock that prints out some stuff to the console checking if the lock is held by the current thread.

Here is the DebugClosureLock class:

Here is the class that uses it:

So, to review: My question was would DebugClosureLock blow up when it was called since it does not import groovy.time.Duration? And the answer is it did not.

Here is the console output:

Paul King shows how to do this on slide 36 of his talk Groovy and Concurrency using metaprogramming:

Then you can use it like this:

You’re welcome.

Image from “Evangeliarium [Evangéliaire dit de Charlemagne ou de Godescalc]”, an 8th century manuscript housed at the Bibliothèque nationale de France. Source gallica.bnf.fr / BnF; image assumed allowed under Fair Use. It also has a Wikipedia page.

Some Groovy Threading

There is an example of using threads with Groovy in Venkat’s book.

He uses the GDK extension to Thread and calls Thread.start. I wondered if the main thread would wait until the started thread completed, and what would happen if you started another thread.

I made some changes, and ran it.

Here is my class:

Here is the result:


So the main thread gets to its last statement pretty quickly. The two threads run simultaneously. I already knew this, but it’s good to review threads every now and then.

I was always intimidated by threading and concurrent  programming. Maybe it’s not too bad in Groovy.

You’re welcome.

Image from Aurora Consurgens, a 15th century manuscript housed at Central Library of Zurich. Image from e-Codices. This image is assumed to be allowed under Fair Use.

Changing Immutable Objects

Recently someone at work someone sent me a link to an article saying that getters and setters are evil by a guy named Yego256. He has a few other interesting posts about immutable objects here and here. They can be added to the list I made of pages about immutable objects here.

Hopefully Grails 3 will utilize immutability.

It takes a shift in mindset to start using immutable objects. Plus, in Java (and Groovy) a lot of web frameworks don’t utilize them, so people are not too familiar with them.

One issue some developers have is: Many things in the world are mutable. How can you make “changes” to immutable objects?

One answer is that you can use the values you don’t want to change to populate a new immutable object, and your variable/field name points to a new immutable object.

(If you can think of a better way to explain this, let me know. It’s kind of confusing to say that an immutable object is referenced by a “variable name”.)

I will demonstrate this with Groovy (using the annotations EqualsAndHashCode, Immutable and ToString). I made a class to model a state in the USA. States do not change their names, abbreviations or year they joined the union. But many of them have changed their capital cities. So we will make an object, and change its capital city while keeping everything else the same.

Here is our class:

Now let’s look at using it in a Groovy shell. The first capital of Illinois was a town called Kaskaskia, which no longer exists. Then it moved to Vandalia (which does still exist) and finally to Springfield.

I will try to change a few fields to prove the objects are immutable, and I will call hashCode() to show that we are dealing with different objects.

So we used the existing values to make new objects. The JVM used the same variable name (“il”) to refer to different immutable objects.

Here is the code without the output:

2015-06-06_14.23.56 update: I now realize that in order to prove the objects were truly different objects, I should have called System.identityHashCode(il) instead.

You’re welcome.

Image from the Rheinau Psalter, a 13th century manuscript housed at Central Library of Zurich. Image from e-Codices. This image is assumed to be allowed under Fair Use.


I started working on the 4Clojure problems. Functional is certainly a different way of doing things.

I follow a couple of people. Sometimes I will solve a problem with a complex solution: if, loop, recur, let, etc, etc. And then I find out one of the people I follow solved it with one line. Then sometimes the opposite happens.

I am also looking at Luminus web framework. It would be nice if the Clojure community would get behind a web framework, or if there was a default choice for beginners/people new to the language. Any time this topic comes up, there is always a lecture on how Clojure/Lisp people like to put together libraries instead of use frameworks. Ruby has other frameworks besides Rails, and plenty of libraries at various levels of abstraction. But if you ask how to make a web app in Ruby, people will point you towards tutorials, not start preaching. Sometimes I wonder if Clojure will be the Lisp that takes off, or if it will just be another Lisp that never gained escape velocity.

Image from  Aurora Consurgens, a 15th century manuscript housed at Central Library of Zurich. Image from e-Codices. This image is assumed to be allowed under Fair Use.


Locks And Closures In Groovy

In a previous post, I wrote about using lambdas in Java to simplify the use of locks. I was able to get this up and running in Groovy.

I have the code for it at Github. I made a class called ClosureLock which has a method called “lockSomeCode” which takes a closure as the argument. It creates an instance of ReentrantLock, and calls the closure in a try/finally block. Now that I look at it, i suppose I should refactor it so the ReentrantLock is a private variable and can be re-used, since ClosureLock can be reused.

I also created a class called FirstLockRunner, which instantiates ClosureLock and uses it a few times.

After I figured this out, I did some googling, and a few other people have done this before. A guy named David Clark has a class called DemoLockEnhancements in a project called GroovyThreading.

A guy named Johannes Link also did something similar.

There are a couple of new Groovy books out. I think I will look them over in order to understand closures/lambdas better. I have been reading that lambdas can be used to improve concurrency in Java and are thread safe.

I know that the consensus is to move away from locks and instead use other concurrency techniques. But I think that if locks can be done cleanly and safely, then their use should be considered.

Image from Rudolf von Ems’ Chronicle of the World, a 14th century manuscript housed at Central Library of Zurich (Wikipedia page here). image from e-Codices, assumed to be allowed under Fair Use.

Done With Venkat S’s Book

I am finally all through Programming Concurrency on the JVM by Venkat Subramaniam.

I am looking at some code he wrote to test classes in the java.util.concurrent.lock package. I might go through that.

I have a feeling that a lot of what I went through might be obsolete soon. There was a lot of code that could be done more concisely with lambdas, which are coming soon in Java 8. I think you might be able to do extend the ReadWriteLock class and have it take some code that you want to make thread-safe. In one of the times I saw him, he said that locks take care of any issues you would have with the synchronized keyword.

The problem is that you really need to use try/catch/finally blocks, and you should use the tryLock() method, and use a timeout. It can be a lot of code.  You might want to change one variable, but instead of having one line in your setter, you might need about 6. But if you could send that one line to a LockLambda (or whatever you want to call it), it would be better. You might need two, which granted is still more than one, but the LockLambda could handle all the try/catch/finally/get the lock/set the timeout/release the lock work for you, and you might be able to re-use the same LockLambda class.

I might blog about Akka, although I am not too thrilled with it. I do not like Scala, and some of the Akka APIs call Scala APIs. I liked GPars better. Although there were some things in GPars that are not worth the effort to use in Java, you can pretty much use just the GPars APIs. You do not need to look at any Groovy if you do not want to. I think some things I did not like with Akka will be easier once lambdas come out. But I still do not like Scala.

I may still look at the Play Framework.


Image from an 11th century manuscript made at the Abbey Library of Saint Gall  (Wikipedia page here), image from e-Codices, assumed allowed under Fair Use.


2013-07-14 Concurrency Update

I have been going through Venkat’s book, but I think I might bail on going any further.

The book came out in 2011, and there have been a LOT of changes to Akka in that time. They stopped using Multiverse for STM, and started using ScalaSTM. There were a lot of changes from version 1 to version 2 (starting actor systems, creating actors, sending messages, pretty much everything). There was a change in the API as I was going through the book.

Some of the method calls require inline Runnables or Callables, and that gets kind of awkward. I think I will move on to other things and come back when there is another edition of the book. Or I might wait until Java 8 comes out. I think the closures will make things easier.

Plus, since Akka is written in Scala, a lot of things are unwieldy in Java. I really do not like Scala. It has the syntactical elegance of Perl, with the simplicity and focus of C++. It is like religion or cigarettes: Its advocates tell me that if I gave it a chance I would see how wonderful it is. I did, and it’s not.

That said, I have no issue with the idea of using something written in Scala. I might take a look at the Play Framework. That is not as odd as it seems. I do not like working with C code. But a LOT of important stuff is written in C: The JVM, MRI, databases, the Linux kernel. If I refused to work with anything written in C, I would need to find a new line of work.

One language I think I will take a closer look at is Clojure. At first I was skeptical about it: It’s just another Lisp, and Lisp has never caught on, so why bother? But I have kept an eye on it, and I think this might be The Lisp That Sticks.

Plus, I have noticed something interesting about Lisp. A lot of people will start out developing in one language, and then they will become dissatisfied, and move to another. And then another. And then another. But it seems like when people find Lisp, they stop moving around. They seem to find what they are looking for.

In the next few months there are some Clojure meetups that will go over some Clojure web frameworks. I might look at them before that.

When Ken Kousen was here for the Groovy meetup, he said that he had a boss who kept saying, “Oh, we could do that in Lisp 20 years ago.” He thought it was really annoying. He said it was more annoying when he realized his boss was right.

Image from the Rheinau Psalter, a 13th century manuscript housed at Central Library of Zurich. Image from e-Codices. This image is assumed to be allowed under Fair Use.

Concurrency Without Pain in Pure Java

Here are the notes I took when Venkat Subramaniam was here for Austin JUG. I will mostly present them as I typed them.

Why Concurrency? Why do we care?

We now have mutli-core processors. Things that work well on one core may not work well on multiple cores.

The Java memory model controls how info goes back and forth from main memory and working memory.

His arguments in his classes are all final.

In the account class. Should the getBalance() be synchronized?

“synchronized” gets a lock and crosses the memory barrier, working and main memory.

Accessing a volatile variable, wait, join, a few other calls where you cross the memory barrier. We could compromise the invariant, like during a constructor.

So if you do not make getBalance synchronized, you might get an old value.

The code could have deadlock, or livelock (you wait, but nothing waits for you).

When you call synchronized, you might wait forever. There is no time out. So you use the Lock interface.

So for ReentrantLock you could call tryLock, or throw the Exception, then you have to unlock it.

You can order the accounts by ID to removed threat of dead lock.

95% of Java code is broken for concurrency. (Is he including code for Groovy, Scala, Clojure?)

Synchronize and suffer model.

Solutions: Software Transactional Memory (from Clojure) and Actors (Scala)

STM: We modify a variable (data) in our code. Shared mutability is bad. A few solutions: Avoid mutability. STM says let’s have a special type of variable: Managed mutable variable. It will only change in certain circumstances: a transaction.

Identity: Values are immutable. The stock price at 1:00 is different than at 1:10. But the 1:00 price will never change.

The identity is “google stock price”, points to a different value at different times.

We must protect the identity.

The problem in Java is that many times the problems happen without warning or error message. In STM, you get an error message.

This changes it in a transaction. If there are conflicts, the transaction is aborted and retried. Up to 100,000 times.

Can we do it in Java? He used ref and dosync, so can we use them in Java?

clojure.lang.Ref. Look at his code, see what class he uses for Ref.

The set method will use Lambda expression in Java 8. For now, we could do a Callable. LockingTransaction.runInTransaction

He is using an inline Callable.call()

What if you try to put more in the to account than is the from amount? It will throw an exception.

When to use STM? When you have multiple readers, multiple writers, and infrequent write collision. Not too many things are good for high-write collision.

You should not modify unmanaged variables in a transaction. Do not write to a file, print out messages, do not send email.

What about a long-running task

Another solution to shared mutability: isolated mutability.

Problem with encapsulation: Many threads can call the methods that change your private variables.

It is like voice mail: Many people can leave him a message, he gets a lot of messages, he goes through them one at a time. Actors have built-in message queue. Asynchronous messages.

The Actor runs on a separate thread.

When do actors make sense?

The fields are immutable. The messages are not, so they should be immutable.

You can have variables in an Actor, since only one thread will touch an Actor.

When does Actor make sense: Single threaded tasks sequentially

It punishes readers as much as writers. In STM, readers are never blocked. Writers are punished.

Purpose of Actor is not to deal with shared mutability, you want to prevent concurrent access, you want to serialize.

New concurrency API in Java 8.

You need Java 8 with Lambdas.

Look at java.util.streams.*

:: is a method reference

can be done as

Java is getting more functional

Synchronize and shared mutability can be avoided

JDK 5 concurrency is the assembly code of concurrency

Image from the Rheinau Psalter, a 13th century manuscript housed at Central Library of Zurich. Image from e-Codices. This image is assumed to be allowed under Fair Use.


Still Going Through Venkat’s Book

I am still going through Venkat S’s book.

I had gotten through the chapter on STM. A lot of the examples for STM dealt with some old versions of APIs. The first time I just read through them pretty slowly. I was able to follow. But then I did a bit more googling, and I was able to find later versions of the STM libraries he was using.

I had already started the chapter on actors. I decided to go back and work on the STM examples. It is taking a bit of time.

Image from  Aurora Consurgens, a 15th century manuscript housed at Central Library of Zurich. Image from e-Codices. This image is assumed to be allowed under Fair Use.