4Clojure

I started working on the 4Clojure problems. Functional is certainly a different way of doing things.

I follow a couple of people. Sometimes I will solve a problem with a complex solution: if, loop, recur, let, etc, etc. And then I find out one of the people I follow solved it with one line. Then sometimes the opposite happens.

I am also looking at Luminus web framework. It would be nice if the Clojure community would get behind a web framework, or if there was a default choice for beginners/people new to the language. Any time this topic comes up, there is always a lecture on how Clojure/Lisp people like to put together libraries instead of use frameworks. Ruby has other frameworks besides Rails, and plenty of libraries at various levels of abstraction. But if you ask how to make a web app in Ruby, people will point you towards tutorials, not start preaching. Sometimes I wonder if Clojure will be the Lisp that takes off, or if it will just be another Lisp that never gained escape velocity.

Image from  Aurora Consurgens, a 15th century manuscript housed at Central Library of Zurich. Image from e-Codices. This image is assumed to be allowed under Fair Use.

 

2013-07-14 Concurrency Update

I have been going through Venkat’s book, but I think I might bail on going any further.

The book came out in 2011, and there have been a LOT of changes to Akka in that time. They stopped using Multiverse for STM, and started using ScalaSTM. There were a lot of changes from version 1 to version 2 (starting actor systems, creating actors, sending messages, pretty much everything). There was a change in the API as I was going through the book.

Some of the method calls require inline Runnables or Callables, and that gets kind of awkward. I think I will move on to other things and come back when there is another edition of the book. Or I might wait until Java 8 comes out. I think the closures will make things easier.

Plus, since Akka is written in Scala, a lot of things are unwieldy in Java. I really do not like Scala. It has the syntactical elegance of Perl, with the simplicity and focus of C++. It is like religion or cigarettes: Its advocates tell me that if I gave it a chance I would see how wonderful it is. I did, and it’s not.

That said, I have no issue with the idea of using something written in Scala. I might take a look at the Play Framework. That is not as odd as it seems. I do not like working with C code. But a LOT of important stuff is written in C: The JVM, MRI, databases, the Linux kernel. If I refused to work with anything written in C, I would need to find a new line of work.

One language I think I will take a closer look at is Clojure. At first I was skeptical about it: It’s just another Lisp, and Lisp has never caught on, so why bother? But I have kept an eye on it, and I think this might be The Lisp That Sticks.

Plus, I have noticed something interesting about Lisp. A lot of people will start out developing in one language, and then they will become dissatisfied, and move to another. And then another. And then another. But it seems like when people find Lisp, they stop moving around. They seem to find what they are looking for.

In the next few months there are some Clojure meetups that will go over some Clojure web frameworks. I might look at them before that.

When Ken Kousen was here for the Groovy meetup, he said that he had a boss who kept saying, “Oh, we could do that in Lisp 20 years ago.” He thought it was really annoying. He said it was more annoying when he realized his boss was right.

Image from the Rheinau Psalter, a 13th century manuscript housed at Central Library of Zurich. Image from e-Codices. This image is assumed to be allowed under Fair Use.

Concurrency Without Pain in Pure Java

Here are the notes I took when Venkat Subramaniam was here for Austin JUG. I will mostly present them as I typed them.

Why Concurrency? Why do we care?

We now have mutli-core processors. Things that work well on one core may not work well on multiple cores.

The Java memory model controls how info goes back and forth from main memory and working memory.

His arguments in his classes are all final.

In the account class. Should the getBalance() be synchronized?

“synchronized” gets a lock and crosses the memory barrier, working and main memory.

Accessing a volatile variable, wait, join, a few other calls where you cross the memory barrier. We could compromise the invariant, like during a constructor.

So if you do not make getBalance synchronized, you might get an old value.

The code could have deadlock, or livelock (you wait, but nothing waits for you).

When you call synchronized, you might wait forever. There is no time out. So you use the Lock interface.

So for ReentrantLock you could call tryLock, or throw the Exception, then you have to unlock it.

You can order the accounts by ID to removed threat of dead lock.

95% of Java code is broken for concurrency. (Is he including code for Groovy, Scala, Clojure?)

Synchronize and suffer model.

Solutions: Software Transactional Memory (from Clojure) and Actors (Scala)

STM: We modify a variable (data) in our code. Shared mutability is bad. A few solutions: Avoid mutability. STM says let’s have a special type of variable: Managed mutable variable. It will only change in certain circumstances: a transaction.

Identity: Values are immutable. The stock price at 1:00 is different than at 1:10. But the 1:00 price will never change.

The identity is “google stock price”, points to a different value at different times.

We must protect the identity.

The problem in Java is that many times the problems happen without warning or error message. In STM, you get an error message.

This changes it in a transaction. If there are conflicts, the transaction is aborted and retried. Up to 100,000 times.

Can we do it in Java? He used ref and dosync, so can we use them in Java?

clojure.lang.Ref. Look at his code, see what class he uses for Ref.

The set method will use Lambda expression in Java 8. For now, we could do a Callable. LockingTransaction.runInTransaction

He is using an inline Callable.call()

What if you try to put more in the to account than is the from amount? It will throw an exception.

When to use STM? When you have multiple readers, multiple writers, and infrequent write collision. Not too many things are good for high-write collision.

You should not modify unmanaged variables in a transaction. Do not write to a file, print out messages, do not send email.

What about a long-running task

Another solution to shared mutability: isolated mutability.

Problem with encapsulation: Many threads can call the methods that change your private variables.

It is like voice mail: Many people can leave him a message, he gets a lot of messages, he goes through them one at a time. Actors have built-in message queue. Asynchronous messages.

The Actor runs on a separate thread.

When do actors make sense?

The fields are immutable. The messages are not, so they should be immutable.

You can have variables in an Actor, since only one thread will touch an Actor.

When does Actor make sense: Single threaded tasks sequentially

It punishes readers as much as writers. In STM, readers are never blocked. Writers are punished.

Purpose of Actor is not to deal with shared mutability, you want to prevent concurrent access, you want to serialize.

New concurrency API in Java 8.

You need Java 8 with Lambdas.

Look at java.util.streams.*

http://download.java.net/lambda/b78/docs/api/java/util/stream/package-summary.html
:: is a method reference

can be done as

Java is getting more functional

Synchronize and shared mutability can be avoided

JDK 5 concurrency is the assembly code of concurrency

Image from the Rheinau Psalter, a 13th century manuscript housed at Central Library of Zurich. Image from e-Codices. This image is assumed to be allowed under Fair Use.

 

2012-09-04 Thoughts on Clojure

I am looking at Clojure more these days. Right now I am going through the Clojure lectures on Pluralsight. I came for the concurrency. I stayed for the elegance.

From what I have heard, it handles concurrency better than most other languages. At Lone Star Ruby Conf, a few presenters talked about how Ruby will handle the coming “concurrency freight train”. Robert Martin said that we should look at a purely functional language. I think that was a hint to look at Clojure and avoid Scala.

I am looking at a great site called Clojure Docs, which I heard about from Colin Jones. It has better documentation than the official Clojure site. The docs at cloure.org for ->> are not useful at all. And good luck googling that.

A few weeks ago I attended a presentation at Geekfest about logic programming in Clojure. One of the examples the presenter gave was about chess. He programmed the rules of chess in his application. From there, you could arrange some pieces, and the applicaiton could tell you if that particular arrangement could have resulted from a legitimate game of chess, and what the moves would have been, and also what could happen going forward.

Someone asked me why you couldn’t do that with a rule engine, and honestly I did not have an answer.

But still, I can see how logic programming could be useful for scientific research. You could say that some material could handle X pounds of pressure per square inch. Good stuff.

Image from World Digital Library, assumed allowed under Fair Use. Image from the Ashburnham Pentateuch, or Tours Pentateuch, a Latin manuscript of the first five books of the Old Testament from the 6th century or 7th century. Its place of origin is unknown.

Looking at Clojure Koans

I worked on the Clojure koans.

The sixth file had this function:

Then one of the problems was like this;

And the answer was this:

I admit that I copied and pasted it.

I am having a hard time figuring this one out. If you call square, you would do it like this:

But in the answer, “square” is at the end. It seems like to get inline functions with higher-order functions you have to alternate between reading left-to-right and right-to-left. So I will try to parse it myself.

Here is the function:

((fn [f] (f 5)) square)

So let’s get started.

So first we have the inline function:

(fn [f] (f 5))

So we are declaring a function that takes an argument of 5. And so “square” is the argument. I guess that even though the name of the function is a string, that if the string matches a function name then Clojure will realize it’s a function.

So let’s see what might happen with this code runs (keep in mind I have not actually looked at the Clojure source code or tried to run any sort of debugger; I am not sure if Clojure has or needs a debugger).

Here is the function again:

((fn [f] (f 5)) square)

So let’s get started.

Let’s put the argument into the definition:

(fn [square] (f 5)) 

Now let’s put the argument into the function body:

(square 5)

Somehow this exercise is not as enlightening as I thought it would be. I don’t know if this sequence could be used by someone to figure out the koan.

To paraphrase Barbi: Clojure is hard.

 

Notes From No Fluff Just Stuff November 2011

I attended the No Fluff Just Stuff conference here in Chicago back in November 2011. I took  a few notes on some of the lectures. Here are my notes on some of the lectures on concurrency. I may post in a while with notes from more lectures, as well as some more thoughts on concurrency.

Venkat: concuurrency without pain in pure Java
Why Concurrency?
Java gave us a consistent API for multi-threading
Errors are worse on multi-core machines – multiple levels of cache in application
To get concurrency correct, you have to be God.
That model is the synchronize and suffer model
2 other models: Software Transaction Memory and Actors

Synchronized helps you cross the memory barrier synchronizing main and working memory
Locking and race condition is only part of the story
Do not synchronize the accounts, synchronize the transactions
You should also synchronize in the class using the Account classes

There could be livelock or deadlock
But you could wait a long time.
But you cannot tell synchronized() to timeout.

So instead in Account, use a ReentrantLock
in Account.deposit method, use ReentrantLock.tryLock(long, TimeUnit)

This makes code more complex. synchronized blocks are pretty easy.
Don’t forget to call Reentrant.unlock()
But what if there is an exception before that? Put it in a finally.
This is why we don’t use raw JDBC

Deadlock: I have lock on A, I need a lock on B. You have a lock on B, you need a lock on A.
You could impose an order
If someone introduces a new method, all your locking could be ruined. It’s like a bathroom with a lot of doors. You got to lock every one.
Concurrency on JDK is like a mother-in-law: It’s waiting for you to fail.
95% of Java is broken for concurrency, and we don’t know it yet.

STM: Software Transaction Memory
Everything is immutable (almost everything)
Managed Mutable Identity is also in Clojure
Sharing is good, mutability is good, shared mutability is bad
Separates state from identity
identity points to state
Like stock price: $600 at 1:00 PM, $610 at 1:10
The price at 1PM on November 11, 2011 will always be $600
We can flip a switch to point identity to a new value
Immutability means no locks

the problem is not that it is failing, but that it is misbehaving quietly and you do not know it.
MMI fails loudly if you mess with it outside of a transaction
transaction thread runs in a cone of silence
final change is visible to outside world

If there is a collision, a transaction could repeat
Clojure is written in Java – it’s another JAR, and you can use it in Java

change int in Account to clojure.lang.Ref

int will be replaced with java.lang.Integer
You can only put immutable stuff in there.
Also

Clojure:

You could still have race conditions. The AccountService that does the transfer still needs a transaction to wrap around the transactions in the Account classes
A transaction in STM could repeat 100,000 times – so do not print, send email

STM: we don’t see any locks (so no deadlocks)
2 drawbacks:
1. STM requires state be immutable – greate for Clojure – the only language where concurrency is safe
2. Code could ge slow or fail due to retries
Hard to be functional in Java
Another option: Isolated mutability – only one thread can touch it at a time
Actors and active objects
An actor can receive many messages at once, but they only work on one at a time
Akka library – now we use Scala’s implementation
Akka gives us STM and Actors

So we passed a message. We did not call a method directly.
Actors do not hold Threads hostage
Actors could fail
Each Actor is sequential
Could deadlock with 2-way messaging

JDK concurrency APIs are assembly language of concurrency

But can Java web developers do this?

Ted Neward Guava:
Get rid of mutable state

Image from  Codex Amiatinus, a 6th century Vulgate manuscript housed at the Laurentian Library in Florence. Image from Wikimedia. This image is assumed to be allowed under Fair Use.

Programming Clojure

CJUG Downtown Presentation: 2009-08-05.

Clojure is a dynamic language for the Java Virtual Machine, with a compelling combination of features:

* Clojure is elegant. Clojure’s clean, careful design lets you write programs that get right to the essence of a problem, without a lot of clutter and ceremony.
* Clojure is Lisp reloaded. Clojure has the power inherent in Lisp, but is not constrained by the history of Lisp.
* Clojure is a functional language. Data structures are immutable, and functions tend to be side-effect free. This makes it easier to write correct programs, and to compose large programs from smaller ones.
* Clojure is concurrent. Rather than error-prone locking, Clojure provides software transactional memory.
* Clojure embraces Java. Calling from Clojure to Java is direct, and goes through no translation layer.
* Clojure is fast. Wherever you need it, you can get the exact same performance that you could get from hand-written Java code.

Many other languages offer some of these features, but the combination of them all makes Clojure sparkle.

Presenter: StuartHalloway

Stuart Halloway is the CEO of Relevance, Inc. www.thinkrelevance.com. With co-founder Justin Gehtland, Stuart helps companies adopt agile, as well as innovative technologies such as Clojure and Ruby on Rails. Stuart is the author of Programming Clojure, Rails for Java Developers, and Component Development for the Java Platform. Prior to founding Relevance, Stuart was the Chief Architect at Near-Time, and the Chief Technical Officer at DevelopMentor.

You can follow Stuart on Twitter.

 

Image from Cognitect website, assumed allowed under Fair Use.