Category Archives: Groovy

2017-07-16 Update

I took a couple of weeks off from work, so I did not get much done over the past two weeks.

At my company, “vacations” is now “PTO”: paid time off. I have noticed a few people I know at other companies who also use this terminology. When did “vacation” get replaced with a corporate term?

Anyway, I did go to the local Processing meetup. I did not stay the whole time. I noticed I did not have my wallet, so I left early to go look for it. It turns out I just left it at home.

I did a bit with Simply Scheme. I am now on chapter 6. I am wondering if/when I will get the “Lisp enlightenment” that will make me better in all things CS. It looks like there are things you can do in Scheme and Lisp that are just impossible in other languages. We shall see.

I did spend some time on the Groovy Email Server. I am keeping track of the ip addresses and host names of all incoming requests. I might be able to put this to good use someday.

You’re welcome.

2017-05-21 Update

The big news in the JDK space this week is that Android now supports Kotlin. See this page, on the Android site and this post on Hacker News. So I will start learning Kotlin.

I don’t want to turn into one of those Scala weenies who goes on about how much they hate Java (see Note 1 below), but Android development was always pretty ghastly. Lots of XML, lots of anonymous inner classes (at least the last time I looked).

When Java started, it was basically: We took C++, and removed the hard parts you don’t like. Android is pretty much: We are using Java, but mostly the hard parts you don’t like.

I might look into React Native and ClojureScript at some point. But for now I am going through the Kotlin koans. Back when I was living in Chicago (back when Google told you to use Eclipse for Android), I spoke to a mobile developer who wrote apps for both Android and iDrone. He did not use any of the Javascript wrappers. He said the best way to do it was to just do it the way Google and Apple tell you to do it.

Maybe I will love Kotlin, but to be upfront I am not too happy with it at the moment. It is kind of a competitor to Groovy. I have spent a lot of time with Groovy, and I wish that it had more traction. But the world isn’t going to change for me. A lot of people at my job think every gosh darn thing has to be in a Word file. We make a web app for our client, why not make one for ourselves? But whatever.

There is no Kotlin group in Austin at the moment. Officially, there is a Groovy group, but there hasn’t been a meeting since January. The past two companies that hosted are no longer give the group the space. The group organizer works for OCI, the company that supports Grails. He works with clients all over Texas. I guess there is more Groovy out there than people think. Groovy is one of those languages where the user group is full of people who think the language is great, but their companies won’t even look at it. At the last meetup when he said he works with companies large and small all over Texas,  everyone got pretty excited. But he couldn’t give out any more info because of NDAs.

I know this sounds like I am not oozing with enthusiasm about leaving a language I like for one that I do not know very well. Frankly, at this point, I am not. But it looks like this is where the world is going. Kotlin had already gained a lot of momentum on Android before this (more than Groovy, certainly). If Google is willing to listen to its developers, then maybe Google is a vendor to pay attention to. Unlike Shiny-Wrapper-Microsoft (aka Apple).

Kotlin does have some similarities to Scala, which I do not like, so we will see how this all goes.

So, not a whole lot on the Clojure front.

I am still going through Simply Scheme.

You’re welcome.

Note 1: I always found Scala developers’ hatred of Java odd for two reasons: 1. The language you say you love was/is implemented in the language you say you hate, and 2. Why does Javascript get a complete pass from everybody?

Thoughts On Native, GPU, Groovy, Java and Clojure

One interest of mine is math and scientific computing, although I admit right now my interest is far greater than my knowledge.

It seems like for a lot of math and scientific computing, you either need to do stuff in C, C++ or even Fortran, or use a library in a higher-level language that lets you interface with libraries in C/C++/Fortran. Another new trend is to use code that runs on the GPU, like OpenCL.

Lately I have been looking at a few Java libraries that interface with lower level code. I think it is a good idea to keep learning new things. I think AI and machine learning will become more important over time.

I contributed briefly to the SciRuby project a few years ago. One of their main subprojects is a Ruby wrapper around BLAS, ATLAS and LAPACK. I think some of the instructions for compiling BLAS, ATLAS and LAPACK might still have a few sentences written by yours truly. (It’s a very involved process.) Another one of their projects is a Ruby wrapper/interface around the GNU Scientific Library, or GSL.

One huge language in AI and data science these days is Python, with the NumPy and SciPy libraries. Again, some of the functionality is provided by linking to ATLAS and LAPACK. If it worked for them, maybe it will work for the JVM family of languages.

There is at least one project in Clojure that does this: Uncomplicate. It has sub-projects for OpenCL and BLAS and ATLAS. Along with Java and Groovy, Clojure is a language I am interested in. But for the time being I cannot use these projects. I think my graphics card is a bit too old and the drivers are too old for many of these libraries.

Uncomplicate is part of the reason I am looking at this area again. Somewhere on the site, the author points at the for development, it is okay to use pre-baked packages for ATLAS and LAPACK. One of these projects that I mention says for the best performance, you may want to hire someone to install ATLAS and LAPACK and other native libraries optimized for your production hardware. I did not know people made a living doing just that.

Another reason I am looking at this is GPU computing was mentioned in one of the side talks at the most recent ClojureConj. This is second- or third-hand, but apparently Goldman Sachs runs a nightly report analyzing the market (or some part of it, I am not too clear). It used to take nine hours to run. It was refactored to run on the GPU, and now it takes 20 minutes. I think all developers should become familiar with some of these technologies.

Will any of this gain traction in the Groovy space? I know things have been rough for Groovy for the past couple of years, especially after getting dropped by Pivotal. It seems like a lot of Groovy developers are happy making web apps with Grails. I like Grails, and it has worked out well for a lot of people. I will go through the Groovy Podcast backlog; perhaps Groovy developers are looking into this and I am just not aware of it.

Groovy developers can use the Java wrappers for native and GPU computing. No need to re-invent the wheel. Maybe going from Groovy to Java to native is not quite as fast as going from Python or Ruby to native, but I don’t think so. Even Ruby developers joke that Ruby is pretty slow. They say that sometimes the speed of the programmer is more important than the speed of the program. (But for some reason the Ruby devs who are moving to Elixir are now saying performance and concurrency are important after years of saying they were not. Whatevs.)

I do not have much experience in native or GPU programming. I am not too clear how NumPy and SciPy are used. I don’t want to swallow the Python ecosystem to get the answers to just a few questions, but: Are NumPy and SciPy programs (or bare-metal math/science apps in general) rewritten frequently? Or do they stay the same for years at a time? It seems to me if a GPU program does not change, you might be better off biting the bullet and doing things in C or C++.

But given the success of NumPy and SciPy, perhaps Groovy and Java can be used in this space. Maybe it’s not the end of the world if a bank’s nightly report takes 22 minutes. (How successful is SciPy? Not only are there conventions for general Python, there are also conferences for just SciPy). I doubt any technology will dethrone Python anytime soon, but it is an area to look at.

Having looked a small bit at some of these libraries, I do wonder how much can the unique features of Groovy be used. A lot of the tests deal with arrays of primitive types.  Someone on the Groovy mailing list wrote: I’ve found that you can really shrink down the lines of code just by using the basics that Groovy provides around collections, operator overloading, Groovy truthiness, etc.. Maybe that is enough reason to use Groovy for high performance computing (or at least higher performance). Can we make code calling native and GPU libraries better with features from Groovy, or will we just get Java without semicolons? Some of these functions do not want an ArrayList of objects, they want a fixed-length array of integers. Granted, you can make primitive arrays in Groovy:

but I think that is more keystrokes than in Java.

Just looking at a small part of TensorFlow, here is a line from the original Java test:

Here are two Groovy versions that both work (meaning the tests still pass):

Granted, it’s one line, but I’d say that’s idiomatic Groovy. Perhaps this could be a (small) market opportunity for the Groovy community. Or maybe I’m just crazy.

If you google “groovy jni” the first hit is this page from Object Partners. It mentions JNA and BridJ which I think are used by a lot of the libraries mentioned below. Frankly, it sounds like a lot of work making this sort of thing happen.

Regardless, even if I have to stick with Java and Clojure, I will still keep an eye on these libraries and this space. I don’t think I can become an expert in all of these, but as I said, I think it is important for developers to keep an eye on this spae. I might start some pages about the different libraries, and perhaps share my thoughts on them. Below are a few notes on what I have found so far.

I started by googling “Java BLAS” or “Java GPU”, and at first I only got a few libraries. But as I kept looking, I kept finding more.

The first two I found where jblas and Aparapi. From the jblas website, jblas “is essentially a light-wight wrapper around the BLAS and LAPACK routines.” (And from what I can gather, the correct capitalization is “jblas”.)

Aparapi is a wrapper around GPU and OpenCL code. If you cannot get your GPU drivers installed properly, it will do everything on the CPU. Sometimes I get core dumps running the tests on my Ubuntu laptop. It is about six years old, and has 6 GB of memory, so perhaps that is the issue. But on my 8 GB Windows laptop two of the tests fail. I plan on getting a new laptop soon, so I will try this again when I do. I bet getting all this set up on an Apple laptop is really easy; for the first time in my life I am thinking about buying an Apple product.

TensorFlow has some Java APIs, although they warn that there may be API changes and breakages for languages other than Python and C. This can use either the CPU or the GPU, but for GPUs it can only work with NVidia cards.

The Lightweight Java Game Library works with OpenCL, OpenGL, Vulkan, EGL and a lot of other stuff I have never heard of. As the name suggests, it is primarily for game development, and the wrappers around the underlying native libraries and standards require more knowledge of those underlying native libraries and standards than most libraries.

There is ND4J (N-Dimensional Arrays for Java) and Deeplearning4j. I am not sure if they are related somehow, but they were both started by developers at a company called Skymind.

And then there is the granddaddy: ByteDeco. There is a lot of stuff here. Some of the contributors work on ND4J and Deeplearning4j, and also work at Skymind. This project provides Java interfaces to 21 different C/C++ libraries: video, math, science, AI, robotics, facial recognition, lots of good stuff here. And after looking at their list, yes, I think “Skymind” is a bit too close to “Skynet”. But at least they’re giving you a hint.

You’re welcome.

2016-04-21 Update

I added an SSL cert to this site with Let’s Encrypt. So now I am secure. The only issue is that for some reason the WordPress permalinks stopped working, so I had to go back to the default URL style. I spent about 20 minutes on the google, and all the sites gave the same suggestion that did not work. Perhaps it is time to look at Nginx.

I am also working on the Groovy Mail Server. I know I keep saying I will get to Clojure, and I promise I will. I got a few things working with SSL, so I decided to keep going just a little bit longer.

You’re welcome.

Update To Groovy Validator

I made a change to the Groovy Validator project.

I changed the names of the annotations. For example, “IntAnnotation” is now “ValidInt”. The same change was made for the other types.

Here is the new README file:

This project has a few annotations that validate fields in POGOs, sort of like Grails constraints.I will attempt to make some annotations for properties in Groovy.Here is a POGO:

It’s clean, and has no getters and setters. But what I do not like is there is no validation for your data. What if you want your String to be between 10 and 20 characters? What if you want your int field to be more than 100? And what’s to stop some dingo from trying to create a book object with less than 0 pages?

So I made some annotations that can do some validation for you.

For POGOs, if a numeric field is declared as “def”, it will become null if the argument does not meet the validation constraints. If it is declared as a primitive, it will be set to 0 if the argument does not meet the validation constraints.

This project can also validate fields in immutable objects. In addition to using the annotations for the fields, you annotate the class with ImmutableValidator:

To process the annotations, put your properties in a Map, and add a boolean called “validation” and set it to true (since I couldn’t overload the Map constructor, I added a boolean):

Adding the “throwEx” will throw an exception if the arguments do not meet the validation constraints. It is optional and is set to false by default. If an exception is thrown, it will print out the value and the constraints.

You might get a message like this:

You can also use it with immutable objects annotated with the ImmutableValidator annotation. This would be a second boolean after the Map with your properties, since the first boolean controls validation:

In that case, you get a message with a line for each field. So you might get a message like this:

If “throwException” is true for an immutable object and an exception is thrown, then the object will not be created.

This library can also handle final fields in mutable objects.

As with immutable validation, you need to use a map in the constructor to validate a final field.

Right now it only handles String, double, float, int and long. For String, it checks the string is checked against a minimum (“minLength”) and maximum (“maxLength”) length, and against a regular expression (“regEx”). For integers and longs, the field is checked against minimum (“minValue”) and maximum (“maxValue”) values, and a set of divisors (“divisorSet”). For double and float, the field is checked against minimum (“minValue”) and maximum (“maxValue”) values. There are defaults for all of these.

To use this project: Run

and use build/libs/groovy-validator.jar in your project.

You’re welcome.

2016-02-27 Update

I am still working on my Groovy Mail Server. Security is a lot harder than I thought.

Not that I thought it would be easy. But just when I think I am getting somewhere, I realize I need to look at ANOTHER RFC.

I will have to look into the STARTTLS command, and try to get an SSL socket in my application.

Sometimes I wonder if I should keep up with this, or if I should drop it and move on to something else. Like Grails 3. Or Clojure.

Plus I have gone about security a bit wrong. I spent some time trying to get the hang of the Java SASL API to work with CRAM-MD5. Then I realized that I am storing the passwords in an SHA-512 hash.  I am not a security expert, but I do not think there is a way to compare a password with two different one-way hashed. So I might try STARTTLS and use PLAIN auth. Or try storing the passwords with MD5.

Or just go on to Luminus and Grails 3.

You’re welcome.

2016-02-22 Update

I bought Living Clojure by Carin Meier, and I am working through the second half. It seems like a nice language and a nice book. I hope to use it professionally in the future.

I also like Groovy. As Dick Wall would say, I am all about the Groovy, man. I also made some progress on the Groovy Mail Server. I made some progress on authentication using SASL. I using SASL classes in the JDK, but I had to implement some Callback handler code myself. I had some help from Stack Overflow. In general I like Java the language, but sometimes I feel like the API designers like to make things more complex than necessary.Interfaces, extensions, listeners, factories, handlers, the knee bone instantiated the thigh bone, etc, etc, etc.

One thing I found out today is that when calling static methods on a class in the Groovy shell, you need to include the full package name for the class. I prefer the shell over the console, but I may have to switch. I wish the shell could run any Groovy code as-is. After finding out about setting interpeterMode to true in the shell, I thought the shell would run everything fine. Oh well.

I also made some small changes. I was not catching a possible exception, and the code was not closing connections. When I logged into the server it had almost maxed out.

You’re welcome.

Testing Groovy Actors

One thing I did not quite get when I was looking at multithreading technologies was how to unit test them.

I have not looked at Akka for a while, but a few years ago it seemed like the consensus was to use a logger like Log4J, and then parse the logging messages. That is kind of unwieldy and feels wrong. (Akka seems to have more ways to test now.)

I emailed the GPars list about putting private fields in an actor, changing them in the onMessage method, and then checking them afterwards; something like this:

You could do that. There is nothing wrong with having private fields in an actor. You might want to keep some data between requests (like an encryption key). But making a field just for testing seems wrong.

Then I had an idea: Have your actor call another class that does the actual work, and unit test the worker class:

I suppose that will not help with functional testing. It might violate some SOLID principle, or DRY, or GRASP or the Law of Demeter or Principle of Least Astonishment or Poe’s Law,
but it seems like a pretty good idea to me.

You’re welcome.

2015-11-08 Update

I have been traveling for work, but now I am back.

I did some work on the Groovy Mail Server. I am ready to start working on sending mail. I will have to look into authentication, which will complicate things a lot.

I am thinking about going forward with a Grails 3 tutorial.

I am thinking about looking more into Common Lisp/Racket/Scheme/Clojure. I know a lot of the pragmatic programming crowd says you should learn a language every year, I am starting to question that. It seems like languages in the C/C++ family are moving more towards Lisp and Smalltalk. Why not just learn those and be done with it?

You’re welcome.

Update On Groovy Email Server 002

Here is another update on the Groovy Email Server.

After I made the last update, it stopped working. For some reason, I could retrieve messages in Alpine, yet not look at the bodies of the messages. I could view the headers. I was able to figure it out, and now I can retrieve the messages in Alpine again.

I send myself a message with a text file as an attachment, and I was able to save it.

I also tagged the git repo. I did not tag it the last time I posted, so I wasted some time downloading old versions from github to see when things were okay.

For some reason, I cannot retrieve the messages with Thunderbird. I will try another client later.

You’re welcome.