2017-11-06 Update: Deeplearning4J

One thing I did this weekend was get permalinks working again on my blog. I got an SSL certificate from Let’s Encrypt several months ago, and I was not able to get permalinks to work. It bothered me for the longest time. Problem solved.

Not much on the Clojure API front.

I did look at Deeplearning4J, a deep learning framework written in Java. I cloned their repo of examples, and I got some of them working with gradle files, and learned a bit more about Gradle in the process. You can use a GPU, or a CPU. No messing with CUDA if you don’t want to.

There is even a Clojure wrapper for Deeplearning4J as well.

I had a post about using CPUs and GPUs from JVM languages for math/science/AI a while back. Since then, I have gotten both Neanderthal and Deeplearning4J to work on my system. I did mention Bytedeco, which is used by Deeplearning4J and a few other libraries. I looked at some of the Javadoc, and they use a LOT of static methods (see here for an example). I may leave Bytedeco for others and use some of the higher-level libraries, like Deeplearning4J. I think a lot of static methods is not idiomatic Java. I might consult “Effective Java” later for a second opinion.

I am going to include below one of the build.gradle files that I used to get some of the Deeplearning4J examples to work.

You’re welcome.


2017-10-01 Update

I am still going through Simply Scheme. I am on chapter 8.

I may also start going through the Clojure API. I have tried 4Clojure a few times, and I tried the Kotlin Koans. I don’t think that is a good way for me to learn. I have a Kotlin book that I am looking at. I have thought about going through the functions on the Clojure Cheat Sheet, but I would need some way to keep track of what I have done. I made a page with a list of the functions in the core Clojure API for version 1.8. I have thought about this for a while, but recently I found a page called “One Weird Trick To Become a Clojure Expert“, which advocates going through the API. I may not go alphabetically.

Plus, unlike the Kotlin API, the core Clojure API is small enough that you can actually get through it without feeling overwhelmed.

I also found out that lein and boot do not work with JDK 1.9 on Ubuntu. I gathered from web searches that Clojure 1.9 will solve the issue. It seems to work fine on Windows 10.

You’re welcome.


This week I worked more on Simply Scheme. I got done with Chapter Six. Chapter Seven looks pretty short.

The Austin Clojure group had a code get-together at Cafe Express. There were only a few of us there. I worked on Simply Scheme.

One thing a lot of veteran Lisp/Scheme people do not like about Clojure is that it uses other characters for data structures (vectors, maps, sets) in places where other Lisps just use parentheses. I think I agree with Clojure for changing things a bit. I think it makes things more clear. There was a line about this topic in Chapter 7 of Simply Scheme that made me think about this:

Like cond, let uses parentheses both with the usual meaning (invoking a procedure) and to group sub-arguments that belong together.

There was a lot of discussion about JavaScript at the Clojure Meetup this weekend. I generally do not like anything related to JavaScript, not even ClojureScript. I have heard great things about it, and that David Nolen is a really smart guy, but it just seems like if you use ClojureScript, you will probably have to deal with a bunch of JavaScript anyway. So for now, I am not too eager to do anything mobile with Clojure. It seems like ReactNative still has a bad license, and you have to deal with Node on top of it all.

A couple of guys mentioned WebAssembly. I first read about that on the Racket mailing list in a thread in which a few people (myself included) were lamenting the inescapability of JavaScript. I think it is a drag that a lot of people want to use JavaScript everywhere. I would like an actual alternative (not just yet another *js framework, or a language that transpiles to JS). Will that be WebAssembly?

Someone on Hacker News somewhere warned that if a site uses WebAssembly, a site could prevent you from blocking ads if they really wanted to force you to see them. Stay tuned.

You’re welcome.

2017-09-10 Update: Uncomplicate Neanderthal

This weekend I made another attempt at some numeric programming. I have been working on a Clojure library named Neanderthal. It can use CUDA, OpenCL or native libraries. Instead of using BLAS and LAPACK for native libraries, it uses the Intel Math Kernel Library.

In order to use the native library, I installed the Intel MKL library. I did have a bit of trouble with the library. This worked:

export LD_LIBRARY_PATH=/home/ericm/intel/compilers_and_libraries_2017.4.196/linux/mkl/lib/intel64_lin/

For some reason, the “export” had to be on the front. I tried without, and I got an error. But I was able to copy and paste the code from the native example here.

I was able to get the code running from the first OpenCL example.

These are some packages I installed to get the OpenCL example to work:

aptitude install ocl-icd-opencl-dev
aptitude install dkms
aptitude install beignet-opencl-icd

I will also look at TensorFlow. There are a few libraries that let you use Clojure.

2017-12-21_01.25.32 update: Based on an email from the library author on the mailing list, this is how you run the native tests and skip the CUDA/OpenGL tests:

Just run mkl_test namespace. Note: these are mije tests, so for example in Leiningen:

lein midje uncomplicate.neanderthal.mkl_test

Or, with the REPL running, just evaluate mkl_test namespace.

You’re welcome.

Thoughts On Native, GPU, Groovy, Java and Clojure

One interest of mine is math and scientific computing, although I admit right now my interest is far greater than my knowledge.

It seems like for a lot of math and scientific computing, you either need to do stuff in C, C++ or even Fortran, or use a library in a higher-level language that lets you interface with libraries in C/C++/Fortran. Another new trend is to use code that runs on the GPU, like OpenCL.

Lately I have been looking at a few Java libraries that interface with lower level code. I think it is a good idea to keep learning new things. I think AI and machine learning will become more important over time.

I contributed briefly to the SciRuby project a few years ago. One of their main subprojects is a Ruby wrapper around BLAS, ATLAS and LAPACK. I think some of the instructions for compiling BLAS, ATLAS and LAPACK might still have a few sentences written by yours truly. (It’s a very involved process.) Another one of their projects is a Ruby wrapper/interface around the GNU Scientific Library, or GSL.

One huge language in AI and data science these days is Python, with the NumPy and SciPy libraries. Again, some of the functionality is provided by linking to ATLAS and LAPACK. If it worked for them, maybe it will work for the JVM family of languages.

There is at least one project in Clojure that does this: Uncomplicate. It has sub-projects for OpenCL and BLAS and ATLAS. Along with Java and Groovy, Clojure is a language I am interested in. But for the time being I cannot use these projects. I think my graphics card is a bit too old and the drivers are too old for many of these libraries.

Uncomplicate is part of the reason I am looking at this area again. Somewhere on the site, the author points at the for development, it is okay to use pre-baked packages for ATLAS and LAPACK. One of these projects that I mention says for the best performance, you may want to hire someone to install ATLAS and LAPACK and other native libraries optimized for your production hardware. I did not know people made a living doing just that.

Another reason I am looking at this is GPU computing was mentioned in one of the side talks at the most recent ClojureConj. This is second- or third-hand, but apparently Goldman Sachs runs a nightly report analyzing the market (or some part of it, I am not too clear). It used to take nine hours to run. It was refactored to run on the GPU, and now it takes 20 minutes. I think all developers should become familiar with some of these technologies.

Will any of this gain traction in the Groovy space? I know things have been rough for Groovy for the past couple of years, especially after getting dropped by Pivotal. It seems like a lot of Groovy developers are happy making web apps with Grails. I like Grails, and it has worked out well for a lot of people. I will go through the Groovy Podcast backlog; perhaps Groovy developers are looking into this and I am just not aware of it.

Groovy developers can use the Java wrappers for native and GPU computing. No need to re-invent the wheel. Maybe going from Groovy to Java to native is not quite as fast as going from Python or Ruby to native, but I don’t think so. Even Ruby developers joke that Ruby is pretty slow. They say that sometimes the speed of the programmer is more important than the speed of the program. (But for some reason the Ruby devs who are moving to Elixir are now saying performance and concurrency are important after years of saying they were not. Whatevs.)

I do not have much experience in native or GPU programming. I am not too clear how NumPy and SciPy are used. I don’t want to swallow the Python ecosystem to get the answers to just a few questions, but: Are NumPy and SciPy programs (or bare-metal math/science apps in general) rewritten frequently? Or do they stay the same for years at a time? It seems to me if a GPU program does not change, you might be better off biting the bullet and doing things in C or C++.

But given the success of NumPy and SciPy, perhaps Groovy and Java can be used in this space. Maybe it’s not the end of the world if a bank’s nightly report takes 22 minutes. (How successful is SciPy? Not only are there conventions for general Python, there are also conferences for just SciPy). I doubt any technology will dethrone Python anytime soon, but it is an area to look at.

Having looked a small bit at some of these libraries, I do wonder how much can the unique features of Groovy be used. A lot of the tests deal with arrays of primitive types.  Someone on the Groovy mailing list wrote: I’ve found that you can really shrink down the lines of code just by using the basics that Groovy provides around collections, operator overloading, Groovy truthiness, etc.. Maybe that is enough reason to use Groovy for high performance computing (or at least higher performance). Can we make code calling native and GPU libraries better with features from Groovy, or will we just get Java without semicolons? Some of these functions do not want an ArrayList of objects, they want a fixed-length array of integers. Granted, you can make primitive arrays in Groovy:

but I think that is more keystrokes than in Java.

Just looking at a small part of TensorFlow, here is a line from the original Java test:

Here are two Groovy versions that both work (meaning the tests still pass):

Granted, it’s one line, but I’d say that’s idiomatic Groovy. Perhaps this could be a (small) market opportunity for the Groovy community. Or maybe I’m just crazy.

If you google “groovy jni” the first hit is this page from Object Partners. It mentions JNA and BridJ which I think are used by a lot of the libraries mentioned below. Frankly, it sounds like a lot of work making this sort of thing happen.

Regardless, even if I have to stick with Java and Clojure, I will still keep an eye on these libraries and this space. I don’t think I can become an expert in all of these, but as I said, I think it is important for developers to keep an eye on this spae. I might start some pages about the different libraries, and perhaps share my thoughts on them. Below are a few notes on what I have found so far.

I started by googling “Java BLAS” or “Java GPU”, and at first I only got a few libraries. But as I kept looking, I kept finding more.

The first two I found where jblas and Aparapi. From the jblas website, jblas “is essentially a light-wight wrapper around the BLAS and LAPACK routines.” (And from what I can gather, the correct capitalization is “jblas”.)

Aparapi is a wrapper around GPU and OpenCL code. If you cannot get your GPU drivers installed properly, it will do everything on the CPU. Sometimes I get core dumps running the tests on my Ubuntu laptop. It is about six years old, and has 6 GB of memory, so perhaps that is the issue. But on my 8 GB Windows laptop two of the tests fail. I plan on getting a new laptop soon, so I will try this again when I do. I bet getting all this set up on an Apple laptop is really easy; for the first time in my life I am thinking about buying an Apple product.

TensorFlow has some Java APIs, although they warn that there may be API changes and breakages for languages other than Python and C. This can use either the CPU or the GPU, but for GPUs it can only work with NVidia cards.

The Lightweight Java Game Library works with OpenCL, OpenGL, Vulkan, EGL and a lot of other stuff I have never heard of. As the name suggests, it is primarily for game development, and the wrappers around the underlying native libraries and standards require more knowledge of those underlying native libraries and standards than most libraries.

There is ND4J (N-Dimensional Arrays for Java) and Deeplearning4j. I am not sure if they are related somehow, but they were both started by developers at a company called Skymind.

And then there is the granddaddy: ByteDeco. There is a lot of stuff here. Some of the contributors work on ND4J and Deeplearning4j, and also work at Skymind. This project provides Java interfaces to 21 different C/C++ libraries: video, math, science, AI, robotics, facial recognition, lots of good stuff here. And after looking at their list, yes, I think “Skymind” is a bit too close to “Skynet”. But at least they’re giving you a hint.

You’re welcome.

boot-new Makes Boot Usable

As I mentioned, I am going through Functional Programming in Java. I have a couple of github sub-repos for it. I am going through the Java code, and then re-implementing it in Groovy and Clojure. The Java and Groovy parts are here inside a repo called “my_groovy_apps_001”, and the Clojure part is here inside a repo called “clojure_apps”.

I started doing the Clojure inside the Gradle app inside “my_groovy_apps_001” using the Neptune Gradle plugin for Clojure. It was okay, but I could not get the REPL to work, and I could not figure out how to run only some of the tests.

So I decided this might be a good time to learn Clojure Boot, the new build tool that is an alternative to Leiningen and seems to be gaining some momentum.

Frankly, the docs are terrible. You kind of need the docs to do anything since it doesn’t do a lot out of the box, kind of like Eclipse when it was first released. Until plug-ins were developed, Eclipse was not very useful, and Boot seems the same to me at this point. Boot doesn’t even have a way to run tests out of the box.

One reason I think the docs are terrible is that in the “Getting Started” section on the Boot github page, they redirect you to a ClojureScript tutorial, telling you that it is “an excellent introduction to Boot and ClojureScript”.

I really have no interest in ClojureScript at this time. I do not like Javascript, and while I have heard great things about ClojureScript and its creators/maintainers, I want to avoid the JavaScript swamp as much as possible. As the guy who wrote the Pollen DSL for Racket put it, Lisp makes me smarter, and pretty much any language ending in *Script seems to kill brain cells on contact. The fact that there are so many languages that compile to JavaScript to me is not a good sign for JavaScript. It says to me that people deal with JavaScript because they have to, not because they want to. And that people will go to great lengths to avoid dealing with JavaScript directly. And since the JavaScript community keeps spawning new frameworks all the time, I think on some level they agree. I am sure at some point I will have to learn about ClojureScript, but I would like to put that off for now.

But it’s not just me being anti-anythingScript here. I like the Unix philosophy of a tool does one thing and does it well. I want to learn about Boot. Not ClojureScript. I don’t want to wade through a tutorial for a dessert topping to find the bits about the floor wax.

So I did some searching for some blog posts about how to get started with Boot. I found some by a guy named Sean Cornfield (whom I actually met at ClojureConj). He developed a plug-in called boot-new that pretty much does everything you need to get started using Boot. I would argue that you don’t even need a tutorial for Boot. Just use boot-new.

All you have to do is install Boot, and then run this command:

It will fetch boot-new from Clojars (I assume) and make a project called myapp.

One of the things that boot-new allows you to do is to run tests for specific namespaces. You can run just one:

or multiple namespaces separated by commas:

With “–namespaces”, you specify the namespace of the tests. To specify the namespace of the function, use –include:

So if anyone in your local Clojure group wants a presentation on Boot, you can just tell them to look at boot-new. It would be the shortest presentation ever.

I don’t know how to do all the things that Leiningen does, but if you just want to start a project, compile it, run a REPL, run some tests and get a JAR, I think boot-new covers some of the basics. Correctly or incorrectly, it seems like a lot of people (including me) think of Boot as an alternative to Leiningen. So my approach is basically, “I know how to do X in Leiningen, how do I do that in Boot?”

For a lot of basic tasks, boot-new is your answer.

You’re welcome.

Yet Another Change In Plans

I am getting a bit frustrated with Realm Of Racket.

I am going through chapter 8. I downloaded their source from github and tried to run it locally. It does not work for some reason. It works if I clone the repo and run it from there. But it should still run somewhere else. I have the same directory structure as they do. I really don’t feel like debugging it.

I couldn’t get their example from chapter 6 to work either.

I will try to run some Scheme in DrRacket. If that does not go well, I will just go back to Clojure.

You’re welcome.

Clojure Koan 15: Destructuring

I am working through the Clojure koans.

I found a guy who did them and put his answers on github (username viebel), and when I am done I check mine against his. I found out you can also look at resources/koans.clj for answers, but they are formatting oddly in that file.

Koan 15 is about destructuring. Some of the problems use a map that is provided in the beginning:

Here is the original problem:

Here is my answer:

Here is the more elegant answer:

So they are taking the vector argument and destructuring it in the argument list itself. And destructuring the map right after it, making two arguments into five. Viebel has a few interesting ways to destructure that I did not know about.

I might replace my answers with the provided answers. Now when I run it, lein takes a relatively long time to get through koan 15.

You’re welcome.

Back To Clojure

My Plan To Be The Smartest Person Ever is changing yet again. I have decided I will focus on Clojure for the foreseeable future.

I was at the Austin Clojure Meetup last Saturday, and I spoke with another member who I first met when I lived in Chicago, and was a gung-ho Ruby person for quite some time. Over the past year, this person has decided to look into Clojure. They interviewed recently for a Ruby position and a Clojure position. They got stumped on both interviews, and decided that they wanted to go all-in with Clojure, and recommended that I do the same.

They also recommended I get a subscription to Purely Functional TV. It seems like the whole point of that site is to help people get Clojure jobs.

I thought about it, and I agree with them. I would like to get a job in some form of Lisp/Scheme, and spend the rest of my life as an Enlightened Lisp Developer. The language that is most likely to make that happen quickly is Clojure.

One thing that cemented my decision is I went to a heated Vinyasa yoga class this weekend, and I realized I am not as fit as I wish I was, and I am not where I need or want to be in any area of my life.

(But I might still work on my Groovy Mail Server.)

You’re welcome.