Category Archives: Technology

2017-06-18 Update

Not much going on. I am still working slowly through Simply Scheme. I am at a point where they want you to diagram some functions. Like diagramming sentences in junior high.

There was an article on Bloomberg about quantum computing (Hacker News page here). That started me doing a lot of surfing about quantum computing. IBM has the IBM Quantum Experience, which allows you to try quantum programs on their quantum cloud.

There is a wiki about QC called Quantiki. One page is a page with a list of simulators. One of them is a page called Quantum Playground. I found a couple in Java: one by a guy who goes by the name ardeleanasm, and one called JavaQuil. JavaQuil is a port of pyQuil, an API for a project called Forest, which is another QC cloud. It is run by a company called Rigetti Computing.

You’re welcome.

Thoughts On Native, GPU, Groovy, Java and Clojure

One interest of mine is math and scientific computing, although I admit right now my interest is far greater than my knowledge.

It seems like for a lot of math and scientific computing, you either need to do stuff in C, C++ or even Fortran, or use a library in a higher-level language that lets you interface with libraries in C/C++/Fortran. Another new trend is to use code that runs on the GPU, like OpenCL.

Lately I have been looking at a few Java libraries that interface with lower level code. I think it is a good idea to keep learning new things. I think AI and machine learning will become more important over time.

I contributed briefly to the SciRuby project a few years ago. One of their main subprojects is a Ruby wrapper around BLAS, ATLAS and LAPACK. I think some of the instructions for compiling BLAS, ATLAS and LAPACK might still have a few sentences written by yours truly. (It’s a very involved process.) Another one of their projects is a Ruby wrapper/interface around the GNU Scientific Library, or GSL.

One huge language in AI and data science these days is Python, with the NumPy and SciPy libraries. Again, some of the functionality is provided by linking to ATLAS and LAPACK. If it worked for them, maybe it will work for the JVM family of languages.

There is at least one project in Clojure that does this: Uncomplicate. It has sub-projects for OpenCL and BLAS and ATLAS. Along with Java and Groovy, Clojure is a language I am interested in. But for the time being I cannot use these projects. I think my graphics card is a bit too old and the drivers are too old for many of these libraries.

Uncomplicate is part of the reason I am looking at this area again. Somewhere on the site, the author points at the for development, it is okay to use pre-baked packages for ATLAS and LAPACK. One of these projects that I mention says for the best performance, you may want to hire someone to install ATLAS and LAPACK and other native libraries optimized for your production hardware. I did not know people made a living doing just that.

Another reason I am looking at this is GPU computing was mentioned in one of the side talks at the most recent ClojureConj. This is second- or third-hand, but apparently Goldman Sachs runs a nightly report analyzing the market (or some part of it, I am not too clear). It used to take nine hours to run. It was refactored to run on the GPU, and now it takes 20 minutes. I think all developers should become familiar with some of these technologies.

Will any of this gain traction in the Groovy space? I know things have been rough for Groovy for the past couple of years, especially after getting dropped by Pivotal. It seems like a lot of Groovy developers are happy making web apps with Grails. I like Grails, and it has worked out well for a lot of people. I will go through the Groovy Podcast backlog; perhaps Groovy developers are looking into this and I am just not aware of it.

Groovy developers can use the Java wrappers for native and GPU computing. No need to re-invent the wheel. Maybe going from Groovy to Java to native is not quite as fast as going from Python or Ruby to native, but I don’t think so. Even Ruby developers joke that Ruby is pretty slow. They say that sometimes the speed of the programmer is more important than the speed of the program. (But for some reason the Ruby devs who are moving to Elixir are now saying performance and concurrency are important after years of saying they were not. Whatevs.)

I do not have much experience in native or GPU programming. I am not too clear how NumPy and SciPy are used. I don’t want to swallow the Python ecosystem to get the answers to just a few questions, but: Are NumPy and SciPy programs (or bare-metal math/science apps in general) rewritten frequently? Or do they stay the same for years at a time? It seems to me if a GPU program does not change, you might be better off biting the bullet and doing things in C or C++.

But given the success of NumPy and SciPy, perhaps Groovy and Java can be used in this space. Maybe it’s not the end of the world if a bank’s nightly report takes 22 minutes. (How successful is SciPy? Not only are there conventions for general Python, there are also conferences for just SciPy). I doubt any technology will dethrone Python anytime soon, but it is an area to look at.

Having looked a small bit at some of these libraries, I do wonder how much can the unique features of Groovy be used. A lot of the tests deal with arrays of primitive types.  Someone on the Groovy mailing list wrote: I’ve found that you can really shrink down the lines of code just by using the basics that Groovy provides around collections, operator overloading, Groovy truthiness, etc.. Maybe that is enough reason to use Groovy for high performance computing (or at least higher performance). Can we make code calling native and GPU libraries better with features from Groovy, or will we just get Java without semicolons? Some of these functions do not want an ArrayList of objects, they want a fixed-length array of integers. Granted, you can make primitive arrays in Groovy:

but I think that is more keystrokes than in Java.

Just looking at a small part of TensorFlow, here is a line from the original Java test:

Here are two Groovy versions that both work (meaning the tests still pass):

Granted, it’s one line, but I’d say that’s idiomatic Groovy. Perhaps this could be a (small) market opportunity for the Groovy community. Or maybe I’m just crazy.

If you google “groovy jni” the first hit is this page from Object Partners. It mentions JNA and BridJ which I think are used by a lot of the libraries mentioned below. Frankly, it sounds like a lot of work making this sort of thing happen.

Regardless, even if I have to stick with Java and Clojure, I will still keep an eye on these libraries and this space. I don’t think I can become an expert in all of these, but as I said, I think it is important for developers to keep an eye on this spae. I might start some pages about the different libraries, and perhaps share my thoughts on them. Below are a few notes on what I have found so far.

I started by googling “Java BLAS” or “Java GPU”, and at first I only got a few libraries. But as I kept looking, I kept finding more.

The first two I found where jblas and Aparapi. From the jblas website, jblas “is essentially a light-wight wrapper around the BLAS and LAPACK routines.” (And from what I can gather, the correct capitalization is “jblas”.)

Aparapi is a wrapper around GPU and OpenCL code. If you cannot get your GPU drivers installed properly, it will do everything on the CPU. Sometimes I get core dumps running the tests on my Ubuntu laptop. It is about six years old, and has 6 GB of memory, so perhaps that is the issue. But on my 8 GB Windows laptop two of the tests fail. I plan on getting a new laptop soon, so I will try this again when I do. I bet getting all this set up on an Apple laptop is really easy; for the first time in my life I am thinking about buying an Apple product.

TensorFlow has some Java APIs, although they warn that there may be API changes and breakages for languages other than Python and C. This can use either the CPU or the GPU, but for GPUs it can only work with NVidia cards.

The Lightweight Java Game Library works with OpenCL, OpenGL, Vulkan, EGL and a lot of other stuff I have never heard of. As the name suggests, it is primarily for game development, and the wrappers around the underlying native libraries and standards require more knowledge of those underlying native libraries and standards than most libraries.

There is ND4J (N-Dimensional Arrays for Java) and Deeplearning4j. I am not sure if they are related somehow, but they were both started by developers at a company called Skymind.

And then there is the granddaddy: ByteDeco. There is a lot of stuff here. Some of the contributors work on ND4J and Deeplearning4j, and also work at Skymind. This project provides Java interfaces to 21 different C/C++ libraries: video, math, science, AI, robotics, facial recognition, lots of good stuff here. And after looking at their list, yes, I think “Skymind” is a bit too close to “Skynet”. But at least they’re giving you a hint.

You’re welcome.

No Longer Using aTunes

First off, that is not a typo: I really meant aTunes and not iTunes.

For the past few years, I have used an application called aTunes to fetch and listen to podcasts. One reason is I am pro-Java, and I generally prefer to use Java technologies where and when I can. Another is that it is NOT something from Apple. I have always found it strange that lot of people in technology will say they are pro-open source, and they don’t trust Microsoft or other big corporations and think are they too smart to let The Man tell them what to do, yet they will buy anything from Apple without question. I like to strive for consistency.

aTunes has not been updated for a couple of years. I know that the makers of the JDK strive for backwards compatibility, but I think at some point it might be best to use something that is maintained. Plus I had a hard time getting aTunes to work the last few times I tried it. I think I had to go a couple of versions back on one of my laptops.

There is another music player written in Java called Jajuk, but that has not been updated for a while either.

I did some digging, and I found a multi-platform music player called Clementine. So far I like it.

So far, I am still using jEdit, but I am getting more into emacs lately. There is an emacs Meetup here in Austin. For a long time it met around noon, but now it meets in the evening.

You’re welcome.

Image of aTunes from Wikipedia, image of Clementine from Wikipedia. Images assumed allowed under Fair Use.

Notes On Using Emacs Buffers

A few keyboard commands to use multiple buffers in Emacs:

You’re welcome.

Going Through “Functional Programming in Java”

I am starting to go through “Functional Programming in Java” by Venkat Subramaniam. I need to learn more about the stuff that is in JDK 8. We are still on JDK 6 where I work, but I think we will go to JDK 7 soon.

I think a lot of the new features in JDK 8 are similar to stuff that has been in Groovy for a long time.

There is an interesting paragraph in the Foreword by Brian Goetz:

The Java 8 design was heavily influenced by the core principles of functional programming: functions as values, immutability, and statelessness. We didn’t do this because functional programming is trendy or cool; we did this because programs that are expressed as stateless transformations on immutable data, rather than as modification of mutable data structures, tend to be easier to read and maintain and less error-prone, and to parallelize more gracefully.

You’re welcome.

Is the Lisp Curse Lifting?

One question about Lisp is if it is as wonderful as its advocates say, why isn’t is used more?

One hypothesis is the Lisp Curse (see here, here, here, here, here). In essence, it comes down to “Not Invented Here” on steroids. Once you get proficient with Lisp, one person can do things more quickly with Lisp than entire teams using other languages. So one result is that there are very few commonly used libraries in Lisp (Common Lisp, Scheme, etc).

You might find a library that does 80% of what you want, but its creator won’t help you, so you just make your own custom library. And now there are two libraries that do almost the same thing. Repeat this for a few decades.

I first heard about this from Neal Ford at a talk on Clojure or general functional programming either at a No Fluff Just Stuff conference or a Ruby/Rails conference. He said that Clojure might be the Lisp that gains traction because it is on the JVM, and can thus use Java libraries.

Then I found out about Quicklisp, a “library manager for Common Lisp” that lists 1,200 libraries for every major implementation of Common Lisp. (Also, see this page on the Common Lisp wiki.)

Recently, I read about Roswell, a launcher for Common Lisp. What sdkman does for the Groovy ecosystem and rvm does for Ruby, it looks like Roswell does the same for Common Lisp. It can manage multiple installations of two implementations of Common Lisp, Steel Bank Common Lisp and Clozure Common Lisp. (Now that sound has three meanings in programming. Are you saying “closure“, “Clojure”, or “Clozure”? We need some closure on this issue.)

One things that Roswell does that sdkman and rvm do not is that it runs on Windows as well as Linux and other Unix-based systems.

I am still just getting started with Clojure and Lisp, but it looks like I showed up at an interesting time.

You’re welcome.

Thinking About Starting A Grails 3 Tutorial

I was at the Austin Groovy and Grails User Group this past week. The speaker was Ken Kousen. I am seeing a pattern with the No Fluff Just Stuff Speakers: they are all really smart guys who tell the same jokes year after year.

Ken Kousen said that he has run into some gotchas with Grails 3, and that the docs are not up to date. He said Grails 3 needs a book, and he doesn’t see one coming out any time soon. A lot of the Groovy and Grails authors either have just finished books and don’t want to go through the grinder again right away.

I have been thinking for a couple of years of making a Grails tutorial, sort of like a Hartl tutorial for Grails. Perhaps it is time to get it started. If I did this, it would only exist online.

I have also thought about making a quick Ratpack tutorial. It would be nice if there was a linear set of steps to make a CRUD app. It’s nice to know some theory, but sometimes I wish projects would make it easier to help newbies get through a “Hello, World”.

I have to think about it. I might be biting off more than I can chew.

You’re welcome.

When I Say “Groovy” or “Clojure”, I Mean “Groovy” or “Clojure”

Two languages that I am looking at these days (and would eventually like to work in) are Groovy and Clojure.

There is a site called Find Grails Jobs. Most of them are for jobs for which you would code in Groovy and/or in Grails.

But there are a few that are really Java jobs. They will mention a lot of Java technologies, and Groovy is mentioned as a “nice to have” way at the bottom. Are some firms hoping you will sneak Groovy into a Java shop? Or are they trying to pull a bit of a bait and switch? I think it might be the latter. I like Java, but I would prefer doing Groovy. If I want a Java job, I will look on a site about Java jobs.

There is no site devoted to Clojure jobs, but there is one called Functional Jobs. They have jobs for a lot of functional languages. So much for purity.

One ad said (and this is a direct quote): If you’re interested in functional languages like Scala, Swift, Erlang, Clojure, F#, Lisp, Haskell, and OCaml, then you’ll enjoy learning Flow.

First off: I tried googling a few combinations of “Flow”, “programming”, “language” and “software”. I could not find a page that appeared to be a home page for a language called “Flow”. I did find a few pages about “flow-based programming“, which I guess is like Dataflows in GPars.

But more to the point: If I am interested in Clojure, don’t tell me that I am interested in some other language that only exists at your company. If you feel that some obscure language is the best way to deliver your products, fine. But don’t tell me what I want. You decided to jump down the rabbit hole. You deal with it.

You’re welcome.

Grails and PostgreSQL

In addition to the Austin Groovy and Grails User Group, I also go to the  Austin Postgres Users Group.

At the last APUG meeting there was no set agenda or presentation. They just had a Q&A with whoever wanted to answer. The company that hosts the APUG is a heavy Postgres user.

I had a few questions. The basic thrust was: What are some things DBAs wish developers knew about databases or did differently?

One answer was that developers should learn to think in sets. If you need to change a bunch of records in the database, just run a query that will update them all at once. A lot of developers would get a query (with an ORM) that would put the objects into a list, then iterate through the list and update them one at a time. This would be a much heavier load on the database. Just run a query.

In GORM, you can call .withSession on your domain classes to get the Hibernate session, and use that to run queries.

Another suggestion was to deal with views instead of tables. I really do not have any knowledge about that WRT Grails.

The last thing that I remember is that you should not think that you are the only person who touches the database. Someone else will want to run queries, and perhaps add data to it. Many web frameworks (like Grails and Rails) have database constraints if you access the database through the web app. But outside the app, you are on your own.

One small project I will start is I will go through the Grails constraints (see here and here) and see how to create them in Postgres. I might do MySQL as well. There are a few that Grails will put in the database when it is generated by Grails, like uniqueness, foreign keys, and whether or not a field can be null. But for most of them, they only exist in Grails.



Error With ForkedTomcatServer in Grails

Recently I had an issue with just about all of the Grails apps on my machine.

When I tried to run them, I was getting this error:

The number for the jar file would be different for different versions of Grails.

What I finally did to get rid of the error was:

  1. Run “grails clean; grails compile; grails list-plugins”
  2. Comment out the line for the Tomcat plugin in BuildConfig.groovy
  3. Run “grails clean; grails compile; grails list-plugins” again
  4. Uncomment the line for the Tomcat plugin
  5. Run “grails clean; grails compile; grails list-plugins” again

Maybe the first “grails clean; grails compile” is unnecessary. But this seemed to work.

I have no idea what caused it. It happened with various versions of Grails. I had no problem with new Grails apps, or with apps downloaded from Github.

You’re welcome.