Category Archives: Java

2017-11-06 Update

One thing I did this weekend was get permalinks working again on my blog. I got an SSL certificate from Let’s Encrypt several months ago, and I was not able to get permalinks to work. It bothered me for the longest time. Problem solved.

Not much on the Clojure API front.

I did look at Deeplearning4J, a deep learning framework written in Java. I cloned their repo of examples, and I got some of them working with gradle files, and learned a bit more about Gradle in the process. You can use a GPU, or a CPU. No messing with CUDA if you don’t want to.

There is even a Clojure wrapper for Deeplearning4J as well.

I had a post about using CPUs and GPUs from JVM languages for math/science/AI a while back. Since then, I have gotten both Neanderthal and Deeplearning4J to work on my system. I did mention Bytedeco, which is used by Deeplearning4J and a few other libraries. I looked at some of the Javadoc, and they use a LOT of static methods (see here for an example). I may leave Bytedeco for others and use some of the higher-level libraries, like Deeplearning4J. I think a lot of static methods is not idiomatic Java. I might consult “Effective Java” later for a second opinion.

I am going to include below one of the build.gradle files that I used to get some of the Deeplearning4J examples to work.

You’re welcome.

 

2017-06-25 Update

Not a whole lot to report this week. Still working on Simply Scheme. Perhaps I should call it Slowly Scheme.

I decided that this website could use a bit sprucing up, perhaps some pictures. So I am looking at a Java app called Processing that does visual stuff. I think some people use it as a CAD tool, while other people use it to make generative art (abstract art, patterns, fractals, etc).

You’re welcome.

Thoughts On Native, GPU, Groovy, Java and Clojure

One interest of mine is math and scientific computing, although I admit right now my interest is far greater than my knowledge.

It seems like for a lot of math and scientific computing, you either need to do stuff in C, C++ or even Fortran, or use a library in a higher-level language that lets you interface with libraries in C/C++/Fortran. Another new trend is to use code that runs on the GPU, like OpenCL.

Lately I have been looking at a few Java libraries that interface with lower level code. I think it is a good idea to keep learning new things. I think AI and machine learning will become more important over time.

I contributed briefly to the SciRuby project a few years ago. One of their main subprojects is a Ruby wrapper around BLAS, ATLAS and LAPACK. I think some of the instructions for compiling BLAS, ATLAS and LAPACK might still have a few sentences written by yours truly. (It’s a very involved process.) Another one of their projects is a Ruby wrapper/interface around the GNU Scientific Library, or GSL.

One huge language in AI and data science these days is Python, with the NumPy and SciPy libraries. Again, some of the functionality is provided by linking to ATLAS and LAPACK. If it worked for them, maybe it will work for the JVM family of languages.

There is at least one project in Clojure that does this: Uncomplicate. It has sub-projects for OpenCL and BLAS and ATLAS. Along with Java and Groovy, Clojure is a language I am interested in. But for the time being I cannot use these projects. I think my graphics card is a bit too old and the drivers are too old for many of these libraries.

Uncomplicate is part of the reason I am looking at this area again. Somewhere on the site, the author points at the for development, it is okay to use pre-baked packages for ATLAS and LAPACK. One of these projects that I mention says for the best performance, you may want to hire someone to install ATLAS and LAPACK and other native libraries optimized for your production hardware. I did not know people made a living doing just that.

Another reason I am looking at this is GPU computing was mentioned in one of the side talks at the most recent ClojureConj. This is second- or third-hand, but apparently Goldman Sachs runs a nightly report analyzing the market (or some part of it, I am not too clear). It used to take nine hours to run. It was refactored to run on the GPU, and now it takes 20 minutes. I think all developers should become familiar with some of these technologies.

Will any of this gain traction in the Groovy space? I know things have been rough for Groovy for the past couple of years, especially after getting dropped by Pivotal. It seems like a lot of Groovy developers are happy making web apps with Grails. I like Grails, and it has worked out well for a lot of people. I will go through the Groovy Podcast backlog; perhaps Groovy developers are looking into this and I am just not aware of it.

Groovy developers can use the Java wrappers for native and GPU computing. No need to re-invent the wheel. Maybe going from Groovy to Java to native is not quite as fast as going from Python or Ruby to native, but I don’t think so. Even Ruby developers joke that Ruby is pretty slow. They say that sometimes the speed of the programmer is more important than the speed of the program. (But for some reason the Ruby devs who are moving to Elixir are now saying performance and concurrency are important after years of saying they were not. Whatevs.)

I do not have much experience in native or GPU programming. I am not too clear how NumPy and SciPy are used. I don’t want to swallow the Python ecosystem to get the answers to just a few questions, but: Are NumPy and SciPy programs (or bare-metal math/science apps in general) rewritten frequently? Or do they stay the same for years at a time? It seems to me if a GPU program does not change, you might be better off biting the bullet and doing things in C or C++.

But given the success of NumPy and SciPy, perhaps Groovy and Java can be used in this space. Maybe it’s not the end of the world if a bank’s nightly report takes 22 minutes. (How successful is SciPy? Not only are there conventions for general Python, there are also conferences for just SciPy). I doubt any technology will dethrone Python anytime soon, but it is an area to look at.

Having looked a small bit at some of these libraries, I do wonder how much can the unique features of Groovy be used. A lot of the tests deal with arrays of primitive types.  Someone on the Groovy mailing list wrote: I’ve found that you can really shrink down the lines of code just by using the basics that Groovy provides around collections, operator overloading, Groovy truthiness, etc.. Maybe that is enough reason to use Groovy for high performance computing (or at least higher performance). Can we make code calling native and GPU libraries better with features from Groovy, or will we just get Java without semicolons? Some of these functions do not want an ArrayList of objects, they want a fixed-length array of integers. Granted, you can make primitive arrays in Groovy:

but I think that is more keystrokes than in Java.

Just looking at a small part of TensorFlow, here is a line from the original Java test:

Here are two Groovy versions that both work (meaning the tests still pass):

Granted, it’s one line, but I’d say that’s idiomatic Groovy. Perhaps this could be a (small) market opportunity for the Groovy community. Or maybe I’m just crazy.

If you google “groovy jni” the first hit is this page from Object Partners. It mentions JNA and BridJ which I think are used by a lot of the libraries mentioned below. Frankly, it sounds like a lot of work making this sort of thing happen.

Regardless, even if I have to stick with Java and Clojure, I will still keep an eye on these libraries and this space. I don’t think I can become an expert in all of these, but as I said, I think it is important for developers to keep an eye on this spae. I might start some pages about the different libraries, and perhaps share my thoughts on them. Below are a few notes on what I have found so far.

I started by googling “Java BLAS” or “Java GPU”, and at first I only got a few libraries. But as I kept looking, I kept finding more.

The first two I found where jblas and Aparapi. From the jblas website, jblas “is essentially a light-wight wrapper around the BLAS and LAPACK routines.” (And from what I can gather, the correct capitalization is “jblas”.)

Aparapi is a wrapper around GPU and OpenCL code. If you cannot get your GPU drivers installed properly, it will do everything on the CPU. Sometimes I get core dumps running the tests on my Ubuntu laptop. It is about six years old, and has 6 GB of memory, so perhaps that is the issue. But on my 8 GB Windows laptop two of the tests fail. I plan on getting a new laptop soon, so I will try this again when I do. I bet getting all this set up on an Apple laptop is really easy; for the first time in my life I am thinking about buying an Apple product.

TensorFlow has some Java APIs, although they warn that there may be API changes and breakages for languages other than Python and C. This can use either the CPU or the GPU, but for GPUs it can only work with NVidia cards.

The Lightweight Java Game Library works with OpenCL, OpenGL, Vulkan, EGL and a lot of other stuff I have never heard of. As the name suggests, it is primarily for game development, and the wrappers around the underlying native libraries and standards require more knowledge of those underlying native libraries and standards than most libraries.

There is ND4J (N-Dimensional Arrays for Java) and Deeplearning4j. I am not sure if they are related somehow, but they were both started by developers at a company called Skymind.

And then there is the granddaddy: ByteDeco. There is a lot of stuff here. Some of the contributors work on ND4J and Deeplearning4j, and also work at Skymind. This project provides Java interfaces to 21 different C/C++ libraries: video, math, science, AI, robotics, facial recognition, lots of good stuff here. And after looking at their list, yes, I think “Skymind” is a bit too close to “Skynet”. But at least they’re giving you a hint.

You’re welcome.

No Longer Using aTunes

First off, that is not a typo: I really meant aTunes and not iTunes.

For the past few years, I have used an application called aTunes to fetch and listen to podcasts. One reason is I am pro-Java, and I generally prefer to use Java technologies where and when I can. Another is that it is NOT something from Apple. I have always found it strange that lot of people in technology will say they are pro-open source, and they don’t trust Microsoft or other big corporations and think are they too smart to let The Man tell them what to do, yet they will buy anything from Apple without question. I like to strive for consistency.

aTunes has not been updated for a couple of years. I know that the makers of the JDK strive for backwards compatibility, but I think at some point it might be best to use something that is maintained. Plus I had a hard time getting aTunes to work the last few times I tried it. I think I had to go a couple of versions back on one of my laptops.

There is another music player written in Java called Jajuk, but that has not been updated for a while either.

I did some digging, and I found a multi-platform music player called Clementine. So far I like it.

So far, I am still using jEdit, but I am getting more into emacs lately. There is an emacs Meetup here in Austin. For a long time it met around noon, but now it meets in the evening.

You’re welcome.

Image of aTunes from Wikipedia, image of Clementine from Wikipedia. Images assumed allowed under Fair Use.

boot-new Makes Boot Usable

As I mentioned, I am going through Functional Programming in Java. I have a couple of github sub-repos for it. I am going through the Java code, and then re-implementing it in Groovy and Clojure. The Java and Groovy parts are here inside a repo called “my_groovy_apps_001”, and the Clojure part is here inside a repo called “clojure_apps”.

I started doing the Clojure inside the Gradle app inside “my_groovy_apps_001” using the Neptune Gradle plugin for Clojure. It was okay, but I could not get the REPL to work, and I could not figure out how to run only some of the tests.

So I decided this might be a good time to learn Clojure Boot, the new build tool that is an alternative to Leiningen and seems to be gaining some momentum.

Frankly, the docs are terrible. You kind of need the docs to do anything since it doesn’t do a lot out of the box, kind of like Eclipse when it was first released. Until plug-ins were developed, Eclipse was not very useful, and Boot seems the same to me at this point. Boot doesn’t even have a way to run tests out of the box.

One reason I think the docs are terrible is that in the “Getting Started” section on the Boot github page, they redirect you to a ClojureScript tutorial, telling you that it is “an excellent introduction to Boot and ClojureScript”.

I really have no interest in ClojureScript at this time. I do not like Javascript, and while I have heard great things about ClojureScript and its creators/maintainers, I want to avoid the JavaScript swamp as much as possible. As the guy who wrote the Pollen DSL for Racket put it, Lisp makes me smarter, and pretty much any language ending in *Script seems to kill brain cells on contact. The fact that there are so many languages that compile to JavaScript to me is not a good sign for JavaScript. It says to me that people deal with JavaScript because they have to, not because they want to. And that people will go to great lengths to avoid dealing with JavaScript directly. And since the JavaScript community keeps spawning new frameworks all the time, I think on some level they agree. I am sure at some point I will have to learn about ClojureScript, but I would like to put that off for now.

But it’s not just me being anti-anythingScript here. I like the Unix philosophy of a tool does one thing and does it well. I want to learn about Boot. Not ClojureScript. I don’t want to wade through a tutorial for a desert topping to find the bits about the floor wax.

So I did some searching for some blog posts about how to get started with Boot. I found some by a guy named Sean Cornfield (whom I actually met at ClojureConj). He developed a plug-in called boot-new that pretty much does everything you need to get started using Boot. I would argue that you don’t even need a tutorial for Boot. Just use boot-new.

All you have to do is install Boot, and then run this command:

It will fetch boot-new from Clojars (I assume) and make a project called myapp.

One of the things that boot-new allows you to do is to run tests for specific namespaces. You can run just one:

or multiple namespaces separated by commas:

With “–namespaces”, you specify the namespace of the tests. To specify the namespace of the function, use –include:

So if anyone in your local Clojure group wants a presentation on Boot, you can just tell them to look at boot-new. It would be the shortest presentation ever.

I don’t know how to do all the things that Leiningen does, but if you just want to start a project, compile it, run a REPL, run some tests and get a JAR, I think boot-new covers some of the basics. Correctly or incorrectly, it seems like a lot of people (including me) think of Boot as an alternative to Leiningen. So my approach is basically, “I know how to do X in Leiningen, how do I do that in Boot?”

For a lot of basic tasks, boot-new is your answer.

You’re welcome.

Going Through “Functional Programming in Java”

I am starting to go through “Functional Programming in Java” by Venkat Subramaniam. I need to learn more about the stuff that is in JDK 8. We are still on JDK 6 where I work, but I think we will go to JDK 7 soon.

I think a lot of the new features in JDK 8 are similar to stuff that has been in Groovy for a long time.

There is an interesting paragraph in the Foreword by Brian Goetz:

The Java 8 design was heavily influenced by the core principles of functional programming: functions as values, immutability, and statelessness. We didn’t do this because functional programming is trendy or cool; we did this because programs that are expressed as stateless transformations on immutable data, rather than as modification of mutable data structures, tend to be easier to read and maintain and less error-prone, and to parallelize more gracefully.

You’re welcome.

2016-02-22 Update

I bought Living Clojure by Carin Meier, and I am working through the second half. It seems like a nice language and a nice book. I hope to use it professionally in the future.

I also like Groovy. As Dick Wall would say, I am all about the Groovy, man. I also made some progress on the Groovy Mail Server. I made some progress on authentication using SASL. I using SASL classes in the JDK, but I had to implement some Callback handler code myself. I had some help from Stack Overflow. In general I like Java the language, but sometimes I feel like the API designers like to make things more complex than necessary.Interfaces, extensions, listeners, factories, handlers, the knee bone instantiated the thigh bone, etc, etc, etc.

One thing I found out today is that when calling static methods on a class in the Groovy shell, you need to include the full package name for the class. I prefer the shell over the console, but I may have to switch. I wish the shell could run any Groovy code as-is. After finding out about setting interpeterMode to true in the shell, I thought the shell would run everything fine. Oh well.

I also made some small changes. I was not catching a possible exception, and the code was not closing connections. When I logged into the server it had almost maxed out.

You’re welcome.

Changing Immutable Objects

Recently someone at work someone sent me a link to an article saying that getters and setters are evil by a guy named Yego256. He has a few other interesting posts about immutable objects here and here. They can be added to the list I made of pages about immutable objects here.

Hopefully Grails 3 will utilize immutability.

It takes a shift in mindset to start using immutable objects. Plus, in Java (and Groovy) a lot of web frameworks don’t utilize them, so people are not too familiar with them.

One issue some developers have is: Many things in the world are mutable. How can you make “changes” to immutable objects?

One answer is that you can use the values you don’t want to change to populate a new immutable object, and your variable/field name points to a new immutable object.

(If you can think of a better way to explain this, let me know. It’s kind of confusing to say that an immutable object is referenced by a “variable name”.)

I will demonstrate this with Groovy (using the annotations EqualsAndHashCode, Immutable and ToString). I made a class to model a state in the USA. States do not change their names, abbreviations or year they joined the union. But many of them have changed their capital cities. So we will make an object, and change its capital city while keeping everything else the same.

Here is our class:

Now let’s look at using it in a Groovy shell. The first capital of Illinois was a town called Kaskaskia, which no longer exists. Then it moved to Vandalia (which does still exist) and finally to Springfield.

I will try to change a few fields to prove the objects are immutable, and I will call hashCode() to show that we are dealing with different objects.

So we used the existing values to make new objects. The JVM used the same variable name (“il”) to refer to different immutable objects.

Here is the code without the output:

2015-06-06_14.23.56 update: I now realize that in order to prove the objects were truly different objects, I should have called System.identityHashCode(il) instead.

You’re welcome.

Another Post on Automatic Getters and Setters

Recently I was looking at the Groovy section on DZone, and there was an article called Evil: Getters and Setters Where They’re Not Required (original here). He reiterates a lot of what I have been saying in my posts about why I am making the Groovy Validators.

I doubt that he and I are the only two that have noticed this issue. I have heard/read a lot of people complain about the verbosity of getters and setters (like we can get in Eclipse), and how much cleaner it is to use a language that will generate them for you (like Groovy and Ruby). Yet it is odd that so many times people gloss over the fact that these blind, automatic getters and setters essentially leave your variables wide open for just about anything. It seems like people hate verbosity more than they like data integrity.

Granted, I am a bit out of practice with Ruby, so it is possible that there is some sort of validation in the non-Rails Ruby space. And this is purely anecdotal. I just thought it was interesting.

UPDATE: I am making some progress on using the validation annotations with immutable objects. Hopefully Mr LaForge hasn’t closed that bug yet, and announcing my validators will bring me fame and fortune.

Notes From Austin JUG 2014-05-27

Here are the notes that I took at the May 27, 2014 meeting of the Austin JUG. There was only one presentation: Craig Walls on Spring Boot. He used quite a bit of Groovy during this presentation. You can find Craig Walls on Twitter and Github.

As always, the notes are presented pretty much as I wrote them.

Criticism of Spring: Lots of config
He says that Boot is “how we do Spring in 2014”
He goes over the evolution of Spring configuration.
Now you can make an app with no configuration.
Spring 4 can use Ruby config. Can I use Groovy?

Spring 4:
Takes away  boilerplate configuration
Heavily opinionated
Can run with Groovy at the command line
Can build to JAR or WAR

Spring Boot CLI: run Groovy apps from command line
Starts embedded Tomcat or Jetty
Heavily employs Spring Boot autoconfig

Auto-config:
creates beans as needed,
resolves beans as needed (CLI)
adds imports (CLI)

Starters:
Pre-baked POM files
Add a dependency in Maven or Gradle
Makes dependencies available

The Actuator:
Reasonable error pages
Endpoints for runtime metrics and info:
/beans: all beans
/env: Configuration properties
/trace: recent requests
/dump: thread details
/metrics: application metrics, gauge tells you how long it took for each request
/autoconfig: auto-configuration explanation
/info: by default has little, look it up. You can put something here
/health: this gives a few indicators, that your DB and services you call are running

Demo:
He installed CLI
rest_hi returns “hi”

spring run app.groovy

That could be a tweet
Spring’s website is using Spring Boot
This could be good for Groovy

Now JSON:

Spring will add the imports to Groovy compiler
He will use Thymeleaf (sp?) for UI
He will use H2 database

There is a Gradle spring-boot plugin

It would be great if the guy behind me would stop yawning

Spring Tool Suite has Boot Support

To secure actuators: Add Spring Security starter to your classpath
default: user/password

You can put stuff in application.properties or application.yaml
Go to Spring.io
Look at the Getting Started Guides, most of them use Spring Boot
Spring Boot requires Spring 4.0