2018-05-28 Update: Clojure AI

I am still working through Clojure For the Brave and True.

I am on the exercises at the end of chapter 5, but I might skip a couple of them. I don’t do too well with these “re-implement function x” exercises. I will also start going through the videos on Purely Functional. I think he is raising the price, and my membership will renew at the end of the year.

I am looking into some Big Data/Deep Learning libraries for Clojure. This was inspired by the last Austin Clojure meetup: There were only four of us and we had a hard time thinking of topics. I tried searching for Clojure meetups around the country for topic ideas, and frankly the pickings were kind of slim.

Frankly, sometimes the technology industry in Austin and in general is kind of a drag. If you don’t want to do out-of-date Java or spend your life on the JavaScript-no-CoffeeScript-no-TypeScript-no-now-its-something-else-Script roller coaster, the pickings can be pretty slim. Sometimes I think about taking a year or so off and going though Structure And Interpretation of Computer Programs or How To Design Programs and become a smarter person (not just better with some particular language or API). The issue is that while a lot of people will say things like “we aren’t looking for people with experience in the technology we use; we just want to hire smart people”, they turn around and only hire people with experience in the technology they use.

Anyway, the consensus in the Clojure community is that Clojure needs to be a bigger player in these spaces. There is a LOT of Python in AI. Being a JVM language, Clojure will have wrappers around a lot of the Java libraries I wrote about in Thoughts On Native, GPU, Groovy, Java and Clojure (even though there was not a lot of Clojure in that post).

I know that Big Data and AI are different things. I was thinking about looking at Sparkling to work with Spark (which I hope I can do on my laptop; do you need big machines to work with Big Data libraries?). This weekend I started looking at some of the videos on the Clojure TV channel on YouTube from Clojure Conj 2017. I did not go, but there seemed to be a LOT of videos about AI/Deep Learning (yes, I am using those terms interchangeably even though a lot of people do not).

There was Deep Learning Needs Clojure by Carin Meier, author of Living Clojure. She wasted the first seven minutes on some stupid joke about Skynet, which is a lot for a thirty minute presentation. I am glad I did not pay to see that. After that it gets better. The talk was pretty general. She mentioned some Clojure libraries, Graal VM, and at about 25:00 talks about how to get into Deep Learning.

Declarative Deep Learning In Clojure by Will Hoyt talked about Deeplearning4j. He says that matrices and linear algebra pretty much IS deep learning. There is some neuroscience in this presentation. He also talks about how the Clojure code is easier to deal with than Java and builders. I do not think he ever posts a link to dl4clj, which according to the Deeplearning4j site is the official port.

The Tensors Must Flow by William Piel is about his library Guildsman, which is a new Clojure interface to Google’s TensorFlow. There are already two Clojure projects that allow access to TensorFlow (clojure-tensorflow and tensorflow-clj). They do some Java interop around the TensorFlow Java API. (You can see Google’s not-quite-Javadoc here.) He wanted something that was more idiomatic for Clojure programmers. TensorFlow is written in Python (what else?), which Google then ports to C++ and other languages. But like most Python AI libs, it seems like it is just a wrapper around CPU or GPU code.

I understood the talk when I watched it. Really, I did. From what I remember, TensorFlow uses something called gradients in its process. I think a gradient implementation is an operation in the process. bpiel says the best way to contribute to Guildsman is to actually contribute C++ gradients to TensorFlow itself. He said in the talk he wanted to be done with Guildsman before the Conj in October. It is almost June, and he is still working on it.

The last one about Deep Learning was Building Machine Learning Models with Clojure and Cortex by Joyce Xu. She talked about a Clojure library called Cortex. This library is closer to Uncomplicate, in that while it does interface with a GPU  library, it is not a wrapper around a Java library in the middle.

The traffic on the Cortex mailing list seems to have dropped off, it’s not at version 1 and there seems to be a drop-off in contributions since January.

I do wish the speakers spent a bit more time talking about the implementation details of these libraries. Hosted languages (like Java or Python) do not do a lot of their AI crunching directly. They usually call a native library to calculate on either the CPU or the GPU. And for the GPU, some can do either CUDA (in other words, NVidia) or OpenCL (every other video card). Some libraries have multiple options, like Uncomplicate or Deeplearning4j. TensorFlow can use a CPU (they have instructions on getting the JNI file here) or GPU withy NVidia only. I have not tried Guildsman, so I do not know how he handles things or if he requires an NVidia GPU. I also have no idea how Cortex handles it. Their instructions tell you to get some SDK from NVidia. Perhaps they default to a CPU if there is no NVidia GPU.

I bought my laptop used, and the one I used before this one is at least six years old. I think the older one had an Intel video card, but I could not find any SDK for that version of the video chip. I think my current laptop may also be too old. (Dealing with the Intel Math Kernel is a LOT easier than wading through their OpenCL pages.) The only reason I can think of to buy an Apple laptop is to not deal with this. It is a bit frustrating. The whole point of using a language like Java or Ruby or Python is that I do not want to deal with hardware details.

Anyway, besides all that, I still have a few ideas for a few web apps to do in Clojure.


2018-05-29_22.14.49 update:

I looked at the Cortex mailing list, and apparently you can run the tests just using the CPU:

It would be great if they put that in the README.

You’re welcome.

2018-05-20 Update

I am still going through Clojure For the Brave and True. He does some pretty mind-bending things in chapter 5 when he walks you through his pegthing program.

Check this out:

In the call to “reduce”, he is sending it a collection of functions called “connect-right”, “connect-down-left” and “connect-down-right”. Then in the function in the reduce, there is a call to “connection-creation-fn”, which is passed to the anonymous function. So he is not actually calling a function called “connection-creation-fn”; that is a placeholder for the functions in the array being passed to “reduce”.

Granted, I knew before this what “reduce” is. But in this chapter he has a lot of functions calling functions. Granted, he does say that reducing over a collection of functions is not something you will do or see very often. Even though I have been looking at Clojure for a while, and this chapter wasn’t really anything new, it was a reminder how different functional programming can be.

You’re welcome.

2018-05-14 Update

I am still going through the the code review of twitter-retriever by Eric Normand of Purely Functional. And going through Clojure For the Brave and True.  Sometimes I come home from work and I just don’t feel like doing anything.

I am seeing a lot of nice core functions in Clojure For the Brave and True. It is helping me in my quest to post examples of as much of the core functions as I can.

I am still watching the code review in bits and pieces. I guess I don’t like being criticized, even though he is very supportive (not only of members of his site, but the Clojure community in general) and not being too critical, and I have incorporated a few of the things he has said. I had to do some funky things with tweet IDs to work Twitter timelines that seemed a bit confusing and might not be idiomatic Clojure; Twitter has a guide for that here. That said, I was able to incorporate some suggestions and reduced the lines of code, and I was able to remove one of the SQL calls and use some functions to get the same result.

He did mention that a lot of the defs should be in a let block. I showed this to the Austin Clojure group, and they had the same suggestion.

I still have a lot of println statements in there. I can run this on the command line or call some of the functions in the REPL, but testing some of this stuff is a bit beyond me. I have no idea how to mock/stub database calls or external calls in Clojure.

You’re welcome.

loop, recur and comp

Here is some code for a few functions that are mentioned in Chapter Five of Clojure For The Brave And True, along with a few modifications/experiments that I did.

 

You’re welcome.

Debugging A Java App

Here are the steps to debugging a Serious Java Application.

  1. The ToeBoneBean is connected to the
  2. FootBoneImpl.isConnectedTo()
  3. AbstractAnkleBone.checkConnection()
  4. ShinBoneFactory.makeConnection()
  5. CompositeKneeBone.connect()
  6. StaticThighBoneBuilder.buildConnection()
  7. HipBoneService.comeUpWithAnotherWayToSayConnect()
  8. BackBoneQueryManager.manageConnection()
  9. NeckBoneDao.executeConnection()
  10. Which leads you to the SQL you were looking for in the first place.

In all seriousness, I am not one of those people who hates Java. It’s just frustrating going to a meetup and hearing about all these great things in Java 8 like lambdas (and yes, I know Java 8 has been out a while) or seeing all the wonderful things that you can do with Spring Boot, and then I go to my job and it’s like the past ten years never happened.

And before any Scala people feel superior, something like _{}\+= is not much better. We did Perl way back when. Not too interested in going back.

You’re welcome.

2018-05-06 Update

I have watched part of the twitter retriever code review. I did make some changes based on what he said.

I am also going through the videos on Purely Functional. I am also working on Clojure For the Brave and True. I am on chapter 4.

I started changing my workouts. I wonder if that is taking energy away from Clojure. There is a lot to learn. I have been working on and off on a post about why I am interested in Clojure. I don’t think I have written a post yet about why Clojure and Lisp in general interests me.

You’re welcome.

(not) and its (complement)

Here is a quick look at the Clojure functions (not) and (complement):

 

 

You’re welcome.

2018-04-22 Update

I am still slowly going through Clojure For the Brave and True. I will keep going for the time being until inspiration hits.

I did try another app that uses the WordPress API. It is called WP-API v2 Java Client. I was not able to get this to work either. I thought (briefly) about making a direct SQL call to insert the post. I turned on SQL logging and decided it might be a bad idea. I have heard that many SQL experts are not impressed with WordPress’s schema. Plus I would either have to refactor (slightly) to use MySQL, or figure out how to use two connections.

Eric Normand of Purely Functional put up his code review of twitter-retriever. Take a look and let me know if it was good. He also has a channel where he talks about functional programming in general. He does these in his backyard.

You’re welcome.

More Collection Functions

I am going through Clojure For The Brave and True, and I have included some of the functions that I encounter while reading the book.

 

You’re welcome.

2018-04-08 Update

I think I am done for the time being with the Twitter-retriever.

As of now, it can get the last 3200 tweets of a user (excluding retweets) and capture any tweets going forward. It prints them out in HTML to a file with each tweet in enclosing “li” tags.

I tried to get the WordPress REST API working. I installed WordPress on my laptop, but I had a hard time getting anything to work with the POST command. GET was easy. It is the authentication that stumped me. I got a couple to work with curl, but no POST commands to work with clj-http. I might try again later, but for now I am done. It seemed to be a rabbit-hole of google searches, adding yet another plugin, editing the .htaccess file, etc, etc. For now, what I have is good enough.

I might look into an alternative to environ, but for the time being I think it will do.

I don’t know what I will work on next. I might get back to Simply Scheme. I might go back to coding examples for the Clojure API. I might go through some lessons at Purely Functional. Maybe TensorFlow (like with guildsman or clojure-tensorflow or tensorflow-clj). Maybe Luminus. As of right now, I have not decided.

You’re welcome.