Category Archives: EKM Updates

Clojure And Gradle Update

I am putting twitter-retriever-gradle on hold. I spent a couple of hours trying to send JVM system properties to the clojureRun task, and I couldn’t figure it out.

I raised an issue on the github repo for the Clojure Gradle plug-in. They made a suggestion, and it did not work out. I was using a few libraries that could use system properties for configuration. That is not the only way to do things, but with Gradle my other option was to have multiple copies of the same file all over the repo. I would rather not do that.

I might ask on the Gradle forum. But for now, I think I will move on to something else.

I think Gradle works well for Groovy and Java. It seems a bit harder to use for Clojure. Back to Purely Functional.

You’re welcome.

2018-08-29 Update: Anonymous Functions and Kafka

I am still going through the tutorials on Purely Functional. It is taking longer than I thought. I might start going through Luminus at the same time as well. I think that building something while going through the lessons are a better way to learn.

In one of the lessons on Purely Functional, there is an interesting use of anonymous functions.

In version 3.17 of core.clj, we saw this in the function add:

We see a call to “cond”. It uses the type of the ingredient to decide which function to call.

In version 3.18 of core.clj, he adds a map with anonymous functions:

In version 3.19 of core.clj, he gets the functions out of the map and sets them equal to a symbol, replacing the cond:

I will also look into Apache Kafka. I went to the Apache Kafka meetup recently. I think it will be important going forward.

You’re welcome.

2018-06-24 Update

There was not a lot of Clojure this week. There will be more in next week’s update.

I am taking a couple of weeks off from work. I have to take a lot of time off. I have accrued so much that if I do not take a lot in the next few months I will lose some.

I will study some Clojure, perhaps make progress on Simply Scheme. I also plan on writing something on why Lisp interests me, and perhaps why I hate Sharepoint. I will also visit some family in another state that we are not from.

You’re welcome.

2018-06-17 Update

I am still going through the tutorials on Purely Functional. There is nothing new to me in the intro videos, but I am getting practice with the REPL and emacs.

The project that I am on is downsizing. I might leave my employer and try my luck in the big, bad world. At one point I told my manager that I was interested in Lisp and Clojure. I probably mentioned that Lisp is one of the oldest languages and also not widely used. At a recent meeting, he said that I was interested in “legacy” technology. I plan on writing a post about why I am interested in Lisp and Clojure. While Lisp is one of the first programming languages invented, I don’t think it is appropriate to call it “legacy”. Lisp can adapt to any paradigm. “Legacy” has a negative connotation.

Part of the appeal of Lisp is that it seems like languages in the C family are becoming more like Lisp and Smalltalk over time. I have written this on this blog a few times. I mentioned it to some people once in Chicago, and they thought it was an interesting insight.

You’re welcome.

2018-06-03 Update

I am working through the tutorials on Purely Functional.

I got a bit hung up right away. I was trying to figure out how to reload files while working in the REPL as I updated the files. I followed this suggestion on StackOverflow by some guy named Dirk. At first, Dirk’s quirk did not work. Was Dirk a jerk? His burst was not voted first. SO at its worst? It made me so weak I could not speak. My REPL I could not tweak, putting an end to my learning streak. I do not want to sound pissed, but I do not wish to be dissed by Lisp. I am too tired for this.

Then I realized that while the files were reloaded, the new functions were not included in any aliases I used when I brought in the namespace with “require”. I could use the new functions, but I had to type out the ENTIRE namespace. O, the huge manatee! I did some googling, and I don’t think anyone mentions that.

You’re welcome.


2018-06-12_23.46.42 Update:

Just so it is easy to copy and paste, here are the commands:

Or you could do this:

You might have a function called “refresh”, but you will probably not have a namespace “repl”.

Do this after you follow the directions in the StackOverflow answer. It is better to bring in the clojure.tools namespace in your profile than in every stinkin’ project.

 

2018-05-28 Update

I am still working through Clojure For the Brave and True.

I am on the exercises at the end of chapter 5, but I might skip a couple of them. I don’t do too well with these “re-implement function x” exercises. I will also start going through the videos on Purely Functional. I think he is raising the price, and my membership will renew at the end of the year.

I am looking into some Big Data/Deep Learning libraries for Clojure. This was inspired by the last Austin Clojure meetup: There were only four of us and we had a hard time thinking of topics. I tried searching for Clojure meetups around the country for topic ideas, and frankly the pickings were kind of slim.

Frankly, sometimes the technology industry in Austin and in general is kind of a drag. If you don’t want to do out-of-date Java or spend your life on the JavaScript-no-CoffeeScript-no-TypeScript-no-now-its-something-else-Script roller coaster, the pickings can be pretty slim. Sometimes I think about taking a year or so off and going though Structure And Interpretation of Computer Programs or How To Design Programs and become a smarter person (not just better with some particular language or API). The issue is that while a lot of people will say things like “we aren’t looking for people with experience in the technology we use; we just want to hire smart people”, they turn around and only hire people with experience in the technology they use.

Anyway, the consensus in the Clojure community is that Clojure needs to be a bigger player in these spaces. There is a LOT of Python in AI. Being a JVM language, Clojure will have wrappers around a lot of the Java libraries I wrote about in Thoughts On Native, GPU, Groovy, Java and Clojure (even though there was not a lot of Clojure in that post).

I know that Big Data and AI are different things. I was thinking about looking at Sparkling to work with Spark (which I hope I can do on my laptop; do you need big machines to work with Big Data libraries?). This weekend I started looking at some of the videos on the Clojure TV channel on YouTube from Clojure Conj 2017. I did not go, but there seemed to be a LOT of videos about AI/Deep Learning (yes, I am using those terms interchangeably even though a lot of people do not).

There was Deep Learning Needs Clojure by Carin Meier, author of Living Clojure. She wasted the first seven minutes on some stupid joke about Skynet, which is a lot for a thirty minute presentation. I am glad I did not pay to see that. After that it gets better. The talk was pretty general. She mentioned some Clojure libraries, Graal VM, and at about 25:00 talks about how to get into Deep Learning.

Declarative Deep Learning In Clojure by Will Hoyt talked about Deeplearning4j. He says that matrices and linear algebra pretty much IS deep learning. There is some neuroscience in this presentation. He also talks about how the Clojure code is easier to deal with than Java and builders. I do not think he ever posts a link to dl4clj, which according to the Deeplearning4j site is the official port.

The Tensors Must Flow by William Piel is about his library Guildsman, which is a new Clojure interface to Google’s TensorFlow. There are already two Clojure projects that allow access to TensorFlow (clojure-tensorflow and tensorflow-clj). They do some Java interop around the TensorFlow Java API. (You can see Google’s not-quite-Javadoc here.) He wanted something that was more idiomatic for Clojure programmers. TensorFlow is written in Python (what else?), which Google then ports to C++ and other languages. But like most Python AI libs, it seems like it is just a wrapper around CPU or GPU code.

I understood the talk when I watched it. Really, I did. From what I remember, TensorFlow uses something called gradients in its process. I think a gradient implementation is an operation in the process. bpiel says the best way to contribute to Guildsman is to actually contribute C++ gradients to TensorFlow itself. He said in the talk he wanted to be done with Guildsman before the Conj in October. It is almost June, and he is still working on it.

The last one about Deep Learning was Building Machine Learning Models with Clojure and Cortex by Joyce Xu. She talked about a Clojure library called Cortex. This library is closer to Uncomplicate, in that while it does interface with a GPU  library, it is not a wrapper around a Java library in the middle.

The traffic on the Cortex mailing list seems to have dropped off, it’s not at version 1 and there seems to be a drop-off in contributions since January.

I do wish the speakers spent a bit more time talking about the implementation details of these libraries. Hosted languages (like Java or Python) do not do a lot of their AI crunching directly. They usually call a native library to calculate on either the CPU or the GPU. And for the GPU, some can do either CUDA (in other words, NVidia) or OpenCL (every other video card). Some libraries have multiple options, like Uncomplicate or Deeplearning4j. TensorFlow can use a CPU (they have instructions on getting the JNI file here) or GPU withy NVidia only. I have not tried Guildsman, so I do not know how he handles things or if he requires an NVidia GPU. I also have no idea how Cortex handles it. Their instructions tell you to get some SDK from NVidia. Perhaps they default to a CPU if there is no NVidia GPU.

I bought my laptop used, and the one I used before this one is at least six years old. I think the older one had an Intel video card, but I could not find any SDK for that version of the video chip. I think my current laptop may also be too old. (Dealing with the Intel Math Kernel is a LOT easier than wading through their OpenCL pages.) The only reason I can think of to buy an Apple laptop is to not deal with this. It is a bit frustrating. The whole point of using a language like Java or Ruby or Python is that I do not want to deal with hardware details.

Anyway, besides all that, I still have a few ideas for a few web apps to do in Clojure.


2018-05-29_22.14.49 update:

I looked at the Cortex mailing list, and apparently you can run the tests just using the CPU:

It would be great if they put that in the README.

You’re welcome.

2018-05-20 Update

I am still going through Clojure For the Brave and True. He does some pretty mind-bending things in chapter 5 when he walks you through his pegthing program.

Check this out:

In the call to “reduce”, he is sending it a collection of functions called “connect-right”, “connect-down-left” and “connect-down-right”. Then in the function in the reduce, there is a call to “connection-creation-fn”, which is passed to the anonymous function. So he is not actually calling a function called “connection-creation-fn”; that is a placeholder for the functions in the array being passed to “reduce”.

Granted, I knew before this what “reduce” is. But in this chapter he has a lot of functions calling functions. Granted, he does say that reducing over a collection of functions is not something you will do or see very often. Even though I have been looking at Clojure for a while, and this chapter wasn’t really anything new, it was a reminder how different functional programming can be.

You’re welcome.

2018-05-14 Update

I am still going through the the code review of twitter-retriever by Eric Normand of Purely Functional. And going through Clojure For the Brave and True.  Sometimes I come home from work and I just don’t feel like doing anything.

I am seeing a lot of nice core functions in Clojure For the Brave and True. It is helping me in my quest to post examples of as much of the core functions as I can.

I am still watching the code review in bits and pieces. I guess I don’t like being criticized, even though he is very supportive (not only of members of his site, but the Clojure community in general) and not being too critical, and I have incorporated a few of the things he has said. I had to do some funky things with tweet IDs to work Twitter timelines that seemed a bit confusing and might not be idiomatic Clojure; Twitter has a guide for that here. That said, I was able to incorporate some suggestions and reduced the lines of code, and I was able to remove one of the SQL calls and use some functions to get the same result.

He did mention that a lot of the defs should be in a let block. I showed this to the Austin Clojure group, and they had the same suggestion.

I still have a lot of println statements in there. I can run this on the command line or call some of the functions in the REPL, but testing some of this stuff is a bit beyond me. I have no idea how to mock/stub database calls or external calls in Clojure.

You’re welcome.

2018-05-06 Update

I have watched part of the twitter retriever code review. I did make some changes based on what he said.

I am also going through the videos on Purely Functional. I am also working on Clojure For the Brave and True. I am on chapter 4.

I started changing my workouts. I wonder if that is taking energy away from Clojure. There is a lot to learn. I have been working on and off on a post about why I am interested in Clojure. I don’t think I have written a post yet about why Clojure and Lisp in general interests me.

You’re welcome.

2018-04-22 Update

I am still slowly going through Clojure For the Brave and True. I will keep going for the time being until inspiration hits.

I did try another app that uses the WordPress API. It is called WP-API v2 Java Client. I was not able to get this to work either. I thought (briefly) about making a direct SQL call to insert the post. I turned on SQL logging and decided it might be a bad idea. I have heard that many SQL experts are not impressed with WordPress’s schema. Plus I would either have to refactor (slightly) to use MySQL, or figure out how to use two connections.

Eric Normand of Purely Functional put up his code review of twitter-retriever. Take a look and let me know if it was good. He also has a channel where he talks about functional programming in general. He does these in his backyard.

You’re welcome.