Venkat On Improving Code

The speaker at the Austin JUG last week was Venkat Subramaniam (https://twitter.com/venkat_s, http://www.agiledeveloper.com/). He is to Java what Sandi Metz is to Ruby: If he is speaking in your town, you don’t care what the specific topic is. You know you need to go because you will learn something.

The topic was Twelve Ways to Make Code Suck Less. I will post some notes I took at the presentation. He gave the list counting down from twelve to one.

Why should we care about code quality? You cannot be agile if your code sucks. Code we write is how we tell our colleagues how we feel about them. Quality code is code that can be understood quickly.

  • 12. Schedule time to lower technical debt. Technical debt can take the form of not upgrading a framework (or JDK) to a more current version. Bad code is not technical debt; it is sabotage. You should make a list of your technical debt and ask yourself if you are adding to it, or reducing it. Do not schedule 100% of your developers’ time for adding new features. You need some slack time; you need to spend some time learning.
  • 11. Favor high cohesion. Highly cohesive code does one thing and does it well. It reduces the frequency/reasons for change. High cohesion leads to low complexity. He mentioned Cyclomatic complexity. TL/DR: Be a dessert topping OR a floor wax.
    (I once again point you to the github page for Boot, with this little gem: “The Modern CLJS tutorials are an excellent introduction to Boot and ClojureScript. Pretty much everything you need to know about Boot to get a project off the ground is covered there. Check it out!” What if I want to learn about Boot but do not want to spend any time on ClojureScript? Why not have a tutorial that is just about Boot?)
  • 10. Favor loose coupling. Remove coupling if possible. Code with high coupling is hard to test, hard to extend. Having a lot of mocks in a test is a bad sign.
  • 9. Program with intention. He mentioned Kent Beck’s Design Rules:
    – Passes the tests
    – Reveals intention
    – No duplication
    – Fewest elements
    See also http://wiki.c2.com/?XpSimplicityRules They are in order, so passing the tests takes priority over intention. (So why does Dr Subramaniam go in reverse order?) We should program deliberately.
  • 8. Avoid primitive obsession. Do not write general code. Imperative code has accidental complexity.
    If you write a for-loop, you might wonder if you need less-than, or less-than-or-equal-to. That is a sign. We do imperative because it is familiar. He gave an example of a method with a for-loop. He noted you would need extra variables to hold the result and the count. He contrasted it with an example of functional code using Java lambdas. Functional code is about a series of transformations, not mutations. You can read the functional example in one pass, while with the imperative you need to go up and down while reading it.
    Imperative code is easy to write, but harder to read, while functional is easier to read. Good code is a story, not a puzzle. Lambdas are glue code, and sometimes two lines might be too many.
    Lambdas still take some work to understand. You need to learn the semantics. The functional style is a combination of declarative code and higher-order functions.
  • 7. Prefer clear code to clever code. Clever code makes you feel smart. Do not focus on performance too early.
  • 6. Apply William Zinsser‘s principles on writing: Simplicity, clarity, brevity, humanity.
    He recommended “On Writing Well” by William Zinsser. He said when his son asked to borrow his copy, he refused. Instead he bought his some another copy, and his son has re-read it at least three times.
  • 5. Comment the why, not the what. A lot of developers make a lot of comments because a lot of companies and professors mandate comments. Do not comment to cover up bad code. Write expressive code that is self-documenting. Many times a comment is like explaining a joke. He recommended the book The Design of Everyday Things.
  • 4. Avoid long methods – Apply SLAP (Single Level of Abstraction Principle) Long methods are hard to understand, lack cohesion, tend to become longer, and are harder to test. How to break up methods? Think about what layer of abstraction you are at.
    He said it could be a bad idea to have a hard and fast rule about how many lines a method should be. I know Sandi Metz has a rule that methods should not be any longer than five lines. But is putting a rigid number such a good idea? I am probably not as smart as Sandi Metz, and frankly, you probably are not either. If you dogmatically insist on a hard number, you will probably just get a lot of methods that call methods that call methods, etc. Maybe we should stop using the word “Rule” when “Guideline” would be more appropriate.
    One example he gave is that if someone asks you what you did over the weekend, you might say, “I went to such-and-such city on Saturday, and stayed home and did laundry on Sunday”. If they ask for more details, you could give them. Or you would talk about how you woke up, had breakfast, got in the car, got gas, with route you took, etc, etc, etc. They might not want that level of detail off the bat.
    You might have a method that gets info from a database table, gets info from a web service, combines/transforms them somehow, and writes the result to another table. And has all the try/catch blocks for all the exceptions. That would be a good candidate for breaking it up.
    He related a story about someone who attended one of his lectures who was hired by a company to refactor a method that was 40,000 lines long. After a year, this person got it down to 30,000 lines.
    I stopped to talk to someone after the presentation, and then walked to my car. I think I overheard at least three conversations about this particular point.
  • 3. Give good, meaningful names. Your variable names represent your level of abstraction.
    He gave an example of bad variable names from a company that he could not name. It was software used to control oil refineries. One license is $1 million. He would not name the company, but if one license is $1 million, that narrows it down.
  • 2. Do tactical code reviews. Do code reviews often, and make them incremental. If you put them off, people will just allow code to pass without understanding it or really looking at it.
  • 1. Reduce state and state mutation. Do not make a field in a class (or method) unless you really need it. He related a story about an exhibit of Picasso with twenty versions of a painting, instead of just showing the final result. If Picasso could iterate, so can we. (I have a note asking how do functional languages deal with databases.)

Here are some pages on his site that I found to be informative:

You’re welcome.

2018-06-24 Update

There was not a lot of Clojure this week. There will be more in next week’s update.

I am taking a couple of weeks off from work. I have to take a lot of time off. I have accrued so much that if I do not take a lot in the next few months I will lose some.

I will study some Clojure, perhaps make progress on Simply Scheme. I also plan on writing something on why Lisp interests me, and perhaps why I hate Sharepoint. I will also visit some family in another state that we are not from.

You’re welcome.

2018-06-17 Update

I am still going through the tutorials on Purely Functional. There is nothing new to me in the intro videos, but I am getting practice with the REPL and emacs.

The project that I am on is downsizing. I might leave my employer and try my luck in the big, bad world. At one point I told my manager that I was interested in Lisp and Clojure. I probably mentioned that Lisp is one of the oldest languages and also not widely used. At a recent meeting, he said that I was interested in “legacy” technology. I plan on writing a post about why I am interested in Lisp and Clojure. While Lisp is one of the first programming languages invented, I don’t think it is appropriate to call it “legacy”. Lisp can adapt to any paradigm. “Legacy” has a negative connotation.

Part of the appeal of Lisp is that it seems like languages in the C family are becoming more like Lisp and Smalltalk over time. I have written this on this blog a few times. I mentioned it to some people once in Chicago, and they thought it was an interesting insight.

You’re welcome.

2018-06-03 Update

I am working through the tutorials on Purely Functional.

I got a bit hung up right away. I was trying to figure out how to reload files while working in the REPL as I updated the files. I followed this suggestion on StackOverflow by some guy named Dirk. At first, Dirk’s quirk did not work. Was Dirk a jerk? His burst was not voted first. SO at its worst? It made me so weak I could not speak. My REPL I could not tweak, putting an end to my learning streak. I do not want to sound pissed, but I do not wish to be dissed by Lisp. I am too tired for this.

Then I realized that while the files were reloaded, the new functions were not included in any aliases I used when I brought in the namespace with “require”. I could use the new functions, but I had to type out the ENTIRE namespace. O, the huge manatee! I did some googling, and I don’t think anyone mentions that.

You’re welcome.


2018-06-12_23.46.42 Update:

Just so it is easy to copy and paste, here are the commands:

Or you could do this:

You might have a function called “refresh”, but you will probably not have a namespace “repl”.

Do this after you follow the directions in the StackOverflow answer. It is better to bring in the clojure.tools namespace in your profile than in every stinkin’ project.

 

2017 Tweets

I have not gotten around to setting my twitter-retriever with a cron job yet. In the meantime, I will just copy and paste tweets. The plug-in that I was using stopped working in 2016. Here are all of my tweets from 2017:

You’re welcome.

Remaining 2016 Tweets

I have not gotten around to setting my twitter-retriever with a cron job yet. In the meantime, I will just copy and paste tweets. The plug-in that I was using stopped working in 2016. Here are the remaining 2016 tweets:

You’re welcome.

2018-05-28 Update

I am still working through Clojure For the Brave and True.

I am on the exercises at the end of chapter 5, but I might skip a couple of them. I don’t do too well with these “re-implement function x” exercises. I will also start going through the videos on Purely Functional. I think he is raising the price, and my membership will renew at the end of the year.

I am looking into some Big Data/Deep Learning libraries for Clojure. This was inspired by the last Austin Clojure meetup: There were only four of us and we had a hard time thinking of topics. I tried searching for Clojure meetups around the country for topic ideas, and frankly the pickings were kind of slim.

Frankly, sometimes the technology industry in Austin and in general is kind of a drag. If you don’t want to do out-of-date Java or spend your life on the JavaScript-no-CoffeeScript-no-TypeScript-no-now-its-something-else-Script roller coaster, the pickings can be pretty slim. Sometimes I think about taking a year or so off and going though Structure And Interpretation of Computer Programs or How To Design Programs and become a smarter person (not just better with some particular language or API). The issue is that while a lot of people will say things like “we aren’t looking for people with experience in the technology we use; we just want to hire smart people”, they turn around and only hire people with experience in the technology they use.

Anyway, the consensus in the Clojure community is that Clojure needs to be a bigger player in these spaces. There is a LOT of Python in AI. Being a JVM language, Clojure will have wrappers around a lot of the Java libraries I wrote about in Thoughts On Native, GPU, Groovy, Java and Clojure (even though there was not a lot of Clojure in that post).

I know that Big Data and AI are different things. I was thinking about looking at Sparkling to work with Spark (which I hope I can do on my laptop; do you need big machines to work with Big Data libraries?). This weekend I started looking at some of the videos on the Clojure TV channel on YouTube from Clojure Conj 2017. I did not go, but there seemed to be a LOT of videos about AI/Deep Learning (yes, I am using those terms interchangeably even though a lot of people do not).

There was Deep Learning Needs Clojure by Carin Meier, author of Living Clojure. She wasted the first seven minutes on some stupid joke about Skynet, which is a lot for a thirty minute presentation. I am glad I did not pay to see that. After that it gets better. The talk was pretty general. She mentioned some Clojure libraries, Graal VM, and at about 25:00 talks about how to get into Deep Learning.

Declarative Deep Learning In Clojure by Will Hoyt talked about Deeplearning4j. He says that matrices and linear algebra pretty much IS deep learning. There is some neuroscience in this presentation. He also talks about how the Clojure code is easier to deal with than Java and builders. I do not think he ever posts a link to dl4clj, which according to the Deeplearning4j site is the official port.

The Tensors Must Flow by William Piel is about his library Guildsman, which is a new Clojure interface to Google’s TensorFlow. There are already two Clojure projects that allow access to TensorFlow (clojure-tensorflow and tensorflow-clj). They do some Java interop around the TensorFlow Java API. (You can see Google’s not-quite-Javadoc here.) He wanted something that was more idiomatic for Clojure programmers. TensorFlow is written in Python (what else?), which Google then ports to C++ and other languages. But like most Python AI libs, it seems like it is just a wrapper around CPU or GPU code.

I understood the talk when I watched it. Really, I did. From what I remember, TensorFlow uses something called gradients in its process. I think a gradient implementation is an operation in the process. bpiel says the best way to contribute to Guildsman is to actually contribute C++ gradients to TensorFlow itself. He said in the talk he wanted to be done with Guildsman before the Conj in October. It is almost June, and he is still working on it.

The last one about Deep Learning was Building Machine Learning Models with Clojure and Cortex by Joyce Xu. She talked about a Clojure library called Cortex. This library is closer to Uncomplicate, in that while it does interface with a GPU  library, it is not a wrapper around a Java library in the middle.

The traffic on the Cortex mailing list seems to have dropped off, it’s not at version 1 and there seems to be a drop-off in contributions since January.

I do wish the speakers spent a bit more time talking about the implementation details of these libraries. Hosted languages (like Java or Python) do not do a lot of their AI crunching directly. They usually call a native library to calculate on either the CPU or the GPU. And for the GPU, some can do either CUDA (in other words, NVidia) or OpenCL (every other video card). Some libraries have multiple options, like Uncomplicate or Deeplearning4j. TensorFlow can use a CPU (they have instructions on getting the JNI file here) or GPU withy NVidia only. I have not tried Guildsman, so I do not know how he handles things or if he requires an NVidia GPU. I also have no idea how Cortex handles it. Their instructions tell you to get some SDK from NVidia. Perhaps they default to a CPU if there is no NVidia GPU.

I bought my laptop used, and the one I used before this one is at least six years old. I think the older one had an Intel video card, but I could not find any SDK for that version of the video chip. I think my current laptop may also be too old. (Dealing with the Intel Math Kernel is a LOT easier than wading through their OpenCL pages.) The only reason I can think of to buy an Apple laptop is to not deal with this. It is a bit frustrating. The whole point of using a language like Java or Ruby or Python is that I do not want to deal with hardware details.

Anyway, besides all that, I still have a few ideas for a few web apps to do in Clojure.


2018-05-29_22.14.49 update:

I looked at the Cortex mailing list, and apparently you can run the tests just using the CPU:

It would be great if they put that in the README.

You’re welcome.

2018-05-20 Update

I am still going through Clojure For the Brave and True. He does some pretty mind-bending things in chapter 5 when he walks you through his pegthing program.

Check this out:

In the call to “reduce”, he is sending it a collection of functions called “connect-right”, “connect-down-left” and “connect-down-right”. Then in the function in the reduce, there is a call to “connection-creation-fn”, which is passed to the anonymous function. So he is not actually calling a function called “connection-creation-fn”; that is a placeholder for the functions in the array being passed to “reduce”.

Granted, I knew before this what “reduce” is. But in this chapter he has a lot of functions calling functions. Granted, he does say that reducing over a collection of functions is not something you will do or see very often. Even though I have been looking at Clojure for a while, and this chapter wasn’t really anything new, it was a reminder how different functional programming can be.

You’re welcome.

2018-05-14 Update

I am still going through the the code review of twitter-retriever by Eric Normand of Purely Functional. And going through Clojure For the Brave and True.  Sometimes I come home from work and I just don’t feel like doing anything.

I am seeing a lot of nice core functions in Clojure For the Brave and True. It is helping me in my quest to post examples of as much of the core functions as I can.

I am still watching the code review in bits and pieces. I guess I don’t like being criticized, even though he is very supportive (not only of members of his site, but the Clojure community in general) and not being too critical, and I have incorporated a few of the things he has said. I had to do some funky things with tweet IDs to work Twitter timelines that seemed a bit confusing and might not be idiomatic Clojure; Twitter has a guide for that here. That said, I was able to incorporate some suggestions and reduced the lines of code, and I was able to remove one of the SQL calls and use some functions to get the same result.

He did mention that a lot of the defs should be in a let block. I showed this to the Austin Clojure group, and they had the same suggestion.

I still have a lot of println statements in there. I can run this on the command line or call some of the functions in the REPL, but testing some of this stuff is a bit beyond me. I have no idea how to mock/stub database calls or external calls in Clojure.

You’re welcome.

Debugging A Java App

Here are the steps to debugging a Serious Java Application.

  1. The ToeBoneBean is connected to the
  2. FootBoneImpl.isConnectedTo()
  3. AbstractAnkleBone.checkConnection()
  4. ShinBoneFactory.makeConnection()
  5. CompositeKneeBone.connect()
  6. StaticThighBoneBuilder.buildConnection()
  7. HipBoneService.comeUpWithAnotherWayToSayConnect()
  8. BackBoneQueryManager.manageConnection()
  9. NeckBoneDao.executeConnection()
  10. Which leads you to the SQL you were looking for in the first place.

In all seriousness, I am not one of those people who hates Java. It’s just frustrating going to a meetup and hearing about all these great things in Java 8 like lambdas (and yes, I know Java 8 has been out a while) or seeing all the wonderful things that you can do with Spring Boot, and then I go to my job and it’s like the past ten years never happened.

And before any Scala people feel superior, something like _{}\+= is not much better. We did Perl way back when. Not too interested in going back.

You’re welcome.