Category Archives: Technology

2018-08-29 Update: Anonymous Functions and Kafka

I am still going through the tutorials on Purely Functional. It is taking longer than I thought. I might start going through Luminus at the same time as well. I think that building something while going through the lessons are a better way to learn.

In one of the lessons on Purely Functional, there is an interesting use of anonymous functions.

In version 3.17 of core.clj, we saw this in the function add:

We see a call to “cond”. It uses the type of the ingredient to decide which function to call.

In version 3.18 of core.clj, he adds a map with anonymous functions:

In version 3.19 of core.clj, he gets the functions out of the map and sets them equal to a symbol, replacing the cond:

I will also look into Apache Kafka. I went to the Apache Kafka meetup recently. I think it will be important going forward.

You’re welcome.

Thoughts On WordPress

A week or so ago I went to the Austin WordPress Meetup which featured a presentation about Gutenberg.

Gutenberg is the upcoming editor for WordPress. I admit I did not quite understand everything that was said. But it means there are some big changes coming up. It will allow WordPress to interface with many different types of systems that it cannot talk to today. At one point, the presenter theorized that in a few years WordPress may not use themes as we understand them today. This could put some people out of work; I think there are several companies that sell WordPress themes.

Gutenberg will be written with React. I do not like JS and try to avoid it as much as I can (too much churn and reinvention of wheels), but this might help solidify React’s ranking. As WP people love to point out, WP powers 20-25% of all web sites. The presenter said that Drupal will be moving to Gutenberg as well (although I later found out this is not a sure thing).

I have toyed with the idea of porting WP to another language, even though it would be a massive undertaking. A lot of developers do not like PHP. When I was getting into Ruby and Rails, I joined the mailing lists for local Ruby/Rails groups in about a dozen cities. About once a month, someone in one of the cities would ask about any blog or CMS frameworks written in Ruby. They had made a Rails app for a client, and now the client wanted a blog or CMS. A few replies would mention Radiant or Refinery, but it seems like most of the time the original poster would just go with WordPress. Once someone just posted the Most Interesting Man In the World meme. I did have a go at porting WordPress to Rails.

Eric Normand posted a video titled What Clojure needs to grow – a boring web framework and boring data science.  He said Clojure needs a default web framework. At one point he talks about WordPress. It made me think again about porting WordPress to another language. I have not had problems with WordPress, but some sites have, including Naked Capitalism.

It would also be nice to port the WordPress database from MySQL to Postgres. There is pgloader (written in Common Lisp). There is also pg_chameleon, which is written in Python. But the issue is that some people might not like a new system if it does not look and work just like WordPress, kind of like people do not like other office suites. Plus things might change a lot with Gutenberg.

That said, I think if “ClojurePress” becomes a thing, there are a few things it should do out of the box. There are some things that are pretty common that are handled by plugins. Sometimes multiple plugins, sometimes plugins that have not been updated in a while. Maybe some of these are things that only I want, but here is my list:

  • HTML Tables
  • Static HTML Export
  • Contact Page
  • Twitter intregration (posting AND archiving)
  • Facebook (Granted, Facebook could block this; see here and here)
  • Deleting Revisions
  • Move Comments

 

Reasons I Do Not Like Sharepoint

Here are some reasons I do not like Sharepoint.

First off, I think the underlying concept is wrong. Putting office files in on the web is just stupid.
The web started out at HTML files; even dynamic sites make HTML files. Trying to make the web into a viewer for Office files adds a layer of friction. It is also vendor lock-in. The web was intended to be open. And again, MS is trying to close it.

Speaking of files adding friction to the web: I also do not like PDFs on the web, especially multi-column PDFs.

Why are people putting stuff in Word files anyway? (Spreadsheets are another story.) Why not just use some kind of wiki? Or Atlassian Confluence? Or Drupal? Or Intranet DASHBOARD? Or Jive? Or MindTouch? WYSIWYG editors are not that hard to learn. Why pay for MS Word when you are not using all of its features? And yes, I know SP has a wiki. I have never tried it, and I have no desire to. It is probably terrible. SP is about trying to make proprietary Office files relevant to an open web-centered world. Making a decent wiki is SP would expose the absurdity of SP’s existence.

If you really need to print, I am sure there are other systems that can convert content to PDFs. There are systems that have single sign-on capabilities.

If you use files, why are you using an online system? The impedance mismatch causes problems. People still download the files and email them to each other and get out sync. And even though there is versioning, people will manually add different versions of files. I have seen files in folders like this: file.doc, file_002.doc, file_DECEMBER.doc
I think this is because SP tries to do some many things.

Another reason is almost everyone hates it. At least, almost everyone I have asked about it at companies where I have worked that have used SP have hated it. Maybe everybody tells the big shots they love it and everybody is lying to me, but I really do not have any decision-making authority. Why would people lie to me?

There are many developers who take a course that uses Lisp or Scheme, or get a job using Lisp, and they will rave about it years or even decades later. They will say that Lisp/Scheme changed their perception unlike anything else. That they learned more by using Lisp than they learned using anything else. Nobody really holds MS technologies in the same esteem. A lot of non-technical people love MS products because they are generally unaware of any alternative. We should use technologies that we admire, not simply out of ignorance.

We are always told on the technology side that we have to “learn to understand the needs of the business.” Why don’t they have to learn anything about technology? Or what is good technology?

The only people who seem to like SP are managers and SP consultants. I asked someone at my current employer why we used SP, and he told me that we got a good deal from our cloud provider. Not that we need it. Or that it is good at its intended purpose (whatever that is).

I have found that many things that are justified based on cost and not quality are usually not very good. When I was at BofA, we had to use some computer-based training. Everybody hated it, except our finance guy. When he was asked to justify it at a meeting, he said it was “cost-effective.” If the users hate it and nobody learns anything from it, how is it effective?

Another reason is that it is from MS. It’s the same old bait and switch. They sell it as something easy, and you don’t need an admin. But when you hit a wall, or want to do something advanced, then MS says you need an admin.

Some people have this weird blindness about MS. If you present them with another spreadsheet or word processor, they will reject it because it is not exactly like MS Office. But if MS changes the interface (like from a toolbar to a ribbon), people just go along with it.

Another reason is that the search capabilities in SP are terrible. This is important for something used to store documents. I have always gotten better search results on blogs and wikis. Content on wikis can be versioned, so why put stuff into SP? I think most businesses could replace SP with a wiki. I think a lot of wikis can do single-sign on, permissions, whatever. Bookmarks would not change. A lot of people love to move SP folders.

SP is a complex product sold as a simple product. It is a desert topping and a floor wax. If SP does not fit your business, you have two choices: change your life to fit SP, or replace SP. Desktop PCs, laptops, cel phones induced people to change their behavior and workflow. SP is not amazing enough for people to want to change for it.

At my current employer, search is so bad that a committee was started to come up with solutions. I went to one meeting, and I realized it was a waste of time. Someone suggested a SP site with links to all the other SP sites. For some reason, dropping SP was not considered. I do not get the hold SP has on some people. Yes, migrating would be expensive. But using it is expensive. And painful.

complex product sold as a simple product
impedance mismatch
vendor lock-in
bad search

You’re welcome.

2018-06-17 Update

I am still going through the tutorials on Purely Functional. There is nothing new to me in the intro videos, but I am getting practice with the REPL and emacs.

The project that I am on is downsizing. I might leave my employer and try my luck in the big, bad world. At one point I told my manager that I was interested in Lisp and Clojure. I probably mentioned that Lisp is one of the oldest languages and also not widely used. At a recent meeting, he said that I was interested in “legacy” technology. I plan on writing a post about why I am interested in Lisp and Clojure. While Lisp is one of the first programming languages invented, I don’t think it is appropriate to call it “legacy”. Lisp can adapt to any paradigm. “Legacy” has a negative connotation.

Part of the appeal of Lisp is that it seems like languages in the C family are becoming more like Lisp and Smalltalk over time. I have written this on this blog a few times. I mentioned it to some people once in Chicago, and they thought it was an interesting insight.

You’re welcome.

Command Line Passphrase Generation

I figured out another way to calculate passphrases, this time using the command line.

To cut to the chase, here is the command:

I found out about a command called “shuf” which according to its man page can “generate random permutations”. You can run this against a word list. Many Unix-based systems have a word list.

On my system, the word list is at /usr/share/dict/words, which is a link to /etc/dictionaries-common/words, which itself is a link to /usr/share/dict/american-english. You can also find one online here.

After you run the list against “shuf”, you need to put them all on one line. You can do this with the “tr” command (see man page here). You can use it to “translate or delete characters”. I want to replace the carriage return-new line combination with a space. In the command, we can represent that combination with either ‘\r\n’ or [:cntrl:]. I prefer [:cntrl:] because the only way to represent the space is with a space between single quotes, and I think using one less pair cuts down on ambiguity (“tr [:cntrl:] ‘ ‘” as opposed to “tr ‘\r\n’ ‘ ‘”).

Even though the word list is “american-english”, it still has some characters that we do not use, like vowels with umlauts or the letter “A” with a circle over it (I guess that is a “super-umlaut”). There are also a lot of words in there with an apostrophe followed by an “s”, words with upper-case letters and words with less than 4 characters. Taking out all of that takes us from a file with 100K words to one with about 26K words.

Here is the full grep sequence I used:

You’re welcome.

2018-01-07 Update

I took a few weeks off from doing anything productive. I went on a trip over the holidays, but now I am back.

I worked a bit on nftables. I went to the Austin Linux Meetup last week, and one of the attendees and I had a chat about firewalls. He was not too keen on things like ufw as a front to iptables, and felt it was just better to learn iptables. I think that the plan is for nftables to replace iptables.

I also worked a bit on some commands to do random passphrase generation on the command line. I will post about that soon.

I worked a bit on my lightning talk for Austin Clojure on HugSQL. I might not prepare too many slides, or I might not prepare any slides at all.

You’re welcome.

A Hacker News Comment About Emacs

I have started using emacs a bit more lately. I spent a bit of time upgrading packages and getting CIDER to work. I would like to share a comment that was left by someone  going by “maehwasu” (perhaps a disciple of Deganawida) on Hacker News on August 26, 2015 (or whatever day was 818 days ago from today).

OP is getting hate, but I write a lot of Clojure code in Emacs, and I find the following commands cover 97%+ of my usage, maybe more. For config I use Emacs Live out-of-the-box, and remap Command on OSX to be my Ctrl. That’s it.

C-a, C-e, C-b, C-f, C-p, C-n, C-v, M-v for line/screen maneuvering.

C-o for Ace-jump to go to the start of any words I see on the page.

C-d to delete. C-s to search for words not on the page.

C-k, C-y, and also how to use them with paredit to yank whole s-expression chunks.

Buffer switching and killing.

File saving and opening.

I teach the above to any friends who want to learn Emacs/Clojure/LISP, leaving out a few other features like whole word deletion and more advanced paredit. With halfway decent aptitude and a little practice, they become productive very quickly (within a few hours, which is a pretty good investment relative to “normal” text editors in my book).

You’re welcome.

2017-06-18 Update

Not much going on. I am still working slowly through Simply Scheme. I am at a point where they want you to diagram some functions. Like diagramming sentences in junior high.

There was an article on Bloomberg about quantum computing (Hacker News page here). That started me doing a lot of surfing about quantum computing. IBM has the IBM Quantum Experience, which allows you to try quantum programs on their quantum cloud.

There is a wiki about QC called Quantiki. One page is a page with a list of simulators. One of them is a page called Quantum Playground. I found a couple in Java: one by a guy who goes by the name ardeleanasm, and one called JavaQuil. JavaQuil is a port of pyQuil, an API for a project called Forest, which is another QC cloud. It is run by a company called Rigetti Computing.

You’re welcome.

Thoughts On Native, GPU, Groovy, Java and Clojure

One interest of mine is math and scientific computing, although I admit right now my interest is far greater than my knowledge.

It seems like for a lot of math and scientific computing, you either need to do stuff in C, C++ or even Fortran, or use a library in a higher-level language that lets you interface with libraries in C/C++/Fortran. Another new trend is to use code that runs on the GPU, like OpenCL.

Lately I have been looking at a few Java libraries that interface with lower level code. I think it is a good idea to keep learning new things. I think AI and machine learning will become more important over time.

I contributed briefly to the SciRuby project a few years ago. One of their main subprojects is a Ruby wrapper around BLAS, ATLAS and LAPACK. I think some of the instructions for compiling BLAS, ATLAS and LAPACK might still have a few sentences written by yours truly. (It’s a very involved process.) Another one of their projects is a Ruby wrapper/interface around the GNU Scientific Library, or GSL.

One huge language in AI and data science these days is Python, with the NumPy and SciPy libraries. Again, some of the functionality is provided by linking to ATLAS and LAPACK. If it worked for them, maybe it will work for the JVM family of languages.

There is at least one project in Clojure that does this: Uncomplicate. It has sub-projects for OpenCL and BLAS and ATLAS. Along with Java and Groovy, Clojure is a language I am interested in. But for the time being I cannot use these projects. I think my graphics card is a bit too old and the drivers are too old for many of these libraries.

Uncomplicate is part of the reason I am looking at this area again. Somewhere on the site, the author points at the for development, it is okay to use pre-baked packages for ATLAS and LAPACK. One of these projects that I mention says for the best performance, you may want to hire someone to install ATLAS and LAPACK and other native libraries optimized for your production hardware. I did not know people made a living doing just that.

Another reason I am looking at this is GPU computing was mentioned in one of the side talks at the most recent ClojureConj. This is second- or third-hand, but apparently Goldman Sachs runs a nightly report analyzing the market (or some part of it, I am not too clear). It used to take nine hours to run. It was refactored to run on the GPU, and now it takes 20 minutes. I think all developers should become familiar with some of these technologies.

Will any of this gain traction in the Groovy space? I know things have been rough for Groovy for the past couple of years, especially after getting dropped by Pivotal. It seems like a lot of Groovy developers are happy making web apps with Grails. I like Grails, and it has worked out well for a lot of people. I will go through the Groovy Podcast backlog; perhaps Groovy developers are looking into this and I am just not aware of it.

Groovy developers can use the Java wrappers for native and GPU computing. No need to re-invent the wheel. Maybe going from Groovy to Java to native is not quite as fast as going from Python or Ruby to native, but I don’t think so. Even Ruby developers joke that Ruby is pretty slow. They say that sometimes the speed of the programmer is more important than the speed of the program. (But for some reason the Ruby devs who are moving to Elixir are now saying performance and concurrency are important after years of saying they were not. Whatevs.)

I do not have much experience in native or GPU programming. I am not too clear how NumPy and SciPy are used. I don’t want to swallow the Python ecosystem to get the answers to just a few questions, but: Are NumPy and SciPy programs (or bare-metal math/science apps in general) rewritten frequently? Or do they stay the same for years at a time? It seems to me if a GPU program does not change, you might be better off biting the bullet and doing things in C or C++.

But given the success of NumPy and SciPy, perhaps Groovy and Java can be used in this space. Maybe it’s not the end of the world if a bank’s nightly report takes 22 minutes. (How successful is SciPy? Not only are there conventions for general Python, there are also conferences for just SciPy). I doubt any technology will dethrone Python anytime soon, but it is an area to look at.

Having looked a small bit at some of these libraries, I do wonder how much can the unique features of Groovy be used. A lot of the tests deal with arrays of primitive types.  Someone on the Groovy mailing list wrote: I’ve found that you can really shrink down the lines of code just by using the basics that Groovy provides around collections, operator overloading, Groovy truthiness, etc.. Maybe that is enough reason to use Groovy for high performance computing (or at least higher performance). Can we make code calling native and GPU libraries better with features from Groovy, or will we just get Java without semicolons? Some of these functions do not want an ArrayList of objects, they want a fixed-length array of integers. Granted, you can make primitive arrays in Groovy:

but I think that is more keystrokes than in Java.

Just looking at a small part of TensorFlow, here is a line from the original Java test:

Here are two Groovy versions that both work (meaning the tests still pass):

Granted, it’s one line, but I’d say that’s idiomatic Groovy. Perhaps this could be a (small) market opportunity for the Groovy community. Or maybe I’m just crazy.

If you google “groovy jni” the first hit is this page from Object Partners. It mentions JNA and BridJ which I think are used by a lot of the libraries mentioned below. Frankly, it sounds like a lot of work making this sort of thing happen.

Regardless, even if I have to stick with Java and Clojure, I will still keep an eye on these libraries and this space. I don’t think I can become an expert in all of these, but as I said, I think it is important for developers to keep an eye on this spae. I might start some pages about the different libraries, and perhaps share my thoughts on them. Below are a few notes on what I have found so far.

I started by googling “Java BLAS” or “Java GPU”, and at first I only got a few libraries. But as I kept looking, I kept finding more.

The first two I found where jblas and Aparapi. From the jblas website, jblas “is essentially a light-wight wrapper around the BLAS and LAPACK routines.” (And from what I can gather, the correct capitalization is “jblas”.)

Aparapi is a wrapper around GPU and OpenCL code. If you cannot get your GPU drivers installed properly, it will do everything on the CPU. Sometimes I get core dumps running the tests on my Ubuntu laptop. It is about six years old, and has 6 GB of memory, so perhaps that is the issue. But on my 8 GB Windows laptop two of the tests fail. I plan on getting a new laptop soon, so I will try this again when I do. I bet getting all this set up on an Apple laptop is really easy; for the first time in my life I am thinking about buying an Apple product.

TensorFlow has some Java APIs, although they warn that there may be API changes and breakages for languages other than Python and C. This can use either the CPU or the GPU, but for GPUs it can only work with NVidia cards.

The Lightweight Java Game Library works with OpenCL, OpenGL, Vulkan, EGL and a lot of other stuff I have never heard of. As the name suggests, it is primarily for game development, and the wrappers around the underlying native libraries and standards require more knowledge of those underlying native libraries and standards than most libraries.

There is ND4J (N-Dimensional Arrays for Java) and Deeplearning4j. I am not sure if they are related somehow, but they were both started by developers at a company called Skymind.

And then there is the granddaddy: ByteDeco. There is a lot of stuff here. Some of the contributors work on ND4J and Deeplearning4j, and also work at Skymind. This project provides Java interfaces to 21 different C/C++ libraries: video, math, science, AI, robotics, facial recognition, lots of good stuff here. And after looking at their list, yes, I think “Skymind” is a bit too close to “Skynet”. But at least they’re giving you a hint.

You’re welcome.

No Longer Using aTunes

First off, that is not a typo: I really meant aTunes and not iTunes.

For the past few years, I have used an application called aTunes to fetch and listen to podcasts. One reason is I am pro-Java, and I generally prefer to use Java technologies where and when I can. Another is that it is NOT something from Apple. I have always found it strange that lot of people in technology will say they are pro-open source, and they don’t trust Microsoft or other big corporations and think are they too smart to let The Man tell them what to do, yet they will buy anything from Apple without question. I like to strive for consistency.

aTunes has not been updated for a couple of years. I know that the makers of the JDK strive for backwards compatibility, but I think at some point it might be best to use something that is maintained. Plus I had a hard time getting aTunes to work the last few times I tried it. I think I had to go a couple of versions back on one of my laptops.

There is another music player written in Java called Jajuk, but that has not been updated for a while either.

I did some digging, and I found a multi-platform music player called Clementine. So far I like it.

So far, I am still using jEdit, but I am getting more into emacs lately. There is an emacs Meetup here in Austin. For a long time it met around noon, but now it meets in the evening.

You’re welcome.

Image of aTunes from Wikipedia, image of Clementine from Wikipedia. Images assumed allowed under Fair Use.