Author Archives: Eric MacAdie

2018-03-18 Update

The “due date” for my Clojure project is tomorrow. I am not done.

Part of it is because I did not get started until a week into it. And I have been using “Organic-Oriented Development” (a new form of OOD): I did not plan a whole lot out ahead of time. I have just been trying different APIs and libraries to see what they do. I looked at propertied, but I don’t think I will wind up using it. I may use environ, but I have not decided.

I am working on some of the tests now. The author of the Twitter API library tells you to set his API keys as environment variables and run the tests against one of his Twitter accounts. I am not a TDD-purist, but I think that is not the best way to go. Running a database for tests I have no problem with, but needing something external to be live is a big no-no to me. On the other hand, I don’t think I know enough about any Clojure testing APIs to say what is a better way to do it. Maybe the author is correct.

I have not looked at the WordPress JSON API. I think for the time being I will just write the tweets in HTML format to a file, and copy and paste.

I have also started getting back into meditation, so perhaps that will help me to be more disciplined.

I did mention this to one of the organizers of the Austin Clojure group. I did not volunteer, but I should probably have something working by then in case they need someone to present.

You’re welcome.

2018-03-04 Update

I finally got started on the Twitter retriever app in Clojure. I did not do much the first week.

I am able to get tweets for a couple of accounts. I have to make database tables. I think for the most part I will just follow the structure of what Twitter sends back.

I was dealing with a few libraries to deal with configuration. Important stuff. For the database properties and twitter keys, I am using envron. It uses environment variables for configuration. One of the ideas in the 12 Factor App pattern is to put your configuration in the environment. I did some googling, and not everybody thinks that is always a good idea. Plus, what if you have a LOT of configuration?

In non-prod, environ can take variables/fields from a profiles.clj file if you are using Leiningen. So at first I thought environ sidestepped all the security concerns. But for prod, the Github page says to just use environment variables. For now I will go with this as-is.

What I like about environ is that it should be pretty easy to use for the database configuration. I might use cprop. I will look at it later.

I might stick with propertied for some configuration.

At least now I know a few more libraries.

You’re welcome.

2018-02-25 Update

On the Groovy mail server: I am still unable to send to ProtonMail. I might just shelve it for the time being. I would like to get back to it in the future, but I would like to spend more time on Clojure and the Lisp family.

I will get around to installing iRedMail soon. I can’t run an old version of Apache James forever. I think iRedMail uses Postfix under the hood, which is what ProtonMail uses. Perhaps I will get it all working them.

I had my call with Eric Normand of Purely Functional. At one point we touched on ClojureScript. I have been under the impression that you have to deal with JavaScript to use ClojureScript. As the guy who wrote the Pollen DSL for Racket put it, Lisp makes me smarter, and pretty much any language ending in *Script seems to kill brain cells on contact. There seems to be a lot of wheel re-invention in JavaScript. I always get the feeling that a lot of people in the JS community think that there is nothing to software besides JS. Has any other language been called “stupid” by its creator? And it all seems unstable.

Back in June 2015, Bloomberg wrote this:

JavaScript is fast-moving right now. Too much of what you know today will be useless in six months. Every hard-fought factoid about the absolute best and most principled way to use the language will be fetid zoo garbage by the end of the year. And some sniveling, bearded man-toddler will be looking slightly to your right with his pale, buzzword-infected eyes and awkwardly mumbling, “Yeah, no, wow, it says you have a lot of Gulp and Angular, but I’m guessing you don’t use Fleejob or Grimmex with the Snurt extensions? (Long sigh.) I’m just not sure if you’re gonna like working here.”

Maybe it’s better now, but they thought they were smarter than the rest of us while making their ecosystem a total morass.

Anyway, Mr Normand said he used JavaScript several years ago, and he had to use it again recently, and the libraries and ecosystem was totally different. But in four years, ClojureScript has not really changed.

I mentioned that I do not like 4Clojure. I tried it a few times, and I keep finding myself in a position where I am randomly scanning the Clojure Cheat Sheet looking for a function that might help at that moment. He said a side project is a good way to learn a language. He asked me if there is a problem I would like to solve, and I thought of one.

I am starting on a project to publish tweets to a webpage. I used to use a plug-in called twitter-tools that once a week would take all my tweets and make a WordPress post. It is old, and does not work with more recent versions of WordPress. (I think the developer passed away from cancer a few years ago.) I looked recently and I could not find anything else that has the same functionality.

There are a lot of plugins and widgets that will put your twitter stream on the side. In fact, I think Twitter also has a script for that. The other major category is sending a tweet when you make a blog post.

Granted, you can log into twitter and download your archive, but it would be nice to have something automated.

We will see what I can get done in three weeks.

You’re welcome.

The editorializing is by me, not Mr Normand.

2018-02-18 Update

I spent a bit more time on the Groovy Mail Server. I am still unable to send to ProtonMail, which uses Postfix. I opened up a ticket and sent them a log from tshark, but they were not able to find anything wrong. A lot of people use Postfix. Not being able to send to Postfix might be a show stopper.

I will install iRedMail; the purpose is two-fold: 1. Try to get Groovy Mail Server working with Postfix (and later TLS), and 2. And just to try it out in general. I am currently using Apache James. The version I am using is getting old, and it is time to look at other options. I might go with a newer version, but I figured I should try something else too. Eventually the version of James I am using might not work on a future JDK.

I posted a command that you can use to generate passphrases on a Linux laptop. I also added a page about the core Clojure API. Slowly but surely I am making progress. I might look more at Simply Scheme, but I plan on focusing more on Clojure.

You’re welcome.

Some Work With Collections

Here are some functions used for working with collections (vectors, maps, sets). I did not include too much with lists, since I think a lot of Clojure people prefer vectors.


You’re welcome.

Command Line Passphrase Generation

I figured out another way to calculate passphrases, this time using the command line.

To cut to the chase, here is the command:

I found out about a command called “shuf” which according to its man page can “generate random permutations”. You can run this against a word list. Many Unix-based systems have a word list.

On my system, the word list is at /usr/share/dict/words, which is a link to /etc/dictionaries-common/words, which itself is a link to /usr/share/dict/american-english. You can also find one online here.

After you run the list against “shuf”, you need to put them all on one line. You can do this with the “tr” command (see man page here). You can use it to “translate or delete characters”. I want to replace the carriage return-new line combination with a space. In the command, we can represent that combination with either ‘\r\n’ or [:cntrl:]. I prefer [:cntrl:] because the only way to represent the space is with a space between single quotes, and I think using one less pair cuts down on ambiguity (“tr [:cntrl:] ‘ ‘” as opposed to “tr ‘\r\n’ ‘ ‘”).

Even though the word list is “american-english”, it still has some characters that we do not use, like vowels with umlauts or the letter “A” with a circle over it (I guess that is a “super-umlaut”). There are also a lot of words in there with an apostrophe followed by an “s”, words with upper-case letters and words with less than 4 characters. Taking out all of that takes us from a file with 100K words to one with about 26K words.

Here is the full grep sequence I used:

You’re welcome.

2018-02-11 Update

This week there was not much Clojure. I have the call with the Purely Functional guy next Monday. He has a conference this Thursday and Friday. I am not attending. I think a few people from Austin Clojure are going.

I spent more time on the Groovy Mail Server. I just don’t know when to quit. I cannot send mail to Proton Mail. I think they are using Postfix. Every time I try to send an RCPT command, they return “550 5.5.1 Protocol error”. If I am reading the Postfix source code correctly, that can happen if you send an invalid command (RCPT is valid) or if you send a new line either by itself or without a carriage return. I do not think I am sending a command in the wrong form. For one thing, just about every other server I try accepts my mail.

I also had problems with I was getting a socket exception. From what I gathered from googling, they were ending the socket after the QUIT command. I was not having this problem with mails before. I added a try/catch block, and now I save the emails.

I don’t know how much longer I will continue the Groovy Mail Server. Part of me wants to complete it. On the other hand, since Clojure and Lisp are the ultimate direction I want to go in, maybe I should put it aside. On the first hand, it is always good to keep up with a few extra languages. I would also like to get through Simply Scheme and then SICP, because that would make me a really smart person.

If I really wanted to finish my mail server, I would probably have to get another VPS account and set up something like mail-in-a-box for TLS. I might need a few attempts to get TLS right, and I don’t want everybody banning my emails while I figure it out. There also might be an issue with TLS. I don’t know a whole lot about setting up certificates or how to set up a certificate keystore in Java or how to use keytool. I have Let’s Encrypt certs, but those only last for a few months. I don’t know if you can re-import a cert into a Java app while the app is running. Sometimes it seems like there is not a lot of overlap between security and development.

You’re welcome.

2018-02-05 Update

I know it’s a day late, but here is the weekly update.

I worked some more on the Groovy Email Server, even though I said I would not. Sometimes I hit a wall on something, say I am going to stop, but it just bugs me that I left things in a bad state. Then at some point I get back to it.

I was getting a lot of null strings, which was causing exceptions, and (after some changes I had to revert) sometimes infinite loops. It turns out I was not checking to see if the arguments to EHLO were proper domains. Apparently there are some bots that send bad arguments to some commands to try to take them over. At first I thought I was doing something wrong, but now I know it’s not me. I can exchange messages with Apache James, Gmail, Yahoo and, so if you are sending null strings you are doing something wrong. (I think Yahoo used to use QMail, but now I think they have moved to something else.)

I also have to work on delivery status notifications (RFC 3461). I got rejected by a server that told me I was not implementing the protocol correctly. I guess that SMTP extensions can really change things.

Not much Clojure. I have a call in a couple of weeks with the Purely Functional guy. He offered to come up with a lesson plan for people. I have noticed a few job posts in the Clojure mailing list and Brave Clojure requiring people to have paid Lisp/Clojure experience. One huge complaint in the Lisp world is there are not enough Lisp jobs. I want to email these companies and ask them where I can get the Lisp job that will enable me to apply for the Lisp job they are offering.

You’re welcome.

2018-01-28 Update

This week I did not do much Clojure. I did work on my Groovy mail server. I am starting to use it for my own mail, replacing Apache James. I will work on TLS next.

Update 2018-01-29_02.00.43:
I may put aside the Groovy mail server project, perhaps permanently. I thought I was on a roll, but now I am having issues sending. I cannot figure out what is wrong. I did make a change to my firewall, but I changed back and restarted the server. Still not able to send.

You’re welcome.