Category Archives: Clojure

2018-04-22 Update

I am still slowly going through Clojure For the Brave and True. I will keep going for the time being until inspiration hits.

I did try another app that uses the WordPress API. It is called WP-API v2 Java Client. I was not able to get this to work either. I thought (briefly) about making a direct SQL call to insert the post. I turned on SQL logging and decided it might be a bad idea. I have heard that many SQL experts are not impressed with WordPress’s schema. Plus I would either have to refactor (slightly) to use MySQL, or figure out how to use two connections.

Eric Normand of Purely Functional put up his code review of twitter-retriever. Take a look and let me know if it was good. He also has a channel where he talks about functional programming in general. He does these in his backyard.

You’re welcome.

2018-04-08 Update

I think I am done for the time being with the Twitter-retriever.

As of now, it can get the last 3200 tweets of a user (excluding retweets) and capture any tweets going forward. It prints them out in HTML to a file with each tweet in enclosing “li” tags.

I tried to get the WordPress REST API working. I installed WordPress on my laptop, but I had a hard time getting anything to work with the POST command. GET was easy. It is the authentication that stumped me. I got a couple to work with curl, but no POST commands to work with clj-http. I might try again later, but for now I am done. It seemed to be a rabbit-hole of google searches, adding yet another plugin, editing the .htaccess file, etc, etc. For now, what I have is good enough.

I might look into an alternative to environ, but for the time being I think it will do.

I don’t know what I will work on next. I might get back to Simply Scheme. I might go back to coding examples for the Clojure API. I might go through some lessons at Purely Functional. Maybe TensorFlow (like with guildsman or clojure-tensorflow or tensorflow-clj). Maybe Luminus. As of right now, I have not decided.

You’re welcome.

2018-04-01 Update

I am still working on the twitter retriever.

I am almost ready to start looking at the wordpress API. I think I will start a local instance so I can work with the REST API and not clutter up this site.

I also looked a bit more at the Clojure library environ. What I like about it is that it gets all the environment variables at start time, and any old time you want you just call the env function, and it works. No fuss, no getting files in every namespace. But I am not so sure if it would work in a production environment. There are a lot of pages from people saying that putting usernames and passwords in the environment is a bad idea, and I am leaning that way too. I have not been able to get my app to use environment variables in .lein-env, profiles.clj or .boot-env when I run it with “lein run”.

I also found out that (after some trial and error) that I cannot compile my app with “lein uberjar”. I use environ for my database creds, and it looks for them during the uberjar process. I was not able to use the profiles.clj or .boot-env for this either. I had to put the database variables into the environment when I compile. (So maybe there is not security risk during production after all.)  Here is what I did in bash:

That is not a production database, so no big deal if the creds are here. But that seemed to do the trick. I was able to use the uberjar in another window without any problems (from what I remember).

Also: Eric Normand from Purely Functional got back to me. I told him I was nowhere near done. Partially due to not spending as much time as I should, partially parsing the infamous error messages, partially due to find and trying out libraries as I get this up and running. He wants to do a code review. I think I will clean up a few things before I say yes.

Before I do, I would like to copy and paste in here a few functions for the sake of posterity:

Those should parse a string with a timestamp like “Tue Feb 11 15:03:45 +0000 2014” and convert it to “2014-02-11_15:03:45”. I then found clj-time.

I also think I found a couple of function calls you can try in the REPL to interactively work on regular expressions:

I know those are pretty wide, but I found it helpful.

Here is just a little let:

 

You’re welcome.

2018-03-25 Update

I am still working on the Twitter retriever. There is not much too else to tell right now. I am just working on some tests.

You’re welcome.

Addendum:

I got this message in the tests:

I had a var called “result-str”, and I had misstyped “result-string” in the comparison. Usually that is what the cause when you get a stack trace with “clojure.lang.Var$Unbound” in it somewhere.

I do agree that sometimes the stack traces can be hard to read.

2018-03-18 Update

The “due date” for my Clojure project is tomorrow. I am not done.

Part of it is because I did not get started until a week into it. And I have been using “Organic-Oriented Development” (a new form of OOD): I did not plan a whole lot out ahead of time. I have just been trying different APIs and libraries to see what they do. I looked at propertied, but I don’t think I will wind up using it. I may use environ, but I have not decided.

I am working on some of the tests now. The author of the Twitter API library tells you to set his API keys as environment variables and run the tests against one of his Twitter accounts. I am not a TDD-purist, but I think that is not the best way to go. Running a database for tests I have no problem with, but needing something external to be live is a big no-no to me. On the other hand, I don’t think I know enough about any Clojure testing APIs to say what is a better way to do it. Maybe the author is correct.

I have not looked at the WordPress JSON API. I think for the time being I will just write the tweets in HTML format to a file, and copy and paste.

I have also started getting back into meditation, so perhaps that will help me to be more disciplined.

I did mention this to one of the organizers of the Austin Clojure group. I did not volunteer, but I should probably have something working by then in case they need someone to present.

You’re welcome.

2018-03-04 Update

I finally got started on the Twitter retriever app in Clojure. I did not do much the first week.

I am able to get tweets for a couple of accounts. I have to make database tables. I think for the most part I will just follow the structure of what Twitter sends back.

I was dealing with a few libraries to deal with configuration. Important stuff. For the database properties and twitter keys, I am using envron. It uses environment variables for configuration. One of the ideas in the 12 Factor App pattern is to put your configuration in the environment. I did some googling, and not everybody thinks that is always a good idea. Plus, what if you have a LOT of configuration?

In non-prod, environ can take variables/fields from a profiles.clj file if you are using Leiningen. So at first I thought environ sidestepped all the security concerns. But for prod, the Github page says to just use environment variables. For now I will go with this as-is.

What I like about environ is that it should be pretty easy to use for the database configuration. I might use cprop. I will look at it later.

I might stick with propertied for some configuration.

At least now I know a few more libraries.

You’re welcome.

2018-02-25 Update

On the Groovy mail server: I am still unable to send to ProtonMail. I might just shelve it for the time being. I would like to get back to it in the future, but I would like to spend more time on Clojure and the Lisp family.

I will get around to installing iRedMail soon. I can’t run an old version of Apache James forever. I think iRedMail uses Postfix under the hood, which is what ProtonMail uses. Perhaps I will get it all working them.

I had my call with Eric Normand of Purely Functional. At one point we touched on ClojureScript. I have been under the impression that you have to deal with JavaScript to use ClojureScript. As the guy who wrote the Pollen DSL for Racket put it, Lisp makes me smarter, and pretty much any language ending in *Script seems to kill brain cells on contact. There seems to be a lot of wheel re-invention in JavaScript. I always get the feeling that a lot of people in the JS community think that there is nothing to software besides JS. Has any other language been called “stupid” by its creator? And it all seems unstable.

Back in June 2015, Bloomberg wrote this:

JavaScript is fast-moving right now. Too much of what you know today will be useless in six months. Every hard-fought factoid about the absolute best and most principled way to use the language will be fetid zoo garbage by the end of the year. And some sniveling, bearded man-toddler will be looking slightly to your right with his pale, buzzword-infected eyes and awkwardly mumbling, “Yeah, no, wow, it says you have a lot of Gulp and Angular, but I’m guessing you don’t use Fleejob or Grimmex with the Snurt extensions? (Long sigh.) I’m just not sure if you’re gonna like working here.”

Maybe it’s better now, but they thought they were smarter than the rest of us while making their ecosystem a total morass.

Anyway, Mr Normand said he used JavaScript several years ago, and he had to use it again recently, and the libraries and ecosystem was totally different. But in four years, ClojureScript has not really changed.

I mentioned that I do not like 4Clojure. I tried it a few times, and I keep finding myself in a position where I am randomly scanning the Clojure Cheat Sheet looking for a function that might help at that moment. He said a side project is a good way to learn a language. He asked me if there is a problem I would like to solve, and I thought of one.

I am starting on a project to publish tweets to a webpage. I used to use a plug-in called twitter-tools that once a week would take all my tweets and make a WordPress post. It is old, and does not work with more recent versions of WordPress. (I think the developer passed away from cancer a few years ago.) I looked recently and I could not find anything else that has the same functionality.

There are a lot of plugins and widgets that will put your twitter stream on the side. In fact, I think Twitter also has a script for that. The other major category is sending a tweet when you make a blog post.

Granted, you can log into twitter and download your archive, but it would be nice to have something automated.

We will see what I can get done in three weeks.

You’re welcome.

The editorializing is by me, not Mr Normand.

2018-02-18 Update

I spent a bit more time on the Groovy Mail Server. I am still unable to send to ProtonMail, which uses Postfix. I opened up a ticket and sent them a log from tshark, but they were not able to find anything wrong. A lot of people use Postfix. Not being able to send to Postfix might be a show stopper.

I will install iRedMail; the purpose is two-fold: 1. Try to get Groovy Mail Server working with Postfix (and later TLS), and 2. And just to try it out in general. I am currently using Apache James. The version I am using is getting old, and it is time to look at other options. I might go with a newer version, but I figured I should try something else too. Eventually the version of James I am using might not work on a future JDK.

I posted a command that you can use to generate passphrases on a Linux laptop. I also added a page about the core Clojure API. Slowly but surely I am making progress. I might look more at Simply Scheme, but I plan on focusing more on Clojure.

You’re welcome.

2018-02-05 Update

I know it’s a day late, but here is the weekly update.

I worked some more on the Groovy Email Server, even though I said I would not. Sometimes I hit a wall on something, say I am going to stop, but it just bugs me that I left things in a bad state. Then at some point I get back to it.

I was getting a lot of null strings, which was causing exceptions, and (after some changes I had to revert) sometimes infinite loops. It turns out I was not checking to see if the arguments to EHLO were proper domains. Apparently there are some bots that send bad arguments to some commands to try to take them over. At first I thought I was doing something wrong, but now I know it’s not me. I can exchange messages with Apache James, Gmail, Yahoo and Outlook.com, so if you are sending null strings you are doing something wrong. (I think Yahoo used to use QMail, but now I think they have moved to something else.)

I also have to work on delivery status notifications (RFC 3461). I got rejected by a server that told me I was not implementing the protocol correctly. I guess that SMTP extensions can really change things.

Not much Clojure. I have a call in a couple of weeks with the Purely Functional guy. He offered to come up with a lesson plan for people. I have noticed a few job posts in the Clojure mailing list and Brave Clojure requiring people to have paid Lisp/Clojure experience. One huge complaint in the Lisp world is there are not enough Lisp jobs. I want to email these companies and ask them where I can get the Lisp job that will enable me to apply for the Lisp job they are offering.

You’re welcome.