Apache James

I am using Apache James for this site. I have made a few articles on getting Apache James 2 running. Version 3 came out recently. I have not tried it yet. When I do, I will update my pages. Here are some articles I have posted about Apache James:

Bear in mind that I am not offering any support for James. I am just doing this because I am such a wonderful person. You’re welcome.

These pages reference the following files: Here is the James file tree before running James. Here is the James file tree after running James. Here is the default James configuration file that is generated the first time James is run. Here is the script that I made to start and stop James. I posted this to the James user list, and somebody said they found it helpful.


Updated:
2009-08-20 01:38:45 Thursday CDT -0500

2009-06-16 23:35:32 Tuesday CDT -0500

2009-04-23 00:14:23 Thursday CDT -0500

2009-02-16 10:34:14 Monday CST -0600

2009-02-08 00:34:46 Sunday CDT -0500

2008-09-24 18:59:46 Wednesday CDT -0500

 

James Start Script For Linux

I have made a script to start James when the system starts up. It can also be used to start, stop and restart James. I use this script on systems running Ubuntu Linux. It can probably be used with modifications on any Linux or Unix-based system.

The “update-rc.d james start 50 0 1 2 3 4 5 6 . stop 50 0 1 2 3 4 5 6 .“ command on line 2 calls the update-rc.d script. This is a Debian script that will make links to the script in the /etc/rcX.d directories to the script in /etc/init.d. 50 is the number that will determine the order the script will be run. If you have dependencies for James (like DNS or a database), those should have a lower number than the James script in the /etc/rcX.d directories. If you are storing messages in a database and it is not running when James starts, then James will quickly stop. Not setting this high enough has stumped a few people on the mailing list.

The PATH and JAVA_HOME variables should also be set in the script. Startup scripts will not pick up environment variables from any users. If they are not set here, the phoenix.sh script will not work.

Using Alpine With James

I have been using James on a few hosts for a while, and for the most part it has gone pretty well. I use James for all the receiving on these hosts and a lot of spam is caught.

For a while I was able to use James for sending, but recently Comcast started blocking port 25. So I was not getting all the James goodness. On one of my sites I started using the Alpine mail client. I used to use Pine, so I thought I could be a real man and not read any documentation. (Alpine is the successor to Pine.) According to Wikipedia, Linus Torvalds, the man who started Linux, uses Alpine. (According to the article used by Wikipedia as a reference, he also does not use GUIS, chat or a cel phone; I assume he does not “tweet” either.)

I had to make a few configuration changes. I had to specify the domain for the POP protocol. The changes showed up in the .pinerc file like this:

I was then able to receive, but not able to send. I kept getting some errors which kept changing so fast I was not able to read them. I do not know if Alpine leaves any logs, but I was not able to find much in the James logs. The last of the error messages was:

>SECURITY PROBLEM: insecure server advertised AUTH=PLAIN<
Mail not sent. Sending error: 334 OK. Continue authentication.

I searched on Yahoo and found a page which said to add this to the .pinerc file:

disable-these-authenticators=PLAIN

The issue was discussed here which references this page.


Updated:

2009-04-23 00:13:00 Thursday CDT -0500

Managing Users With a Database

In a previous post, I added users via the Remote Manager. The default is to store user information on the filesystem. This requires using the Remote Manager to update users. Some people do not like this, since it makes changing user information without logging in very difficult. Someone posted a perl script on the mailing list using a telnet library that could be invoked from a web application.

An easier and more flexible method is to manage user accounts via a relational database. This would allow you to build a web application to manage James users. Searching through the mailing list archives, I found inserting user information into the database seemed to be a common topic. It is not covered very well on the James web site; I was not able to find anything about it. Not to brag, but this may be the first comprehensive explanation with all the info to get you started in one place. A lot of information is scattered in mailing list posts. It is a little frustrating that a common question has not been covered on the James website. Even more frustrating is that the whole procedure is pretty simple.

First I will cover the configuration file. The relevant section is config/users-store/repository, which starts on line 1161. Comment out the filesystem section, and uncomment the database section. You could also set the Remote Manager enabled attribute to “false” at line 778 in the default configuration file (section config/remotemanager). Upload your new config file, and restart James.

The crux of the matter of inserting user accounts is that the database stores the passwords in a one-way hash. The database also stores a string which should contain the name of the hash algorithm. When the user logs in, James hashes the password and compares it to the value in the database. I think a lot of the confusion centers around creating the hash. One post mentioned that you must use the JDK to create the hash, and not the database. Apparently the hashing algorithms are not all compatible. I am not an encryption expert, but I have been able to figure this out.

There is a class in James that can be run on the command line to create the hashes. It is org.apache.james.security.DigestUtil. It can be found in $JAMES_HOME/work/james-$EPOCH_TIMESTAMP/SAR-INF/lib. I assume that it is originally in the SAR file $JAMES_HOME/apps/james.sar, and is expanded when James is first run. To use the class, you need james-2.3.1.jar and the JavaMail jar.

Here is how to use it:

[prompt]

Usage: java org.apache.james.security.DigestUtil [-alg algorithm] [-file] filename|string

[prompt]

Hash is: rVVGgoar2pVf/mIzegYL5mWo

You must do this for each user on your system.

org.apache.james.security.DigestUtil uses the java.security.MessageDigest class to hash the password. The algorithms that it uses are MD2, MD5, SHA-1, SHA-256, SHA-384 and SHA-512. These are listed in the Java Cryptography Architecture API Specification & Reference Appendix A, which itself refers you to the Standard Names Document.

Then log into your database, and do the insert. James gets its SQL statements from the file $JAMES_HOME/apps/james/conf/sqlResources.xml. The statement for inserting a user into the database in the XML file is:

So you would do something like this at the command line:

You would never believe how long it took to get THAT hash.

The hash column can hold up to 50 characters with the default configuration. The MD2 and MD5 algorithms create hashes of 20 characters. The SHA-256 algorithm creates a hash of 40 characters. The SHA-384 algorithm creates a hash of 64 characters. The SHA-512 algorithm creates a hash of 76 characters. You could dig around in $JAMES_HOME/apps/james/conf/sqlResources.xml and change the “create table” statement. Or you could do this:


Updated:

2009-02-16 10:35:39 Monday CST -0600

Configuring James to Store Messages in a Database

In this post, I will demonstrate how to configure James to use a database instead of the filesystem to handle and store email messages. For the time being, I will still use the filesystem to store information about users.

First off: create a database. I am using MySQL. You should just run a few statements to create the database, as well as a user and give that user permissions. James will create the tables when it is started and configured to use a database. Put in the database information in the config.xml file in the config/database-connections section, which is from lines 1188 to 1270 in the default config file. There are commented out blocks for many databases: Apache Derby, MySQL, Microsoft MSSQL, and hsqldb (formerly HypersonicSQL).

There are quite a few places in which we must change the configuration file in order for the database to process messages. There is an inbox repository, there is a spool repository, and several processors. There is also an nntp repository, but I have no desire to run a news server. Also, it appears that the news server can only use the filesystem. To disable the news server, you can set the “enabled” attribute to “false” for the “nntpserver” element. This is at config/nntpserver, which is at line 929 in the default configuration file. As stated, the user repository will still use the filesystem for the time being.

There are three ways that James stores data. The default is to use the filesystem. This is done in the configuration file with XML attributes like destinationURL=”file://var/mail/inboxes/”. To use a database, this attribute value is replaced with “db://<data-source>/<table>”. The <data-source> should match the “name” attribute in the config/database-connections/data-source element. By default it is “maildb” throughout the configuration file. The third option is to store message delivery and headers in the DB, and the body to the filesystem. This has a value of “dbfile://maildb/inbox/”. All we need to do is comment out the elements with the attribute destinationURL=”file://var/some/path”, and uncomment the corresponging elements with the attribute  destinationURL=”db://maildb/someTable”. Usually these elements are one right after the other.

Inbox repository: config/James/inboxRepository, line 66,
“ToRepository” mailet: config/spoolmanager/processor@name=”error”/mailet@class=”ToRepository”, line 435
“RemoteDelivery” mailet: config/spoolmanager/processor@name=”transport”/mailet@class=”RemoteDelivery”, line 543
“ToRepository” mailet: config/spoolmanager/processor@name=”spam”/mailet@class=”ToRepository”, line 627
“ToRepository” mailet: config/spoolmanager/processor@name=”local-address-error”/mailet@class=”ToRepository”, line 680
“ToRepository” mailet: config/spoolmanager/processor@name=”relay-denied”/mailet@class=”ToRepository”, line 706
Spool repository: config/spoolrepository, line 994

There are several elements with filesystem URLs in the config/mailstore/repositories section, but I think these can be left as-is.

Next the new configuration file and the proper JDBC JAR file should be uploaded. The configuration file should go to $JAMES_HOME/apps/james/SAR-INF/config.xml. The JDBC JAR file should go to $JAMES_HOME/lib.


Updated:

2009-02-16 10:30:57 Monday CST -0600

Adding Users In James

To add users in James, you must use the Remote Manager. This is a command-line utility used to manage user accounts. The user account info is stored on the file system. You can also manage user accounts in an RDBMS. I will cover configuring James to use an RDBMS in a later post. You can use a database to handle some tasks (like storing mail), while using the file system for others (such as managing users). I will cover using a database in a later post.

The default James configuration file has a section for the Remote Manager beginning on line 778, and ending on line 806. To use the Remote Manager, you must telnet to the machine running James on the port specified in the configuration file. By default it is port 4555. The James web site has a page about configuring the Remote Manager, but I was not able to find any information about using it. It is pretty easy to use, but it would be nice if there was a list of the commands. You can get a list of the commands by typing help:

Here is a quick sample session:

 

user@machine:/path/to/james$

I have been able to configure James to use a database to store and process messages, but I have not been successful in getting James to use a database to store and manage user accounts. I read on the mailing list that the password must be encrypted using a particular algorithm (SHA I believe). The password must be encrypted using Java’s implementation of the algorithm, and not the database’s. I have not looked into this in any great detail. I plan in configuring James to use a database to manage users.

The James site has a page on adding user accounts via the Remote Manager, but nothing that I can see on using a database for this purpose. (There is an introductory page on setting up a database.) There is an XML file included with James that has the SQL statements that James uses for every database operation, but it did not give many answers (or perhaps I did not read it thoroughly enough). But there is nothing wrong with doing this in stages.


Updated:

2009-02-16 00:37:33 Monday CST -0600

Configuring James

In order to get James to do something useful, you need to edit the configuration file. This file is $JAMES_HOME/apps/james/SAR-INF/config.xml. You can view the default James configuration file here. To edit this file I used NetBeans, since it has good XML editing features. It has drop-down menus which make it easy to find sub-tags in XML. In this post I use notation for XML tags like this: root/tag/sub-tag/sub-sub-tag.

First, you need to put in the IP address of your DNS servers. On a Unix or Linux machine, you can get this information by looking at /etc/resolf.conf. Put the IP addresses in the config/dnsserver/servers tag, with a “server” tag for each IP address. This will be at lines 758-765 in the default config.xml file.

Next, type the domain names you want James to handle. As far as I know, even if you type in two different domains, like “daydomain.com” and “nightdomain.com”, then any user accounts you put into James will be for both domains. So joe.schmoe@daydomain.com will be the same user as joe.schmoe@nightdomain.com. Put the domains in the config/James/servernames section, with a servername for each domain. This will be lines 54-57 in the default config file.

It is also a good idea to change the password (and maybe the username as well) for the Remote Manager. This is a utility that James uses to manage users. User information can be stored in the database, but so far the documentation for that is not very good. To use Remote Manager, you must log into the server upon which James is running, and telnet to the port specified at config/remotemanager/port, which is line 778 in the default config file. The username and password information is at config/remotemanager/handler/administrator_accounts section, which is at lines 795-799 in the default config file. You may also want to change the server name in the config/remotemanager/handler/helloName section to match what you entered in the config/James/servernames/servername section, but I do not think this is necessary.

It is also good to change the postmaster address, which is line 35 in the default config file, at config/James/postmaster.

Upload the file, restart James, add some users, and you are ready to go.
Updated:

2009-02-16 00:24:07 Monday CST -0600

Installing Apache James

I have been running Apache James on this site for a while. I will now list the steps I went through to install James and get it running. A lot of this information mirrors what is on the Quick Start Guide at the Apache James Wiki. I am running Apache James on an Ubuntu Linux machine.

Download the latest tar file from the Apache site, and upload it onto the machine that will be running James.

Make sure your firewall allows the email ports.
Extract the tar file.
Go to the $JAMES_HOME/bin directory.

You may have to change permissions on a few of the shell scripts to get James to run.

At this point, you can run $JAMES_HOME/bin/run.sh, or $JAMES_HOME/bin/phoenix.sh
Running $JAMES_HOME/bin/phoenix.sh gives this:

So run

Let it go for a few minutes.
Then run

Running the file started the Avalon/Phoenix server, and expanded the James app which was in the file $JAMES_HOME/apps/james.sar. (I guess SAR files are another type of archive, unique to Avalon, similar to JAR , EAR and WAR files.) Now there is a new file tree in $JAMES_HOME/apps/james. Here is the James file tree before running James. Here is the James file tree after running James.

After this, you must make some changes to the default James configuration file that was generated in $JAMES_HOME/apps/james/SAR-INF/config.xml. I will cover this in a future post.
Updated:

2009-02-15 23:56:47 Sunday CST -0600

A Few Words On Apache James

One of the applications that I will run on this site is the Apache James email server. One of the reasons I am using this is because it is written in Java, so it helps fulfill the site’s mission of promoting Java.

Briefly, I will discuss what James is, why I have decided to use it, and then a few words on how I will configure it and what my plans are.

James is an email and news server written in Java. It runs within Apache Avalon, which from what I can gather is some sort of a container or kernel, similar to Pico Container, Tomcat or Spring. Avalon has been dropped as an Apache project and absorbed within Apache Excalibur. There are references in the file names, scripts and documentation to “Phoenix”, which appears to be some sort of code name for Avalon, just as Catalina is a code name for Tomcat.

The configuration is handled by a (very large) xml file. James can handle SMTP and POP protocols. It can be configured to store emails and user information either in a database, or on a filesystem.

I have run a few other domains on VPS servers as practice. A lot of the users opted to use Google Mail or a third-party email provider,  like FastMail or HushMail. I decided to run a mail server myself, although I may wind up regretting that.

Another reason that I chose James is that it can handle SMTP and POP. SMTP is used to send emails from one email server to another. POP is used by client applications (such as Columba, Thunderbird or Outlook) to read email on a server. There is also another protocol to retrieve mail called IMAP, but I did not have any interest in that.

Looking around, I noticed that not all email servers were able to handle both sending and retrieving. I only wanted to configure one server. There is one called Courier, but I was a put off  by some of the documentation. I found a few web pages with titles like, “Configuring an Email Server with Postfix AND Courier AND Amavis  AND ClamAV AND SpamAssassin AND MySQL blah blah blah etc etc”. That sounds like more that I wanted to handle. It turns out that James can be difficult to configure, but only dealing with one server just suits me better.

A lot of the servers that did both protocols either did not run on Linux, were proprietary (yes, there is proprietary software that runs on Linux), or was part of a collaboration or groupware suite with lots of stuff I did not want (like calendaring or scheduling). There was another Java email server called the “Java Email Server”, but I was not able to get that to run as a service via a script on /etc/init.d. Either I would have to hit Control-C or run a “kill” command. I could have tried using the Java wrapper from Tanukisoft, but once again I wanted to just use one thing.

I plan on writing about how I am going to install and configure James. There is a quick start guide on Apache’s site, that I will wind up parroting at first. It is easy to get a James server running that receives emails via SMTP, and can allow a client to read them via POP and to store information on the filesystem. Setting it up to handle SMTP authentication or send email to non-existent users to the database takes more configuration. Beyond that, the documentation is pretty scattered in wikis, web pages and the mailing list. Granted, putting more stuff on yet another blog might make that worse, but at least things should go in a linear fashion. I will also reference the default config.xml file as I go forward and make changes.


Updated:

2009-02-15 23:49:37 Sunday CST -0600

I may look for something to replace Apache James

I think I may soon look for another mail server to replace Apache James. Right now I am using 2.3.2, which will someday be obsolete. I downloaded the new version, 3.0-M2. It looks like Apache James may no longer be a mail server that can be used by “retail users”. I am running it on a VPS account with 512 MB of memory.

The new version has IMAP. It also uses OpenJPA and Spring (before it was using some other Dependency Injection framework from another Apache project that almost nobody else used). It can be managed via JMX. The project seems to be geared towards the “Enterprise”. James 2 uses ~60 jar files. James 3 uses about 125.

I ran James 3 on my desktop to see if I could port my admin web app to James 3. Unlike James 2, James 3 does not create the tables upon startup. I emailed the list and I was told that I would have to add some users or domains in order to create tables. I was told via the email list that I could add a user via JMX or the remote manager (which I just do not like).

So I started James 3, and then I started jconsole. I was able to connect to James via jconsole. But I was not clear on what to do after that. For example, I saw a screen for adding a user. But the parameters on the form/screen are labelled “p1”, “p2” and “p3”. I assumed having worked with James 2 that they would be username, encrypted password, and encryption algorithm, but it would be nice if jconsole was clearer on this. I tried adding a user. I put “SHA-256” in the last field, and I got an exception message telling me “user repository does not exist: SHA-256”. Looking in other windows, I found that there is a repository called “LocalUsers”. I tried that and got a message about java.rmi.UnmarshalException. So I decided to stop at that point.

Then I tried the remote manager. I really do not like the remote manager. I do not like having unnecessary services and ports open. But I thought that until I get some tables created I have no choice. I logged in, and I tried a few commands. I tried adding a user a couple of times and adding a domain. Each time the remote manager hung, and I had to kill James. I could run the commands to list users and domains. The only command that returned any information was the command to list domains, but it only listed the domains that I put in the domainslist.xml file. So no tables were created.

So I think I will start looking for another email server. Perhaps I will try to write one myself in Groovy. I have started looking into Groovy and Grails again recently. I would rather run something myself than use GMail.

Image from Wikimedia, assumed allowed under Fair Use. Image from the Vatican Virgil, a 5th century manuscript of poems by Virgil.