When it comes to handling my work-related email, many people will probably conclude that I am an old fart in Linux terms: I still use the console-based Pine as my MUA of choice, as my fingers have been hardcoded to its key combinations over the years (my first encounter with Pine was around 1994 on a shared DEC Ultrix box in my university). So far, I have not found any other application that allows me to process email as quickly as by using this tool.
I admit that I do use Thunderbird for my personal email, though, to gather experience with it (and to toy around with the various extensions, especially Nostalgy is a gift from heaven for people like me!). And of course because the mail volume there is less critical to cope with! But I am not going to start a holy war here - this article is supposed to explain how I have configured my mail setup to rotate the log files that keep track of all my incoming mails.
For getting and filtering my mails I rely on two other commandline workhorses that have been around for a while: fetchmail for receiving email from remote POP3 and IMAP mail servers, and procmail, for the local delivery of email into the respective mail folders. Both tools provide a vast range of functionality and are very effective. They also offer good logging mechanisms that help me to keep track the email flow.
To enable logging in procmail I added the following directive to my personal ~/.procmailrc file:
LOGFILE=$MAILDIR/.procmail.log
To enable logging in fetchmail, I added the following line to my ~/.fetchmailrc configuration file:
set logfile "/home/lenz/Mail/.fetchmail.log"
However, these log files tend to grow significantly over time and keeping these logs in a single file quickly becomes cumbersome to handle (and difficult to back up). I therefore needed to find a solution that takes care of rotating the log files automatically, without me having to worry about it. For this, I utilize another commandline utility named logrotate, as shipped with openSUSE (other distributions may supply alternative tools for rotating logfiles, so the following may not fully apply there).
I created a logrotate configuration file ~/.logrotate.conf in my home directory that looks as follows:
# Global options
# Old versions of log files are compressed with gzip by default.
compress
# Log files are rotated the first time logrotate is run in a month
monthly
# Log files are rotated 12 times before being removed (just keep one year of history)
rotate=12
# Archive old versions of log files adding a daily extension like YYYYMMDD instead of simply adding a number.
dateext
# If the log file is missing, go on to the next one without issuing an error message.
missingok
# Log files to rotate
/home/lenz/Mail/.fetchmail.log { }
/home/lenz/Mail/.procmail.log { }
Now I need to make sure that the logrotate script is involved frequently to take care of the log file rotation. This could probably be done by creating a cron job, but I have chosen a different approach, to make sure that the files are not rotated while the fetchmail process is just performing its work. Fortunately fetchmail provides an option to perform arbitrary actions before fetching a new batch of emails from the server, so this is where I decided to invoke the log rotation. From my ~/.fetchmailrc:
# Perform log rotation (see ~/.logrotate.conf)
preconnect '/usr/sbin/logrotate -s /home/lenz/.logrotate.status /home/lenz/.logrotate.conf'
The above line invokes logrotate using my local configuration file every time fetchmail attempts to fetch new email. On the beginning of a new month, the log file is renamed with a timestamp and compressed with gzip. Files older than 12 months are deleted automatically. This is how the log file archive in my ~/Mail folder looks like at the moment:
lenz@metis:~> ls -ahl Mail/*.log*
-rw-r--r-- 1 lenz users 132K 2008-02-06 13:57 Mail/.fetchmail.log
-rw-r--r-- 1 lenz users 174K 2007-03-05 12:45 Mail/.fetchmail.log-20070305.gz
-rw-r--r-- 1 lenz users 243K 2007-04-01 00:01 Mail/.fetchmail.log-20070401.gz
-rw-r--r-- 1 lenz users 178K 2007-05-01 00:00 Mail/.fetchmail.log-20070501.gz
-rw-r--r-- 1 lenz users 179K 2007-06-01 00:00 Mail/.fetchmail.log-20070601.gz
-rw-r--r-- 1 lenz users 179K 2007-07-01 00:01 Mail/.fetchmail.log-20070701.gz
-rw-r--r-- 1 lenz users 231K 2007-08-01 00:02 Mail/.fetchmail.log-20070801.gz
-rw-r--r-- 1 lenz users 88K 2007-09-03 11:42 Mail/.fetchmail.log-20070903.gz
-rw-r--r-- 1 lenz users 71K 2007-10-08 09:27 Mail/.fetchmail.log-20071008.gz
-rw-r--r-- 1 lenz users 87K 2007-11-01 00:11 Mail/.fetchmail.log-20071101.gz
-rw-r--r-- 1 lenz users 83K 2007-12-01 00:03 Mail/.fetchmail.log-20071201.gz
-rw-r--r-- 1 lenz users 50K 2008-01-02 12:53 Mail/.fetchmail.log-20080102.gz
-rw-r--r-- 1 lenz users 43K 2008-02-01 00:21 Mail/.fetchmail.log-20080201.gz
-rw------- 1 lenz users 261K 2008-02-06 13:57 Mail/.procmail.log
-rw------- 1 lenz users 324K 2007-03-05 12:45 Mail/.procmail.log-20070305.gz
-rw------- 1 lenz users 417K 2007-04-01 00:01 Mail/.procmail.log-20070401.gz
-rw------- 1 lenz users 351K 2007-05-01 00:00 Mail/.procmail.log-20070501.gz
-rw------- 1 lenz users 353K 2007-06-01 00:00 Mail/.procmail.log-20070601.gz
-rw------- 1 lenz users 370K 2007-07-01 00:01 Mail/.procmail.log-20070701.gz
-rw------- 1 lenz users 319K 2007-08-01 00:02 Mail/.procmail.log-20070801.gz
-rw------- 1 lenz users 281K 2007-09-03 11:42 Mail/.procmail.log-20070903.gz
-rw------- 1 lenz users 259K 2007-10-08 09:27 Mail/.procmail.log-20071008.gz
-rw------- 1 lenz users 338K 2007-11-01 00:11 Mail/.procmail.log-20071101.gz
-rw------- 1 lenz users 315K 2007-12-01 00:03 Mail/.procmail.log-20071201.gz
-rw------- 1 lenz users 195K 2008-01-02 12:53 Mail/.procmail.log-20080102.gz
-rw------- 1 lenz users 267K 2008-02-01 00:21 Mail/.procmail.log-20080201.gz
Very handy! I can now use zgrep or zless to go through the history of all incoming emails from the past 12 months. Since the individual files are rather small, seaching through them is performed very quickly. And as they are already separated by month, it's easy to limit the search to a certain range (e.g. if I search for the whereabouts of a particular message at a certain date).
article