Last week I did a lot of debugging on the Debian server to try and find out what was the issue. And initially it was Dovecots memory usage. After disabling this tool for a couple of days the server was still running fine. However the day after I posted the article on Dovecot the server crashed again. So I had to restart my investigation.

First off I had to had to get a better memory management tool, so I installed Htop on the server (apt-get htop). This shows the current memory usage of each running application. After installing this I enabled all services and applications again and started running stress tests. And though Dovecot was causing some peeks in memory usage it did not keep the high memory usage after the requests where done.

As it turns out for some reason Apache 2.2 was using a lot of memory during peak loads. But even more frustrating it didn’t seem to release any memory any more. Which was causing issues for services that only spawn when they are being accessed like Dovecot and Postfix, which explained why both of these services crashed when the server halted.

After tweaking the maximum amount of servers Apache is allowed to start and the maximum amount of client threads to handle the memory usage dropped dramatically. And I am very happy to report that the server has been running again for more then a week, without any glitches.

Still it doesn’t explain why all of this only happened after updating my server with the latest versions and patches. But I’m glad it’s solved for now.

Now off course it is not the fault of Linux , but with my very lacking skill. I recently did an apt-get upgrade command on the VPS. To my suprise it started updating with Lenny packages, whilst I could have sworn I have Etch installed. Mystery on its own, but worse yet the server now keeps hanging and breaking.

I figured it was a memory issue at first cause some weird service was installed that was eating it alive. So I de-installed that and it ran fine for a few days. But allas, after 3 days of running the VPS hangs itself up again. When I say hangs-up I mean that I cannot connect using SSH, the mailserver goes down. MySQL and Apache keep running though, as does postfix. So I’m still guessing it is memory, but I have no idea why my configuration is no longer working after running an apt-get upgrade command. Something is really wrong .

Doing this in the MySql setup created before should be relatively easy. First of modify the ‘/etc/aliasses‘ file and add ‘devnull: /dev/null‘. This will add a virtual debian user called devnull to the system. Keep in mind this alias is not yet active. To activate it run: newaliases. This will compile the file and load it into the memory.

Now open either PostfixAdmin or MySql and add an alias from ‘@domain.com’ to ‘devnull’. Which will catch all e-mail sent to domain.com and store it into the file belonging to user devnull, which in our case is the blackhole .

Please be carefull, as all mail sent to the domain will be dropped unless either an alias or a mailbox is defined for the domain.

So now to repair my mistake I am adding the following features to my already existing mail server. To find out how to setup the  mail server itself read my earlier article.

• Use spamassassin to mark spam in the header
• Change the mail delivery to dovecot LDA, this is needed for step 3
• Automatically move marked e-mails to the spam folder of the user

Sounds easy right. Well it should be .

Install spamassassin and set it up in postfix

The first step we will take is installing and setting up the spamassassin tool. First lets install it with the default options from Debian by running:


apt-get install spamassassin
useradd -g spamd -s /bin/false -m -d /home/spamassassin spamd

Next we setup the basics of spamassassin configuration, the file called /etc/spamassassin/local.cf

report_safe        0
required_score     2.0
use_bayes          1
bayes_auto_learn   1
use_bayes_rules    1

# Some basic configuration
score DCC_CHECK 4.000
score SPF_FAIL 10.000
score SPF_HELO_FAIL 10.000
score RAZOR2_CHECK 2.500
score BAYES_99 4.300
score BAYES_95 3.500
score BAYES_80 3.000

# Headers to be added for all scanned messages
add_header all Status _YESNO_, score=_SCORE_ required=_REQD_ tests=_TESTS_ autolearn=_AUTOLEARN_ version=_VERSION_
add_header all Level _STARS(*)_
add_header all Checker-Version SpamAssassin _VERSION_ (_SUBVERSION_) on _HOSTNAME_

So what we’ve done so far is set spamassassin up to automatically learn from mail it receives and to mark anything with a bayes score over 2 as spam. But we still need to integrate it into postfix to get it to work. Add the following to the postfix configuration (/etc/postfix/master.cf):

spamassassin unix -     n       n       -       -       pipe
 user=spamd argv=/usr/bin/spamc -f -e
 /usr/sbin/sendmail -oi -f ${sender} ${recipient}

You will also need to change the smtp line in the same file to the following, this will set up spamassassin as a pre delivery filter:

smtp      inet  n     -     -     -     -     smtpd -o content_filter=spamassassin

This will add the spamassassin service as a unix service through a pipe connection. Sounds complicated but it’s really easy it basically uses a binary file to comunicate between postfix and spamassassin. Make sure the user spamd exists and create the folder /usr/bin/spamc with full access to the spamd user. Also edit the file spamassassin to the following:

# /etc/default/spamassassin
# Change to one to enable spamd
ENABLED=1

# SpamAssassin uses a preforking model, so be careful! You need to
# make sure --max-children is not set to anything higher than 5,
# unless you know what you're doing.
SAHOME="/var/lib/spamassassin/"
OPTIONS="--create-prefs --max-children 5 --username spamd --helper-home-dir ${SAHOME} -s ${SAHOME}spamd.log"
PIDFILE="${SAHOME}spamd.pid"
CRON=0

Make sure to create the directory /var/lib/spamassassin and give spamd full access right to it, otherwise the service might not run properly. Spamassassin is now setup and will start scanning your e-mail. So if all you are interested in is the header being changed to indicate if a message is spam then you are done. If you also wish yo move it to a different folder then continue reading.

Settup Dovecot as LDA (Local Delivery Agent)

Next step is setting up dovecot as the default LDA for postfix. This is needed to let dovecot filter messages into the proper locations. First setup the connection to the between postfix and dovecot by adding the following to the /etc/postfix/master.cf:

dovecot   unix  -       n       n       -       -       pipe
 flags=DRhu user=dovecot:dovecot
 argv=/usr/lib/dovecot/deliver -f ${sender} -d ${recipient}

Please note that we now need to keep track of the fact that dovecot will be the user that is running this part of the mail server. So every file needed, log or configuration and mailboxes related, need to be owned by dovecot:dovecot. I have had issues with files having the wrong owner so it is important! Before activating the postfix connection lets configure dovecot for LDA by editing the /etc/dovecot/dovecot.conf file. Add the following piece of code in the group auth default >> socket listen:

        master {
          path = /var/run/dovecot/auth-master
          user = dovecot
          group= ssl-cert
          mode = 0600
        }


This will setup the authentication service that dovecot will use for delivering the e-mail to the correct user and authenticating users. Also add the following at the bottom of the file to configure the LDA of dovecot:

protocol lda {
  mail_plugins = cmusieve
  sieve_global_path = /etc/dovecot/globalsieverc/global.sieve
  log_path = /var/log/dovecot-delivery_log
  postmaster_address = noreply@domain.com
}

plugin {
 sieve = /etc/dovecot/globalsieverc/global.sieve
}

Again make sure that the user dovecot has full access to both the logfile and the global sieve directory (which you will need to make) . The plugin group is just for safety. I don’t know if it is needed but it doesn’t break anything. Also check the dovecot-mysql.conf that you made earlier to make sure that the uid and gid are those of the dovecot user. You can find this out by running:

:> id dovecot

If this is all setup in dovecot restart it by running:

:>/etc/init.d/dovecot force-reload

Now it is time to wrap things up and activate dovecot as the LDA for postfix, edit the file /etc/postfix/main.cf and change the virtual_transport from virtual to dovecot. Last but not least is creating the globe.sieve file.

:> vi /etc/dovecot/globalsieverc/global.sieve

require ["fileinto"];
if anyof (
    header :contains ["X-Spam-Flag"] "Yes"
) {
    fileinto "Spam"; stop;
}

Now restart postfix and send yourself a test mail. If everything is setup correctly it should be delivered to your mailbox. If not then first check ‘/var/log/mail.info’ to gather some information as to what is going on. If this indicates a delivery was attempted but it failed then check ‘/var/log/dovecot-delivery_log’ to find out more.

I hope you have fun using your spam free mail server. If you have any issues let me know, but please include snippets from the log as this will help me help you

My setup is as follows:

• Postfix as a SMTP service
• Dovecot as the IMAPS service and authentication service
• MySQL as a storage location indicating the virtual email boxes and domains
• PostfixAdmin as a package to manage the virtual domains and boxes

Installing the whole thing

The first thing you might wanna do is install the needed services and applications on your Debian machine. You can install everything using:

apt-get install dovecot-common dovecot-imaps postfix postfix-mysql

At this point I am asuming you already have MySQL installed and configured. If not then you should do so, but I won’t include that in this tutorial. The PostfixAdmin needs to be downloaded from http://sourceforge.net/projects/postfixadmin/. You will need to setup this somewhere on the webserver site, either in an already configured apache directory or by configuring a new one.

Setting up everything in MySQL

After everything is installed you will need to prepare everything in MySQL. Connect to your machine and run the following code to generate a database and tables in that database:

create database postfix;
grant all on postfix.* to postfix identified by 'postfix';
grant all on postfix.* to postfixadmin identified by 'postfixadmin';
set password for postfixadmin = old_password('postfixadmin');

This will setup the basic stuff for postfix and postfix admin to work. Next step is setting up the data for postfix. First configure the postfixAdmin plugin by setting some of the settings in ‘config.inc.php’.

Setup Postfix configuration

The next step is configuring Postfix to enable it to sent mails, use the MySQL database and login using the Dovecot application. The latter you will setup in a later stage. First open the ‘/etc/postfix/main.cf’ file and change the entire content to something like:

queue_directory = /var/spool/postfix
command_directory = /usr/sbin
daemon_directory = /usr/lib/postfix
mail_owner = postfix## Host name given by ISP
myhostname = <isp hostname>
mydomain   = <domainname>
unknown_local_recipient_reject_code = 550
debug_peer_level = 1

# Virtual domain administration MySQL
virtual_alias_maps      = mysql:/etc/postfix/mysql_virtual_alias_maps.cf
virtual_gid_maps        = static:104
virtual_mailbox_base    = /usr/postfix/
virtual_mailbox_domains = mysql:/etc/postfix/mysql_virtual_domains_maps.cf
virtual_mailbox_limit   = 51200000
virtual_mailbox_maps    = mysql:/etc/postfix/mysql_virtual_mailbox_maps.cf
virtual_minimum_uid     = 104
virtual_transport       = virtual
virtual_uid_maps        = static:104

# The settings for the SASQL authentication using the autdaemon.
smtpd_recipient_restrictions =
   permit_mynetworks,
   permit_sasl_authenticated,
   reject_unauth_destination,
   reject_unauth_pipelining,
   reject_invalid_hostname,
   reject_rbl_client list.dsbl.org,
   reject_rbl_client sbl-xbl.spamhaus.org
smtpd_sasl_auth_enable           = yes
broken_sasl_auth_clients         = yes

# Setup authentication using Dovecot
smtpd_sasl_type = dovecot
smtpd_sasl_path = private/auth

In the above configuration you need to manually change the domain name to your primary domain and the ISP domain to the name given by your ISP. Another thing you must change is the number 104 this should be the UID of the postfix user on your system. You can find out what the UID is by running: 

 id postfix

Setting up the Postfix SQL files

The last step in the postfix configuration is setting up the SQL files that instruct postfix on how to load the data from the MySQL database. Run the commands as instructed below.

:> vi /etc/postfix/mysql_virtual_mailbox_maps.cf

user     = postfix
password = <password>
hosts    = <ipaddress>
dbname   = postfix
query    = SELECT maildir FROM mailbox WHERE username='%s' AND active = 1

:> vi /etc/postfix/mysql_virtual_domains_maps.cf

user     = postfix
password = <password>
hosts    = <ipaddress>
dbname   = postfix
query    = SELECT domain FROM domain WHERE domain='%s'

:> vi /etc/postfix/mysql_virtual_alias_maps.cf

user     = postfix
password = <password>
hosts  = <ipaddress>
dbname = postfix
query  = SELECT goto FROM alias WHERE address='%s' AND active = 1 

Change the password and the host name to the ones as configured in your MySQL server. If everything went correct your postfix should be up and running by now. Please note that the logon when sending mail does not work yet as dovecot has not yet been configured.

Configuring Dovecot

Your last step should be easy. Edit the dovecot.conf file into the following.

protocols = imaps
log_path  = /var/log/dovecot_log
log_timestamp = "%Y-%m-%d %H:%M:%S "

default_mail_env      = maildir:/usr/postfix/%d/%n
mail_privileged_group = mail
first_valid_uid       = 100

protocol imap {
  imap_client_workarounds = outlook-idle
}

##
## Authentication processes
##
auth_username_chars = abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@
auth default {
  mechanisms = plain login
  userdb sql {
    args=/etc/dovecot/dovecot-mysql.conf
  }
  passdb sql {
    args=/etc/dovecot/dovecot-mysql.conf
  }
  socket listen {
    client {
      path = /var/spool/postfix/private/auth
      mode = 0660
      user = postfix
      group = postfix
    }
  }
}


This simple configuration will enable dovecot to use IMAPS (you could also add POP or IMAP). The auth default group defines how to authenticate the user. In our case this is using the MySQL database to locate passwords and the directory of the user. We also define a socket in this group that indicates a location where Dovecot should offer the authentication service for other application, being Postfix in our case.

The last step for the dovecot setup is creating the query to find the data. Run the following command and then paste the code below:

:> vi /etc/dovecot/dovecot-mysql.conf

driver = mysql
connect = host=<host> dbname=postfix user=postfix password=<password>
default_pass_scheme = CRYPT
password_query = SELECT password FROM mailbox WHERE username = '%u'
user_query = SELECT maildir, <uid> AS uid, <uid> AS gid FROM mailbox WHERE username = '%u'

You should change the host, password and UID. The UID is the one you previously obtained for postfix.

If everything is done correctly you can start adding domains, mailboxes and aliasses using PostfixAdmin. As well as starting the Postfix and dovecot service. You can do this by running:

:> /etc/init.d/dovecot start
:> postfix start

And you’re all done. Congratulations you now have a mail service running on you Debian machine. If you have any difficulties please let me know and please post a snippet of the ‘/var/log/mail.info’ to help me understand what’s going on. Trust me that I’ve probably seen most of the errors that you can encounter come by.

So far I ran every website on Debian Sarge, and though it’s stable they choose to no longer developer for it nor support it. So it was time to move to a newer version. Wich I did yesterday. Here’s some of the steps you should take before upgrading to the next version of Debian.

• Make a full backup of any website’s / databases / scripts running (don’t forget your crontab and logrotate files)
• Copy the backup to a directory not affected by the upgrade. In my VPS the provider has one folder which will be left untouched, but also pull the backup to a local system (just in case )
• Notify any other users also using your server, don’t want any suprises to potential paying customers.

I was lucky that my hosting provider offers an easy way to upgrade or reinstall a Linux distro on the VPS. Just a couple of mouse clicks and they prepare the VPS for the installation. Please note that this will take up to an hour to complete. Once this base installation is complete you will want to perform the following steps:

1. Un rar the backup of the websites / conf files and scripts (tar xf <tar name>) and move them to the right location.
2. Setup the proper user account /groups to have access to these files (or your users won’t be able to upload or change the website’s any more)
3. Install apache / PHP and mysql by running the following command:
     apt-get install php5 mysql5-server apache2 mysqllib
4. Setup the apache conf file to load in your restored conf files (eg: Include /export/conf/apache/*.conf)
5. Replace the mysql my.cnf with the one you backed up previously
6. Enable the rewrite and ssl modules of apache (I forgot and got some upset calls )

From this point on your ready to go. All of the websites should be running without any problems. You do need to configure any other tools you had installed, like AWStats / Subversion or anything else. Though these should not require to much configuration as all of the configuration files where included in your backup. At least if you followed a setup similar to my first article on setting up linux.