Mail service
Postfix/Dovecot (v2) on Ubuntu 11.xx

Warning!

This is quite old!

The following description is related to the situation when mail clients are virtual users, i.e., they have no system accounts (no records in /etc/passwd), all info about users (including passwords) is stored in a database (MySQL), and mail server handles e-mail for virtual domain(s). Virtual domain is any legal Internet domain which is external for a mail server (i.e., the mail server is not a part of such domain). Canonical domain is the mail server's own domain.

Another important note: in this example SMTP client authentication is based on Dovecot SASL, not Cyrus SASL!

First of all,

be sure your server has a real FQDN (see hostname):

hostname --fqdn

Check/change date/time and TZ (must correspond to the server's location). To change time zone data in Ubuntu:

dpkg-reconfigure tzdata

Software installation

(some packages may be already installed)

apt-get update

While installing Postfix, choose "Internet Site" in config dialog:

apt-get install postfix postfix-mysql

MySQL is going to be a backend database:

apt-get install mysql-server

Installing pop3/imap server (Dovecot version 2.* is assumed):

apt-get install dovecot-pop3d dovecot-imapd dovecot-mysql dovecot-sieve

(the Dovecot Sieve plugin provides mail filtering facilities at the stage of the final msg delivery; using Sieve language /RFC 5228/ you can write delivery customization scripts).

The web-server (for administration, webmail, and stats gathering):

apt-get install apache2

PHP modules, libs, apps, etc.:

apt-get install php5 php5-mysql php5-mcrypt php5-intl phpmyadmin

Miscellaneous utilities, that can be useful for mail server:

apt-get install wget tcpflow dnsutils nmap unzip whois

The Bayes and spf antispam filters:

apt-get install spamassassin pyzor razor

apt-get install tumgreyspf

Note! Antispam config is not considered here!

Installing Apache

Actually, mail service does not require Apache or any other web server, but there are many reasons (like web mail, misc web/php admin tools) to have it.

First of all, we need a certficate for Apache (mail service requires passwords, password over HTTP requires SSL, and SSL requires certificates). Self-signed certificate is bad for a big, serious, crictical, financial, etc. system, but this is (what can I say?) very popular choice for simple, local, internal, experimental, ... system. So be it, let's create this trivial, self-made, self-signed certificate (which, besides all, should not request a password when Apache starts):

openssl req -new > new.cert.csr

At this point we'll be prompted to enter some info about organization, PEM pass phrase (prepare beforehand), etc:

...

If this stage is O.K., let's do the rest:

openssl rsa -in privkey.pem -out new.cert.key

openssl x509 -in new.cert.csr -out new.cert.cert -req -signkey new.cert.key -days 1825

If there is no ssl dir in /etc/apache2, create it (or, if you prefer /etc/ssl, modify config):

cp new.cert.cert /etc/apache2/ssl/server.crt

cp new.cert.key /etc/apache2/ssl/server.key

Now we must configure DNS for a first domain to be able to address the server. Use a2enmod apache2 module config script:

a2enmod rewrite

a2enmod ssl

All unnecessary files from /var/www/ (most likely index.html) should be deleted or moved to some arc location (postpone this step to be able to test the web server access), and some new dirs (for future use) should be created:

mkdir /var/www/webmail

mkdir /var/www/mysql

Ubuntu 11.10 installation usually includes two templates:

/etc/apache2/sites-available/default

/etc/apache2/sites-available/default-ssl

Edit one of them (default-ssl is more appropriate in our case), or create totally new, but remember, web server will only read files referenced by symlinks in /etc/apache2/sites-enabled/ dir, be sure the symlink points to the right file. Actually, there are scripts a2ensite and a2dissite to handle this situation.

There are many ways to set options in this file, see, for example, /etc/apache2/sites-available/default (if you take it, replace names of files, certificates, etc). This config allows access through the standard tcp/80 port and tcp/443 port (SSL).

Also, we have to set some basic rules in
/etc/apache2/conf.d/security:

<Directory>
  AllowOverride None
  Order Deny,Allow
  Deny from all
</Directory>

ServerTokens Prod

ServerSignature Off

Restart Apache and test web server access from the net:

/etc/init.d/apache2 restart

Check/change PHP config /etc/php5/apache2/php.ini, pay attention to the following params:

display_errors = off
upload_max_filesize = 20M
date.timezone = Europe/Rome

Enable default site (a2ensite creates symlinks, but it would make no harm if all those symlinks were created manually):

a2ensite default

Once again restart Apache.

MySQL database

It looks like nowdays everybody wants to use Postfix with a database (usually MySQL). It really provides some useful features.

The name of the database and the name of the mail administrator in the database are quite random, let it be mailserver and postman. Use a reasonably long password (8+).

The following description assumes the use of MySQL client mysql (see MySQL usage notes):

mysql -u root -p

If root login is disabled (e.g., due to empty root password, like in Ubuntu), and you cannot connect to MySQL server, see Setting MySQL root password in Ubuntu. When you're normally connected:

mysql> CREATE DATABASE mailserver;

mysql> USE mailserver;

mysql> GRANT ALL ON mailserver.* TO 'postman'@'localhost' IDENTIFIED BY 'gdp501wste';

mysql> exit

Now it's time to create database objects. Let's make it conveniently, using a carefully prepared SQL script executed by mysql. Move this script to an appropriate dir and run:

mysql mailserver -u postman -p < mailserver.sql

Configuring Postfix

There are some files to be created to provide interaction between Postfix and the database. They look alike, except SQL query string. Be sure it is consistent with your database struct (table/column names, etc).

Virtual domain config file:

/etc/postfix/mysql-virtual-domains.cf

user = postman
password = gdp501wste
hosts = 127.0.0.1
dbname = mailserver
query = SELECT id FROM virtual_domains WHERE name='%s'

Virtual forwarding config file:

/etc/postfix/mysql-virtual-alias-maps.cf

user = postman
password = gdp501wste
hosts = 127.0.0.1
dbname = mailserver
query = SELECT destination FROM virtual_aliases WHERE source='%s'

Virtual mailbox config file:

/etc/postfix/mysql-virtual-mailbox-maps.cf

user = postman
password = gdp501wste
hosts = 127.0.0.1
dbname = mailserver
query = SELECT email FROM users1 WHERE email='%s'

Virtual e-mail mapping file:

/etc/postfix/mysql-virtual-email2email.cf

user = postman
password = gdp501wste
hosts = 127.0.0.1
dbname = mailserver
query = SELECT email FROM users1 WHERE email='%s'

Make sure that all these config files have group postfix and access mode 640 (it's important, because those files contain plaintext pass).

Also, a special OS user is required to handle our mail (postman is a database user, not related to OS). On a system that auto creates a unique group for each new user (Fedora, Ubuntu):

useradd -m vmail

or, if keeping mail in /home is not desired,

useradd -m -d /var/vmail vmail

Find UID and GID in /etc/passwd. Alternative approach (with total control of names and numbers):

groupadd -g 5000 vmail

useradd vmail -u 5000 -g vmail -m -d /home/vmail

The following cmds are required to complete the Postfix config:

postconf -e virtual_mailbox_domains=mysql:/etc/postfix/mysql-virtual-domains.cf

postconf -e virtual_mailbox_maps=mysql:/etc/postfix/mysql-virtual-mailbox-maps.cf

postconf -e virtual_alias_maps=mysql:/etc/postfix/mysql-virtual-alias-maps.cf, mysql:/etc/postfix/mysql-virtual-email2email.cf

postconf -e 'virtual_uid_maps = static:5000'

postconf -e 'virtual_gid_maps = static:5000'

The following example shows how to insert a new domain into the database (NOTE! example.com is used for demonstration, in fact, this table must keep valid domain names only):

mysql mailserver -u postman -p

...

mysql> INSERT INTO virtual_domains VALUES (1, 'example.com');

Let's check that Postfix can execute the query (if everything is O.K., the following cmd must output 1):

postmap -q example.com mysql:/etc/postfix/mysql-virtual-domains.cf

Now you can create new mail users in following manner:

mysql> INSERT INTO virtual_users (domain_id, user, password) VALUES (1, 'pro07', MD5('stp4e39mix'));

Here's a database cmd to add a forwarding record:

mysql> INSERT INTO virtual_aliases (domain_id, source, destination) VALUES (1, 'postmaster', 'alex@example.com');

Testing cmds:

postmap -q postmaster@example.com mysql:/etc/postfix/mysql-virtual-alias-maps.cf

postmap -q info@example.com mysql:/etc/postfix/mysql-email2email.cf

Each of the above cmds should return the queried mail address if it exists in the database, otherwise an empty string is returned.

See also the final version of main.cf as it looks after dovecot config is done.

Configuring Dovecot

Dovecot provides POP3 and IMAP services. To establish relation between Postfix and Dovecot, we must add the following line to

/etc/postfix/master.cf

dovecot  unix  -   n    n   -   -   pipe   flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -d ${recipient}

(all that stuff is a single line!)

Next step is to modify Postfix config:

postconf -e virtual_transport=dovecot

postconf -e dovecot_destination_recipient_limit=1

To activate changes, Postfix must be restarted:

postfix reload

Warning!

There are some differences between Dovecot 1.* and Dovecot 2.* config. The following description assumes Dovecot 2.*.

The default Dovecot config file

/etc/dovecot/dovecot.conf

is the assembly of the multiple include files, and you have to choose: to edit all those files or to create a single clean config file with all settings in one place. Here's an example of a single config file dovecot.conf.

There is also dovecot-sql.conf for database connectivity:

driver = mysql
connect = host=127.0.0.1 dbname=mailserver user=postman password=gdp501wste
default_pass_scheme = PLAIN-MD5
password_query = SELECT user, domain, password FROM users2 WHERE user='%n' AND domain='%d';
user_query = SELECT user, password, 1001 AS uid, 1001 AS gid, '/var/vmail/%d/%n' AS home, 'maildir:/var/vmail/%d/%n' AS mail FROM users2 WHERE user='%n' AND domain='%d';

Note that substitution params mean:

'%u' - entire userid (like user@domain);

'%n' - user part of user@domain;

'%d' - domain part of user@domain;

To see actual Dovecot config:

doveconf -a | more

Or you can select non-default settings only:

doveconf -n | more

Starting/stopping Dovecot (Ubuntu Linux 10.*/11.*):

sudo start dovecot

...

sudo stop dovecot

The Dovecot log is (according to the above dovecot.conf):

/var/vmail/dovecot-deliver.log

Both Postfix and Dovecot use certificates to restrict access and reject bad clients (those without mail certificates). In our case these certificates are not the same that are used by Apache web-server. In fact, certificates can be inherited from an old mail server, if you're upgrading mail system; or created as described above. Anyway, it requires some config lines in Postfix main.cf and dovecot.conf, and these lines are usually added/modified in the end of the installation process when mail service is already functional, i.e., can send and receive mail.

postconf -e smtpd_sasl_type=dovecot

postconf -e smtpd_sasl_path=private/auth

postconf -e smtpd_sasl_auth_enable=yes

postconf -e smtpd_tls_cert_file=/etc/ssl/certs/servcert.pem

postconf -e smtpd_tls_key_file=/etc/ssl/private/servkey.pem

postconf -e smtpd_use_tls=yes

postconf -e smtpd_tls_auth_only=no

Testing SMTP

TEST # 1 (connecting SMTP locally)

At the server:

telnet localhost 25

It should respond with:

Trying 127.0.0.1...

Connected to localhost.

Escape character is '^]'.

220 mx1.example.com ESMTP Postfix (Ubuntu)

At this point enter:

ehlo localhost

It should output something like this:

250-mx1.example.com

250-PIPELINING

250-SIZE 10240000

250-VRFY

250-ETRN

250-STARTTLS

250-AUTH PLAIN LOGIN

250-ENHANCEDSTATUSCODES

250-8BITMIME

250 DSN

Now you can send a test message:

mail from:<pro@example.com>

250 2.1.0 Ok

rcpt to:<postmaster@example.com>

250 2.1.5 Ok

data

354 End data with <CR><LF>.<CR><LF>

Test 1

.

250 2.0.0 Ok: queued as E2604200068

That's enough for a simple test:

quit

221 2.0.0 Bye

Connection closed by foreign host.

TEST # 2 (connecting SMTP from a LAN host)

If this test fails, first try ping (to be sure that SMTP server is accessible) and nmap (to be sure that firewall is set right).

At the client computer/workstation:

telnet 192.168.0.2 25

It should respond with:

Trying 192.168.0.2...

Connected to 192.168.0.2.

Escape character is '^]'.

220 mx1.example.com ESMTP Postfix (Ubuntu)

You should enter the following cmd with your host's IP (or name):

ehlo 192.168.0.25

The rest is like in the previous example.

TEST # 3 (connecting SMTP from a LAN host using TLS)

At the client computer/workstation:

openssl s_client -connect 192.168.0.2:25
-starttls smtp

In case of success server returns a big block of misc info that ends with:

  ...

  Compression: 1 (zlib compression)

  Start Time: 1331709237

  Timeout : 300 (sec)

  Verify return code: 18 (self signed certificate)

---

250 DSN

At this point you can send:

ehlo 201.37.89.208

and the rest goes like in the previous examples.

TEST # 4

If previous tests are successful, try to send a real mail message to a real external mail address (e.g., your own gmail.com acount).

Remember that a real mail server must have an appropriate PTR record. If you do not belong to a big company with PI (provider independent) address space, than only your ISP can do this for you.

Testing POP3/IMAP

TEST # 5 (testing Dovecot/pop3 from a workstation)

telnet 192.168.0.2 pop3

Server should respond with:

Trying 192.168.0.2...

Connected to 192.168.0.2.

Escape character is '^]'.

+OK Dovecot Ready.

Send your name:

user pro7

+OK

Send your password:

pass stp4e39mix

+OK Logged in.

Try some commands:

list

+OK 0 messages:

.

quit

+OK Logging out.

Connection closed by foreign host.

TEST # 6 (testing Dovecot/imap from a workstation)

telnet 192.168.0.2 imap2

Server should respond with:

Trying 192.168.0.2...

Connected to 192.168.0.2.

Escape character is '^]'.

* OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE STARTTLS AUTH=PLAIN AUTH=LOGIN] Dovecot ready.

Send your name and password:

1 login pro07 stp4e39mix

1 OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS] Logged in

Try some commands:

2 list "" "*"

* LIST (\HasChildren) "." "INBOX"

2 OK List completed.

3 select "INBOX"

* FLAGS (\Answered \Flagged \Deleted \Seen \Draft)

* OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted.

* 0 EXISTS

* 0 RECENT

* OK [UIDVALIDITY 1332158528] UIDs valid

* OK [UIDNEXT 1] Predicted next UID

* OK [HIGHESTMODSEQ 1] Highest

3 OK [READ-WRITE] Select completed.

6 logout

* BYE Logging out

6 OK Logout completed.

Connection closed by foreign host.

Now you can try to access mail boxes using a mail client program.

Web mail (roundcube)

Assuming that Apache is configured according to ... (see Apache config), download the Roundcube archive, unpack, copy to /var/www/webmail, read INSTALL.

Create a database named roundcubemail, grant privileges to rouncube@localhost (Note! The password below is just for example, you should provide your own):

mysql> CREATE DATABASE roundcubemail;

mysql> GRANT ALL PRIVILEGES ON rouncubemail.* TO 'roundcube'@'localhost' IDENTIFIED BY 'wsr8nvji32';

mysql> exit

From dir /var/www/webmail exec following cmd to create the database objects (tables):

mysql roundcubemail -u roundcube -p
< SQL/mysql.initial.sql

On success, open in browser http://srvname/installer and follow instructions. When finished, you can edit new config files manually (in my case the database password was not written to a config file, so I had to edit db.inc.php).

Also, check /var/www/webmail/.htaccess, there is at least one line that you may want to change:

php_value     upload_max_filesize   5M

Final notes

While setting Dovecot, put in dovecot.conf:

mail_debug = yes

auth_debug = yes

With these options you can see in /var/vmail/dovecot-deliver.log what is wrong, and how all those database queries look like at runtime. I had to change password_query and user_query in /etc/dovecot/dovecot-sql.conf many times before it started to work right.

mail_location = maildir:/var/vmail/%d/%n

does not work in my case (domain part of the path mystically disappears), but everything is O.K. if user_query returns the same maildir. Also, it's useful to keep in mind the difference between home dir and maildir (virtual users have no home dir in classical Unix/Linux sense).