Mail service
Postfix/Dovecot (v2) on Ubuntu 11.xx
Warning!
This is quite old!
Introduction | Software install |
Apache config | MySQL objects |
Postfix config | Dovecot config |
Testing SMTP | Testing POP3/IMAP |
Web mail | Final notes |
The following description is related to the situation when mail clients are virtual users, i.e., they have no system accounts (no records in /etc/passwd), all info about users (including passwords) is stored in a database (MySQL), and mail server handles e-mail for virtual domain(s). Virtual domain is any legal Internet domain which is external for a mail server (i.e., the mail server is not a part of such domain). Canonical domain is the mail server's own domain.
Another important note: in this example SMTP client authentication is based on Dovecot SASL, not Cyrus SASL!
First of all,
be sure your server has a real FQDN
(see hostname
):
hostname --fqdn
Check/change date/time and TZ (must correspond to the server's location). To change time zone data in Ubuntu:
dpkg-reconfigure tzdata
Software installation
(some packages may be already installed)
apt-get update
While installing Postfix, choose "Internet Site" in config dialog:
apt-get install postfix postfix-mysql
MySQL is going to be a backend database:
apt-get install mysql-server
Installing pop3/imap server (Dovecot version 2.* is assumed):
apt-get install dovecot-pop3d dovecot-imapd dovecot-mysql dovecot-sieve
(the Dovecot Sieve plugin provides mail filtering facilities at the stage of the final msg delivery; using Sieve language /RFC 5228/ you can write delivery customization scripts).
The web-server (for administration, webmail, and stats gathering):
apt-get install apache2
PHP modules, libs, apps, etc.:
apt-get install php5 php5-mysql php5-mcrypt php5-intl phpmyadmin
Miscellaneous utilities, that can be useful for mail server:
apt-get install wget tcpflow dnsutils nmap unzip whois
The Bayes and spf antispam filters:
apt-get install spamassassin pyzor razor
apt-get install tumgreyspf
Note! Antispam config is not considered here!
Installing Apache
Actually, mail service does not require Apache or any other web server, but there are many reasons (like web mail, misc web/php admin tools) to have it.
First of all, we need a certficate for Apache (mail service requires passwords, password over HTTP requires SSL, and SSL requires certificates). Self-signed certificate is bad for a big, serious, crictical, financial, etc. system, but this is (what can I say?) very popular choice for simple, local, internal, experimental, ... system. So be it, let's create this trivial, self-made, self-signed certificate (which, besides all, should not request a password when Apache starts):
openssl req -new > new.cert.csr
At this point we'll be prompted to enter some info about organization, PEM pass phrase (prepare beforehand), etc:
...
If this stage is O.K., let's do the rest:
openssl rsa -in privkey.pem -out new.cert.key
openssl x509 -in new.cert.csr -out new.cert.cert -req -signkey new.cert.key -days 1825
If there is no ssl dir in /etc/apache2, create it (or, if you prefer /etc/ssl, modify config):
cp new.cert.cert /etc/apache2/ssl/server.crt
cp new.cert.key /etc/apache2/ssl/server.key
Now we must configure DNS for a first domain to be able to address the
server. Use a2enmod
apache2 module config script:
a2enmod rewrite
a2enmod ssl
All unnecessary files from /var/www/ (most likely index.html) should be deleted or moved to some arc location (postpone this step to be able to test the web server access), and some new dirs (for future use) should be created:
mkdir /var/www/webmail
mkdir /var/www/mysql
Ubuntu 11.10 installation usually includes two templates:
/etc/apache2/sites-available/default
/etc/apache2/sites-available/default-ssl
Edit one of them (default-ssl is more appropriate in our case),
or create totally new, but remember, web server will only read files
referenced by symlinks in /etc/apache2/sites-enabled/ dir,
be sure the symlink points to the right file. Actually, there are
scripts a2ensite
and a2dissite
to handle this
situation.
There are many ways to set options in this file, see, for example, /etc/apache2/sites-available/default (if you take it, replace names of files, certificates, etc). This config allows access through the standard tcp/80 port and tcp/443 port (SSL).
Also, we have to set some basic rules in
/etc/apache2/conf.d/security:
<Directory> AllowOverride None Order Deny,Allow Deny from all </Directory> ServerTokens Prod ServerSignature Off
Restart Apache and test web server access from the net:
/etc/init.d/apache2 restart
Check/change PHP config /etc/php5/apache2/php.ini, pay attention to the following params:
display_errors = off upload_max_filesize = 20M date.timezone = Europe/Rome
Enable default site (a2ensite
creates symlinks, but it would
make no harm if all those symlinks were created manually):
a2ensite default
Once again restart Apache.
MySQL database
It looks like nowdays everybody wants to use Postfix with a database (usually MySQL). It really provides some useful features.
The name of the database and the name of the mail administrator in the database are quite random, let it be mailserver and postman. Use a reasonably long password (8+).
The following description assumes the use of MySQL
client mysql
(see MySQL usage notes):
mysql -u root -p
If root login is disabled (e.g., due to empty root password, like in Ubuntu), and you cannot connect to MySQL server, see Setting MySQL root password in Ubuntu. When you're normally connected:
mysql> CREATE DATABASE mailserver;
mysql> USE mailserver;
mysql> GRANT ALL ON mailserver.* TO 'postman'@'localhost' IDENTIFIED BY 'gdp501wste';
mysql> exit
Now it's time to create database objects. Let's make it conveniently,
using a carefully prepared SQL script
executed by mysql
. Move this script to an appropriate dir
and run:
mysql mailserver -u postman -p < mailserver.sql
Configuring Postfix
There are some files to be created to provide interaction between Postfix and the database. They look alike, except SQL query string. Be sure it is consistent with your database struct (table/column names, etc).
Virtual domain config file:
/etc/postfix/mysql-virtual-domains.cf
user = postman password = gdp501wste hosts = 127.0.0.1 dbname = mailserver query = SELECT id FROM virtual_domains WHERE name='%s'
Virtual forwarding config file:
/etc/postfix/mysql-virtual-alias-maps.cf
user = postman password = gdp501wste hosts = 127.0.0.1 dbname = mailserver query = SELECT destination FROM virtual_aliases WHERE source='%s'
Virtual mailbox config file:
/etc/postfix/mysql-virtual-mailbox-maps.cf
user = postman password = gdp501wste hosts = 127.0.0.1 dbname = mailserver query = SELECT email FROM users1 WHERE email='%s'
Virtual e-mail mapping file:
/etc/postfix/mysql-virtual-email2email.cf
user = postman password = gdp501wste hosts = 127.0.0.1 dbname = mailserver query = SELECT email FROM users1 WHERE email='%s'
Make sure that all these config files have group postfix and access mode 640 (it's important, because those files contain plaintext pass).
Also, a special OS user is required to handle our mail (postman is a database user, not related to OS). On a system that auto creates a unique group for each new user (Fedora, Ubuntu):
useradd -m vmail
or, if keeping mail in /home is not desired,
useradd -m -d /var/vmail vmail
Find UID and GID in /etc/passwd. Alternative approach (with total control of names and numbers):
groupadd -g 5000 vmail
useradd vmail -u 5000 -g vmail -m -d /home/vmail
The following cmds are required to complete the Postfix config:
postconf -e virtual_mailbox_domains=mysql:/etc/postfix/mysql-virtual-domains.cf
postconf -e virtual_mailbox_maps=mysql:/etc/postfix/mysql-virtual-mailbox-maps.cf
postconf -e virtual_alias_maps=mysql:/etc/postfix/mysql-virtual-alias-maps.cf, mysql:/etc/postfix/mysql-virtual-email2email.cf
postconf -e 'virtual_uid_maps = static:5000'
postconf -e 'virtual_gid_maps = static:5000'
The following example shows how to insert a new domain into the database (NOTE! example.com is used for demonstration, in fact, this table must keep valid domain names only):
mysql mailserver -u postman -p
...
mysql> INSERT INTO virtual_domains VALUES (1, 'example.com');
Let's check that Postfix can execute the query (if everything is O.K., the following cmd must output 1):
postmap -q example.com mysql:/etc/postfix/mysql-virtual-domains.cf
Now you can create new mail users in following manner:
mysql> INSERT INTO virtual_users (domain_id, user, password) VALUES (1, 'pro07', MD5('stp4e39mix'));
Here's a database cmd to add a forwarding record:
mysql> INSERT INTO virtual_aliases (domain_id, source, destination) VALUES (1, 'postmaster', 'alex@example.com');
Testing cmds:
postmap -q postmaster@example.com mysql:/etc/postfix/mysql-virtual-alias-maps.cf
postmap -q info@example.com mysql:/etc/postfix/mysql-email2email.cf
Each of the above cmds should return the queried mail address if it exists in the database, otherwise an empty string is returned.
See also the final version of main.cf as it
looks after dovecot
config is done.
Configuring Dovecot
Dovecot provides POP3 and IMAP services. To establish relation between Postfix and Dovecot, we must add the following line to
/etc/postfix/master.cf
dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -d ${recipient}
(all that stuff is a single line!)
Next step is to modify Postfix config:
postconf -e virtual_transport=dovecot
postconf -e dovecot_destination_recipient_limit=1
To activate changes, Postfix must be restarted:
postfix reload
Warning!
There are some differences between Dovecot 1.* and Dovecot 2.* config. The following description assumes Dovecot 2.*.
The default Dovecot config file
/etc/dovecot/dovecot.conf
is the assembly of the multiple include files, and you have to choose: to edit all those files or to create a single clean config file with all settings in one place. Here's an example of a single config file dovecot.conf.
There is also dovecot-sql.conf for database connectivity:
driver = mysql connect = host=127.0.0.1 dbname=mailserver user=postman password=gdp501wste default_pass_scheme = PLAIN-MD5 password_query = SELECT user, domain, password FROM users2 WHERE user='%n' AND domain='%d'; user_query = SELECT user, password, 1001 AS uid, 1001 AS gid, '/var/vmail/%d/%n' AS home, 'maildir:/var/vmail/%d/%n' AS mail FROM users2 WHERE user='%n' AND domain='%d';
Note that substitution params mean:
'%u' - entire userid (like user@domain);
'%n' - user part of user@domain;
'%d' - domain part of user@domain;
To see actual Dovecot config:
doveconf -a | more
Or you can select non-default settings only:
doveconf -n | more
Starting/stopping Dovecot (Ubuntu Linux 10.*/11.*):
sudo start dovecot
...
sudo stop dovecot
The Dovecot log is (according to the above dovecot.conf):
/var/vmail/dovecot-deliver.log
Both Postfix and Dovecot use certificates to restrict access and reject bad clients (those without mail certificates). In our case these certificates are not the same that are used by Apache web-server. In fact, certificates can be inherited from an old mail server, if you're upgrading mail system; or created as described above. Anyway, it requires some config lines in Postfix main.cf and dovecot.conf, and these lines are usually added/modified in the end of the installation process when mail service is already functional, i.e., can send and receive mail.
postconf -e smtpd_sasl_type=dovecot
postconf -e smtpd_sasl_path=private/auth
postconf -e smtpd_sasl_auth_enable=yes
postconf -e smtpd_tls_cert_file=/etc/ssl/certs/servcert.pem
postconf -e smtpd_tls_key_file=/etc/ssl/private/servkey.pem
postconf -e smtpd_use_tls=yes
postconf -e smtpd_tls_auth_only=no
Testing SMTP
TEST # 1 (connecting SMTP locally)
At the server:
telnet localhost 25
It should respond with:
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
220 mx1.example.com ESMTP Postfix (Ubuntu)
At this point enter:
ehlo localhost
It should output something like this:
250-mx1.example.com
250-PIPELINING
250-SIZE 10240000
250-VRFY
250-ETRN
250-STARTTLS
250-AUTH PLAIN LOGIN
250-ENHANCEDSTATUSCODES
250-8BITMIME
250 DSN
Now you can send a test message:
mail from:<pro@example.com>
250 2.1.0 Ok
rcpt to:<postmaster@example.com>
250 2.1.5 Ok
data
354 End data with <CR><LF>.<CR><LF>
Test 1
.
250 2.0.0 Ok: queued as E2604200068
That's enough for a simple test:
quit
221 2.0.0 Bye
Connection closed by foreign host.
TEST # 2 (connecting SMTP from a LAN host)
If this test fails, first try
ping
(to be sure that SMTP server is accessible) and
nmap
(to be sure that firewall is set right).
At the client computer/workstation:
telnet 192.168.0.2 25
It should respond with:
Trying 192.168.0.2...
Connected to 192.168.0.2.
Escape character is '^]'.
220 mx1.example.com ESMTP Postfix (Ubuntu)
You should enter the following cmd with your host's IP (or name):
ehlo 192.168.0.25
The rest is like in the previous example.
TEST # 3 (connecting SMTP from a LAN host using TLS)
At the client computer/workstation:
openssl s_client -connect 192.168.0.2:25
-starttls smtp
In case of success server returns a big block of misc info that ends with:
...
Compression: 1 (zlib compression)
Start Time: 1331709237
Timeout : 300 (sec)
Verify return code: 18 (self signed certificate)
---
250 DSN
At this point you can send:
ehlo 201.37.89.208
and the rest goes like in the previous examples.
TEST # 4
If previous tests are successful, try to send a real mail message to a real external mail address (e.g., your own gmail.com acount).
Remember that a real mail server must have an appropriate PTR record. If you do not belong to a big company with PI (provider independent) address space, than only your ISP can do this for you.
Testing POP3/IMAP
TEST # 5 (testing Dovecot/pop3 from a workstation)
telnet 192.168.0.2 pop3
Server should respond with:
Trying 192.168.0.2...
Connected to 192.168.0.2.
Escape character is '^]'.
+OK Dovecot Ready.
Send your name:
user pro7
+OK
Send your password:
pass stp4e39mix
+OK Logged in.
Try some commands:
list
+OK 0 messages:
.
quit
+OK Logging out.
Connection closed by foreign host.
TEST # 6 (testing Dovecot/imap from a workstation)
telnet 192.168.0.2 imap2
Server should respond with:
Trying 192.168.0.2...
Connected to 192.168.0.2.
Escape character is '^]'.
* OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE STARTTLS AUTH=PLAIN AUTH=LOGIN] Dovecot ready.
Send your name and password:
1 login pro07 stp4e39mix
1 OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS] Logged in
Try some commands:
2 list "" "*"
* LIST (\HasChildren) "." "INBOX"
2 OK List completed.
3 select "INBOX"
* FLAGS (\Answered \Flagged \Deleted \Seen \Draft)
* OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted.
* 0 EXISTS
* 0 RECENT
* OK [UIDVALIDITY 1332158528] UIDs valid
* OK [UIDNEXT 1] Predicted next UID
* OK [HIGHESTMODSEQ 1] Highest
3 OK [READ-WRITE] Select completed.
6 logout
* BYE Logging out
6 OK Logout completed.
Connection closed by foreign host.
Now you can try to access mail boxes using a mail client program.
Web mail (roundcube)
Assuming that Apache is configured according to ... (see Apache config), download the Roundcube archive, unpack, copy to /var/www/webmail, read INSTALL.
Create a database named roundcubemail, grant privileges to rouncube@localhost (Note! The password below is just for example, you should provide your own):
mysql> CREATE DATABASE roundcubemail;
mysql> GRANT ALL PRIVILEGES ON rouncubemail.* TO 'roundcube'@'localhost' IDENTIFIED BY 'wsr8nvji32';
mysql> exit
From dir /var/www/webmail exec following cmd to create the database objects (tables):
mysql roundcubemail -u roundcube -p
< SQL/mysql.initial.sql
On success, open in browser http://srvname/installer and follow instructions. When finished, you can edit new config files manually (in my case the database password was not written to a config file, so I had to edit db.inc.php).
Also, check /var/www/webmail/.htaccess, there is at least one line that you may want to change:
php_value upload_max_filesize 5M
Final notes
While setting Dovecot, put in dovecot.conf:
mail_debug = yes
auth_debug = yes
With these options you can see in /var/vmail/dovecot-deliver.log what is wrong, and how all those database queries look like at runtime. I had to change password_query and user_query in /etc/dovecot/dovecot-sql.conf many times before it started to work right.
mail_location = maildir:/var/vmail/%d/%n
does not work in my case (domain part of the path mystically disappears), but everything is O.K. if user_query returns the same maildir. Also, it's useful to keep in mind the difference between home dir and maildir (virtual users have no home dir in classical Unix/Linux sense).