Tutorial - How to create your own E-mail Host

Creating your own email hosting service offers unparalleled control and customization over your communication platform, allowing you to tailor security, privacy, and functionality to your exact specifications. This approach is particularly appealing for those who value data sovereignty, wish to avoid third-party dependencies, or seek to integrate email services deeply with their existing IT infrastructure.

This guide shows a secure, deliverable, and maintainable way to run an email server. It assumes you have basic Linux and DNS knowledge. Keep each step small and test often. Running your own email host requires careful DNS, authentication, and security setup. This tutorial covers recommended components, configuration examples, testing tips, and maintenance.

Before you start

Choose a reliable VPS or dedicated host. Verify port 25 policies first.
Prefer a static IP address with good reputation.
Register a domain you control and can edit DNS records.

High-level architecture

MTA (Mail Transfer Agent) — Postfix routes mail.
MDA/IMAP/POP3 — Dovecot stores and serves mail (Maildir recommended).
DKIM signer — OpenDKIM signs outgoing messages.
Spam/AV — SpamAssassin + Amavis + ClamAV for filtering.
TLS — Let’s Encrypt certificates for encryption.
Monitoring & backups — logs, DMARC reports, and Maildir backups.

Step 1 — DNS records (concrete examples)

Replace example.com and 203.0.113.12 with your domain and IP.

# A record for mail host
mail.example.com.    A    203.0.113.12

# MX record (priority 10)
example.com.         MX   10 mail.example.com.

# SPF
example.com.         TXT  "v=spf1 a mx ip4:203.0.113.12 -all"

# DKIM (selector 'default')
default._domainkey.example.com. TXT "v=DKIM1; k=rsa; p=PUBLICKEYHERE"

# DMARC
_dmarc.example.com.  TXT  "v=DMARC1; p=none; rua=mailto:postmaster@example.com; ruf=mailto:postmaster@example.com; pct=100"

Note on PTR (reverse DNS)
Ask your VPS provider to set PTR for your mail IP. It must resolve to mail.example.com.
Many receivers check PTR and penalize mismatches.

Step 2 — Choose software and initial server setup

OS recommendation: Ubuntu LTS or Debian.
Install basic packages and enable automatic security updates.

Example commands (Ubuntu/Debian):

sudo apt update && sudo apt upgrade -y
sudo apt install -y postfix dovecot-core dovecot-imapd dovecot-pop3d \
  opendkim opendkim-tools spamassassin amavisd-new clamav-freshclam fail2ban certbot

During Postfix install, choose Internet Site and set mail.example.com.

Step 3 — Postfix basics and security

Edit /etc/postfix/main.cf and include these recommended snippets.

myhostname = mail.example.com
mydomain = example.com
myorigin = $mydomain
inet_interfaces = all
mydestination = $myhostname, localhost.$mydomain, localhost, $mydomain
home_mailbox = Maildir/

# TLS
smtpd_tls_cert_file=/etc/letsencrypt/live/mail.example.com/fullchain.pem
smtpd_tls_key_file=/etc/letsencrypt/live/mail.example.com/privkey.pem
smtpd_use_tls=yes
smtpd_tls_security_level=may

# Submission and auth
smtpd_sasl_type = dovecot
smtpd_sasl_path = private/auth
smtpd_sasl_auth_enable = yes
smtpd_tls_auth_only = yes
smtpd_sasl_security_options = noanonymous

# Restrict relaying
smtpd_recipient_restrictions =
  permit_sasl_authenticated,
  permit_mynetworks,
  reject_unauth_destination

# Ports
# - SMTP (MTA-to-MTA): 25
# - Submission (authenticated): 587
# - SMTPS (implicit TLS): 465 (optional)

Important: Ensure port 25 is open and reachable.
If your provider blocks it, use a relay or request removal of the block.

Step 4 — Dovecot and virtual users

Use Maildir for reliability. Configure Dovecot to provide IMAP/POP3 and SASL for Postfix.

Edit /etc/dovecot/conf.d/10-mail.conf:

mail_location = maildir:~/Maildir

Use Dovecot as SASL for Postfix by enabling the auth socket. In Dovecot config set:

service auth {
  unix_listener /var/spool/postfix/private/auth {
    mode = 0660
    user = postfix
    group = postfix
  }
}

Virtual users:
For multiple domains use virtual users. Options:

passwd-file (simple)
SQL backend (MySQL/MariaDB/Postgres)
LDAP (enterprise)

Virtual users avoid creating system accounts for every email user.

Step 5 — DKIM (OpenDKIM)

Generate keys for selector default.
Add the public key to DNS as default._domainkey.example.com.
Configure OpenDKIM and integrate with Postfix using milter socket.

Example Postfix milter settings in main.cf:

smtpd_milters = unix:/opendkim/opendkim.sock
non_smtpd_milters = $smtpd_milters
milter_default_action = accept

OpenDKIM signs outgoing mail so recipients can verify the message integrity.

Step 6 — SPF and DMARC best practices

Start DMARC with p=none to collect reports safely.
Review aggregate reports (rua) and forensic reports (ruf) before enforcing.
Example SPF: v=spf1 a mx ip4:203.0.113.12 -all.
If you send via third parties, add include: statements for them.

Step 7 — Spam and antivirus

Recommended stack:

Amavis to connect Postfix to filtering.
ClamAV for antivirus scanning.
SpamAssassin for spam scoring.

Tuning tips:

Use spamassassin --lint and tune local.cf rules.
Use RBLs judiciously; false positives can block legitimate mail.
Consider greylisting only if your user base tolerates delayed delivery.

Step 8 — TLS certificates and automation

Use Certbot for Let’s Encrypt certs and automate renewals.

Example command to obtain certs:

sudo certbot certonly --standalone -d mail.example.com

Add a deploy hook to reload Postfix and Dovecot after renewal:

--deploy-hook "systemctl reload postfix dovecot"

Set smtpd_tls_security_level = may and require TLS for authentication.

Step 9 — Fail2ban and other hardening

Protect SMTP auth and Dovecot login with fail2ban.
Harden SSH and disable root login.
Keep software updated with unattended-upgrades.

Fail2ban example jail for postfix-auth:

[postfix-sasl]
enabled = true
port = smtp,ssmtp,submission
filter = postfix
logpath = /var/log/mail.log
maxretry = 5

Step 10 — Backups and replication

Backup Maildirs with rsync or block-level backups.
For incremental encrypted backups use Borg or restic.
Test restores monthly.
Dovecot dsync helps replicate mailboxes between servers.

Step 11 — Monitoring and deliverability checks

Monitor logs: /var/log/mail.log and /var/log/mail.err.
Check mail queue: postqueue -p and postfix flush to retry.
Use online tools: mail-tester.com, MXToolbox, and DMARC analyzers.
Subscribe to DMARC aggregate reports and review weekly.

Useful commands:

postqueue -p
postfix flush
postsuper -d ALL  # careful: deletes all queued mail
tail -f /var/log/mail.log
openssl s_client -connect mail.example.com:587 -starttls smtp

Step 12 — Testing and debugging

Use swaks to test SMTP submission and authentication.
Send test messages to Gmail, Outlook, and Yahoo. Check spam folders.
Inspect message headers for SPF/DKIM/DMARC results.

Alternatives and automation

If you want a faster deployment consider:

Mailcow (docker-based, full stack)
iRedMail or Modoboa (installer stacks)

If deliverability is critical, use a trusted relay:

Mailgun, SendGrid, Amazon SES, or SparkPost.

Sample Postfix + Dovecot checklist (quick)

PTR record set and matches host name
A, MX, SPF, DKIM, DMARC records published
Postfix: require SASL on submission (587)
Dovecot: Maildir and auth socket configured
OpenDKIM signing outgoing mail
SpamAssassin + Amavis + ClamAV configured
Let’s Encrypt TLS with auto-renew hooks
fail2ban protecting auth endpoints
Backups scheduled and tested
DMARC reports monitored

Short maintenance schedule

Daily: check mail queue and critical logs.
Weekly: review DMARC reports and server updates.
Monthly: test restore from backup and review spam filtering.
Quarterly: check IP reputation and RBL status.

Like my content? Support me with a tip!

Tutorial – How to create your own E-mail Host