Creating an email (SMTP) relay server in Windows 2008 and 2012

The Why

There’s several reasons you might need to create a mail relay on your Windows server. The most common is that you might have network devices that wish to send emails that do not support the encryption and authentication protocols required by your email provider. Google Apps, for example, require that any device sending through the smtp.gmail.com server supports either SSL or TLS. Some older devices don’t, and it seems a little excessive to replace a multi-thousand-pound device for something so trivial. For sending to internal users on Google Mail it’s possible to simply use the aspmx.l.google.com MX server but this isn’t a terribly scalable solution and won’t work in a lot of cases.

The What

The simple solution, as the title hints, is to set up a mail relay server. This will be configured to accept email from inside your network, either anonymously or with basic authentication, over an unencrypted connection and to then send the email onwards via your mail provider’s server, connecting using whichever protocols they require.

SMTP

The H0w

The first step is to install the SMTP server service. IIS 7 improves over IIS 6 in many ways, however it does not include any form of SMTP service (FTP was also not offered at release but was shipped with 7.5 and as an out-of-band update for 7.0). This means we need to use the IIS 6 SMTP service. This can be installed from Server Manager’s Add Roles and Features; it’s listed as “SMTP Server” in the “Features” section. Selecting it for install will trigger a popup to prompt for dependency install:
install-smtp-service

It can also be installed using the PowerShell Add-Windowsfeature cmdlet, deprecated in 2012 and replaced by the almost-identical Install-Windowsfeature cmdled:

Add-Windowsfeature SMTP-Server

Notice one of the dependencies installed is “IIS 6 Management Tools”. This should give a hint as to how we’ll be managing and configuring this. Open up the “Internet Inforamtion Services (IIS) 6.0 Management” administrative tool (%windir%\system32\inetsrv\InetMgr6.exe):
IIS-6-managerExpand the server and and right-click –> Properties on the “SMTP Virtual Server #1” to open the properties window for the SMTP server – this window will be familiar to anyone who’s worked with SMTP in IIS 6:

SMTP-Virtual-server-properties

I won’t go into too much detail except for the relevant settings; most of this property dialog is described in some detail elsewhere. The sections important to us are:

  • Access. This tab defines how our devices will be allowed to connect to this server and what restrictions will be applied to protocols and content allowed
  • Delivery: This tab defines how the server sends messages onwards; this is where we’ll configure Google’s SMTP server and the authentication required for it.

Access

There’s 2 obvious ways to restrict who’s allowed to send email through this server. The first is by IP address – if we only allow access to the SMTP server from the IP address assigned to the scanner then no-one else should be able to use it (unless they specifically configure their device to use the address in question), or by user – we  can create a user account for the scanner and configure it to authenticate against the SMTP Relay using basic authentication. I’m going to opt for the latter as it’s a more flexible solution; it will work for additional devices without reconfiguration, it allows me to use dynamic IPs for my scanners and and it’s not vulnerable to clients simply changing their IP address.

First we’ll need to create a user in Active Directory (Or Local Users and Groups if a domain isn’t being used). I’ll leave this as an exercise for the reader. Then we’ll need to go to the “Access” tab on the SMTP Server Properties and click “Authentication”. We’ll disable Anonymous access and enable “Basic Authentication”. Windows will warn that this will result in passwords being sent over the network in plain text – this is unavoidable in this context and to be frank if there’s rogue users running protocol analyzers on your network I’d leave the scanner setup for another day…

basic-authentication

 

While we’re here, click on “Relay” and check that the box labeled “Allow all computers which successfully authenticate to relay…” is ticked.

 

This has configured the server to allow basic authentication for SMTP connections and to relay mail for any authenticated connection but hasn’t actually granted access to our user. This can be done on the Security tab – I imagine to have got this far you’re familiar with that process.

add-user

Delivery

On the “Delivery” tab, there’s 3 buttons at the bottom – we’ll need to configure settings in all 3 of these.

delivery-options

I’d start, counter-intuitively, with the “Advanced” button:
advanced-deliveryHere we’ll need to enter into the “Smart host” box the FQDN of the server we want to send through – in this example Google’s smtp.gmail.com server, but this could equally be and Office 365 or ISP-provided email server. Leave everything else on it’s default settings (although for completeness you may wish to change the “Fully-qualified domain name” to match the reverse DNS entry for the IP address the connections will go out through), and make sure the “Attempt direct delivery….” box isn’t checked.

Next, the Outbound Security window. We’ll need to enter here details of an account that is authorized to send through the server configured on the “Advanced Delivery” page – I’d recommend setting up an account specifically for this purpose with your email provider. You’ll also need to tick the “TLS encryption” checkbox, or we’ll be trying to authenticate in-the-plain against the provider’s server putting us right back to square one!

outbound-security

 

Almost done. Chances are your mail provider’s server isn’t listening for TLS connections on port 25. In the case of the smtp.gmail.com server we’re using, TLS is expected to use port 587. We can configure IIS to send to this port on the “Outbound Connections” page – simply replace 25 with the port required by your provider:

outbound-connections

 

Conclusion

That should be you ready to go. We’ve created a new SMTP virtual server and configured it to:

  • Listen for connections on an internal IP address
  • Accept authentication provided in basic (plain-text) format by the scanner/device
  • Accept emails once the device is authenticated
  • Connect to the mail provider’s SMTP servers using TLS encryption
  • Forward the emails to the mail provider’s server for onward delivery.

If you can’t get this working, please leave a comment below and we’ll see if we can get you working!

6 Comments

  1. Olav

    Can I use one smtp server for different applications or do I need to create an smtp server for every application?

    • Admin

      You can use the same one for multiple applications. They can either share login details or you can create a Windows account for each sender.
      Jim

  2. Nathan

    I’m using this for alerts that come out of my cctv box. If I’m using ip authentication what do I need to put in the Smtp server and port on my cctv box?

    • Admin

      You’ll need to use port 25 and your server’s IP address.

  3. Dan

    I am trying to move some managements to an isolated network and then dual homing a windows server as a relay server. Is there any special configurations I should consider?

  4. After a reboot the SMTP-server was down again.
    I had to start the SMTP-server manually again.
    Is it possible to configure the SMTP-server on such a way that the SMTP-server starts automatically after a reboot?

Leave a Comment

*