• United States



REVIEW: MailScanner and ScrolloutF1 are standouts in open source email security

Jan 11, 201617 mins
Open SourceSecurity

Email security is of paramount concern in any organization. A significant percentage of malware is delivered via email, on the premise that an unsuspecting user will open the message, allowing the malware payload onto the user’s machine. From there, malware can worm its way into the network and wreak various kinds of havoc, often undetected, sometimes for months or even years.

It should then come as no surprise that a significant industry has grown up around the serious business of containing email threats. We decided to review four open source products to see if they could deliver enterprise-grade security. The four products were CipherMail, MailScanner Scrollout F1 and hMailServer.

Each product takes a unique approach to email security. CipherMail has the narrowest focus as it performs only one main task, email encryption, with a small dose of data loss prevention (DLP). It acts as a gateway that uses four different encryption standards to encrypt/decrypt both incoming and outgoing mail.

At the other end of the spectrum is hMailServer, a fully-featured email server that supports all common email protocols and contains built-in support for virus and spam protection. In between, there is Scrollout F1 and MailScanner.

Scrollout F1 is, in the vendor’s own words, “designed for those without advanced email experience”. It is mainly an anti-spam email gateway designed to fit in with an existing email infrastructure, but it also has anti-virus and encryption capabilities.

MailScanner provides anti-spam, anti-virus and other email security tasks. It is a mature product that has been around since 2001 and of the four products we tested provides the largest numbers of email security features.

When testing products that don’t line up feature-for-feature, our focus is on evaluating how well each product delivers on its stated goals. We are always looking at usability and features, plus overall performance. Early on in the testing we identified MailScanner and Scrollout F1 as standouts, mostly because they provide a more complete set of email security features and configuration options.

MailScanner is probably the better solution for those in a more enterprise like environment as it can be installed on a broader selection of platforms and it has a bit more firepower than Scrollout F1. However, Scrollout F1 is an excellent choice for those seeking a solution with comparable features to MailScanner, but who may prefer a streamlined approach. We found Scrollout F1 to be one of the easiest products to get up and running, whereas MailScanner was a bit more temperamental. Both provide excellent online user manuals, with the upper hand going to MailScanner as it provides a detailed explanation of each of the almost 350 configuration options.

Our runners up are hMailServer and CipherMail, two very capable products in their own right, but lacking a bit compared to the other two. CipherMail’s biggest strength and weakness are the one and the same in our minds, it only provides email encryption. However, encryption can be a very important piece of the email security chain and CipherMail does a good job at it without adding too much overhead. It is easy to install, has a modern intuitive interface and great documentation.

+ ALSO ON NETWORK WORLD Best practices for email security +

HMailServer is the only Windows product in our test. It is essentially a full-fledged mail server with multiple built-in email security functions, such as spam and virus checking. Although not designed to provide email security as its primary focus, by carefully configuring the options, you can get pretty decent security going on this product.

Email security pros and cons

 Scrollout F1CipherMailMailScannerhMailServer
ProsEasy to use, preconfigured with settings that makes it work ‘out of the box’, IPv6 support.Modern and easy to use Web GUI, solid documentation, granular control of settingsIntegrates with multiple other open sources, flexible configuration, very good documentationEasy to install and configure, good documentation, Windows capable (for those on Windows platforms)
ConsRuns only on Debian, user interface is hard to read and navigate on most devicesOnly performs email encryptionInitial install and configuration can be a bit cumbersomeLacks a native Windows-based Web interface

Here are the individual reviews (watch screen shots of each product):

CipherMail email encryption gateway

The community version of CipherMail is open source software that can be installed on most versions of Linux using a distribution package or as a virtual appliance for VMware or Microsoft Hyper-V. It runs on Java and requires Postfix as well as Tomcat for the Web GUI. Unlike some of the other products we reviewed, CipherMail focuses on a single function of the email process, namely encryption.

At the core of CipherMail is the use of certificates to keys to encrypt/decrypt/sign email messages. It has a built-in Certificate Authority (CA) that can be used to issue X.509 certificates for both internal and external users and the gateway can also utilize an external CA server. Certificates and keys can be imported, stored and managed from the certificate store with no limits on the number of certificates and keys. A specific valid certificate can be configured for use on the global, domain or user level. This means user A can use one certificate and user B a different one.

CipherMail can be managed and configured from a Web GUI or using a rudimentary system configuration tool from the console. We’re not entirely sure why, but there are certain items that can only be configured from the console and others only from the Web GUI, and there are items that can be configured from both. The Web user interface has a modern look and is easy to navigate, but performance was a bit sluggish, even on our test server with not much traffic and ample resources.

CipherMail provides administrators with quite a bit of granular control, from general server settings down to how mail is handled for individual user accounts. Thankfully you don’t have to configure every user individually, as the settings flow down starting with factory settings, then global preferences followed by domain preferences and then finally the user account. For example, certain accounts can be required to encrypt all outgoing messages whereas other accounts do not need to encrypt messages.

CipherMail encrypts emails using one or a combination of four standards: S/MIME, OpenPGP, TLS and PDF encrypted email. It can encrypt/decrypt both incoming and outgoing emails by processing emails before or after they go through the mail server. CipherMail functions as a SMTP server and uses Postfix as the mail transfer agent (MTA). It can integrate with existing email handling infrastructure in several ways, with one being direct delivery where it resides between the internal mail server and the Internet. Another common configuration would be to place it between the mail server and a SMTP relay or between a virus scanner (and mail server) and the Internet.

Encrypting large messages requires a lot of memory and CipherMail by default allocates 60% of available memory for dynamic memory allocation (heap), something to keep in mind if the server is not dedicated to email. By allowing size limits on encrypted emails, CipherMail ensures that the server does not get bogged down trying to sign and encrypt very large emails. In addition to encrypting the email message itself, CipherMail, when possible, will also encrypt attachments. It provides a variety of settings especially focused on PDF attachments since this format is often used for document exchange via email.

CipherMail also includes a webmail component that allows for secure access to encrypted emails via a browser. There is also a CipherMail application available for Android devices. In addition it provides some data loss prevention (DLP) features in that emails can be scanned for keywords or certain patterns, such as credit card or account numbers, and quarantine messages containing such information.

In our low volume testing environment we found that CipherMail added very little delay in email processing, but this could be a different story in larger installations. As mentioned, encrypting especially larger email messages can be resource-intensive and for organizations with a large volume of email with attachments this may be an issue. Although CipherMail provides a fairly narrow function in the email chain, it is one that can be very important, especially in environments where sensitive information needs to be protected and/or there are regulatory compliance concerns. There is a bit of a learning curve to become familiar with all of the detailed settings, but CipherMail provides excellent documentation that makes this task easier.

Scrollout F1 Anti-Spam Email Gateway

Scrollout F1 is an open source email gateway that installs on Debian Linux and integrates with existing mail servers such as Lotus Domino, Postfix and Microsoft Exchange. Scrollout F1 can be installed with a few commands from the terminal or by using a pre-configured ISO provided by the vendor.

Scrollout F1 can be configured and managed from a Web GUI. We found the initial configuration to be straightforward, requiring just a host name, IP, port and gateway information. The home page provides basic information about the server and utilization metrics for CPU load, memory, users and network.

Scrollout F1 is primarily a gateway for incoming emails, which means all emails go through the gateway before being sent to the actual mail server where emails are delivered to the end user. It can be configured to manage email for multiple domains and if desired, can be used for outgoing as well as incoming mail. Except for the few installation parameters mentioned above, the software comes pre-configured with settings that are likely to be adequate for most environments.

Like many other email security applications, Scrollout F1 uses a scoring system to determine if a message is spam. Administrators have granular control over this and other settings from the Security tab where 15 categories are available to configure, ranging from host name and URL filters to SpamAssassin and rate limits. Each category has a setting from 1 to 10, with 1 being the most restrictive; the settings are also color coded from dark green (1) to dark red (10). For each category you can read details by clicking the information link that displays the basic information for that category. Email that is deemed spam can either be quarantined or it can be deleted based on the score. Emails containing viruses can be handled the same way.

In addition to spam and virus checking, Scrollout F1 provides black and white list options, emails or domains on these lists are either blocked or accepted without any additional checking. One feature we particularly liked was the ability to manage emails by country. Incoming emails can be scored based on three settings for each country, ‘business area’, ‘foreign area’ and ‘out of area’. Countries in the ‘business’ category are allowed through without adding a spam score, the ‘foreign’ ones are given a spam score based on location and the ‘out of area’ emails are attempted blocked, using Geo methods on both the connection level and the mail processing level. Scrollout also includes real time blocklist (RBL) and integrates with Pyzor, a collaborative, networked system designed to detect and block spam using digests of messages.

In addition to the basic server information displayed on the home page, there is a log and statistics presented in graph form. The graphs display the overall volume of email traffic by period such as day or month, with information about how many messages were spam or contained viruses. Integrating with an existing email infrastructure is as easy as just adding the IP address of the mail servers that handle email for each domain.

Scrollout F1 provides several videos on its website that demonstrate how to perform various configuration tasks. For most users this will be more than adequate as the software is easy to use on its own. Paid support is available, and there is a forum where information about more complex topics can be found. The vendor also offers inbound email filtering as a cloud service starting at $15 per month for up to two domains.

Overall we liked Scrollout F1 for its ease of installation and use. However, we found the Web interface a bit hard to read with a white/light gray font on a blackish background. In our opinion the interface could also benefit from a slight facelift to make it look more professional.


HMailServer is a free open source email server that has some compelling security features and is also one of very few open source mail server products that runs natively on Windows.

As with any mail processing software, the system requirements depend on the number of email accounts and volume of email, but hMailServer offers a relatively small footprint, and claims to use less than 100MB RAM for most installations. Since IMAP is more resource intensive than POP, hMailServer recommends using POP for larger installations. It uses a built-in light version of MS SQL Server for storage, but alternatively can be integrated with most versions of SQL Server, MySQL and PostgreSQL.

+ ALSO ON NETWORK WORLD Review: 6 free email servers for small business +

We installed hMailServer on a Windows server using an EXE file and the initial installation was straightforward, with just a few configuration prompts, such as selecting an installation folder and a password. Unlike some of the other reviewed products, hMailServer is managed from a Windows program that runs from the desktop. The management console is somewhat austere and dated, but easy to navigate and with help functionality throughout that explains the various features. Some features can be managed using PHPWebAdmin to accommodate Apache environments.

HMailServer has several built-in spam protections and attempts to determine as early as possible in the process whether a message is spam or not, thereby conserving server resources. It uses a scoring system, where each spam checking method adds a value to an accumulating score that is used to determine how a message is handled.

In addition to the internal checks, which include HELO hosts, SPF and DKIM checks, it also can utilize DNS blacklists from Spamhaus and Spamcop, along with external SURBL checking. It also has built-in support for SpamAssasin. If a message is determined to be spam, it can either be delivered to the recipient with an added spam header or the message can be deleted, depending on the score settings.

In addition to anti-spam features, hMailServer provides grey-listing. This service essentially rejects all mail on the first delivery attempt, taking advantage of the fact that legitimate mail servers will attempt to redeliver whereas spammers often will give up after just one attempt. The downside is that this will cause delays in mail delivery, but there are ways around that by bypassing grey-listing for senders that pass an MX or SPF check. With hMailServer it is also possible to use SSL to encrypt communication between the server and clients as well as connections to other mail servers. It should be noted that this feature encrypts the communication such as handshake and password transmission, not the actual message itself, so it can’t be compared to a product such as CipherMail, which encrypts the entire message.

It is possible to white-list specific email or IP addresses to bypass spam checking for senders that you are comfortable will not be sending spam. This reduces the load on the mail server and may make sense in environments where a large number of emails are received from known sources.

As for anti-virus features, hMailServer comes with built-in support for ClamAV, a common open source anti-virus software. In order to use ClamAV, it needs to be installed as a separate server and you provide the address and the port on the anti-virus configuration tab inside hMailServer. It can also use other third-party anti-virus products by calling an executable from the aforementioned anti-virus tab. With the native anti-virus feature, various types of attachments can also be rejected and there is a starter list of common file types that should be rejected (.exe, .bat etc.). One small minus is that you can only use one external virus scanner at a time (in addition to ClamAV), but this should not be a showstopper for most installations.

Another powerful feature of hMailServer is the ability to create VB scripts to perform certain tasks. For instance, you can look for certain terms inside an email and reject that message as spam based on various terms. The script feature can also be used to various custom messages to the log.

One potential downside is that you need to install a full-fledged mail server in order to take advantage of the built-in in email security features, but this can also be seen as a plus to some. We would like to see a native Web admin module in order to prevent the need to use a remote desktop solution to manage the server. However, we found hMailServer to be a good fit for those who need a Windows-based solution.

MailScanner Email

MailScanner is open source software released under GPL that runs on most flavors of Linux and integrates with Linux-based email gateways. Installation files are provided as downloadable .tar files and the installation is completed through a shell script that walks you through the installation with several prompts. We installed MailScanner on an Ubuntu server, but this was not an easy task. After a lengthy install, which included many steps, MailScanner reported “The MailScanner package failed to install. Address the required dependencies and run the install again.” After a 30-minute install this was pretty frustrating. In our view a system requirements check would be nice on a product that has been around for 15+ years.

+ RELATED: 4 email managers that can sort, organize, and delete +

As part of the install you can select to install a MTA such as Postfix or SendMail, there are also options to install SpamAssassin and ClamAV. MailScanner provides a ‘command center’ for several other open source anti-spam and anti-virus tools as well as over 20 commercial offerings. Like some of the other products reviewed, it is an adjunct to an existing mail server and provides an engine for pre-processing emails. As for system requirements, MailScanner can run on fairly basic hardware. According to MailScanner, a server with dual-core Xeon processors, 2GB of RAM and SCSI drives can process 1.5 million emails per day.

Once the basic install has completed, you can either install one of several Web GUIs available for management or simply manage MailScanner from the command prompt. MailScanner ships with what it calls ‘sensible’ and ‘reasonable’ configurations out of the box, but best practices dictate (and the vendor recommends) that settings be reviewed and modified as needed before putting the product into production. Configurations are mainly kept in a MailScanner conf file, but there are other configuration files available as well. The current release has no less than 346 configuration options, ranging from where to store quarantined mail to which virus and spam lists to integrate with.

Except for a few minor tweaks specific to our test server, we elected to use the defaults for most of our testing.

When mail comes in it is processed in a certain order, with determining if a message is spam as the first step. This is achieved by first checking the incoming message against several RBLs. If the message is determined to be from a known spam source, it is marked as spam and no further checking is done, thus saving processing resources. If it passes the first step, the message is then processed by SpamAssassin, which runs several tests and assigns a spam value and hands it off for virus detection by one or several virus scanners. The last step is to scan attachments, including the contents of compressed files, against a set of rules and either reject or accept the attached file(s).

As mentioned above, we wish MailScanner had a better installation interface providing more feedback about potential problems as you progress through the install. However, once installed we liked how MailScanner provides great granularity for handling messages containing spam and also its ability to integrate with other open source email products such as ClamAV and SpamAssassin as well as third-party commercial products. The documentation, essentially a 400-page PDF book by the founder of MailScanner (Julian Field), is very comprehensive and with in-depth information of all features.

Perschke is a web and database developer with 15+ years of industry experience. You can reach her at