CoderVortex
codervortex5@codervortex.com

Detecting Open Relay SMTP Servers with Free Testing Tools

An open relay SMTP server accepts and forwards mail from any sender to any recipient, regardless of whether either party has a legitimate relationship with the server. This configuration is rarely deliberate. It usually appears after an administrator misconfigures relay rules, leaves default permissions wide open, or follows outdated tutorials that predate modern security baselines. For businesses in Sydney, Melbourne, or Brisbane running their own Exchange or Postfix instances, the risk carries extra weight because Australian privacy expectations and the Notifiable Data Breaches scheme hold organisations accountable when their infrastructure is abused.

The fallout stretches well beyond a configuration footnote. Spammers continuously scan the internet for these servers, then push millions of phishing messages and malware payloads through them. Within hours, the offending IP lands on blocklists like Spamhaus and SORBS. Once blacklisted, legitimate transactional mail from that address — order confirmations, invoice reminders, booking notifications — is rejected by major providers including Gmail, Outlook, and local platforms run by Telstra and Optus. Recovery often takes weeks of correspondence with list operators.

Free online SMTP testers make the audit accessible to anyone who can point a browser at a domain. They automate the conversation between your server and a remote probe, then report whether the relay is open, closed, or partially misconfigured. It pays to combine them with broader infrastructure checks, which is why many Australian sysadmins keep the top 10 free online tools every sysadmin should bookmark handy for routine hygiene.

What an Open Relay Actually Is

A mail server is treated as an open relay when it forwards messages from an unauthenticated sender to an unauthenticated recipient. Authentication here means either the sender has logged in on the submission port (typically 587) or the connection comes from an IP address the server explicitly trusts. When neither condition is met, the server should reject the relay attempt and return a 5xx SMTP error.

The vulnerability typically surfaces after a technician enables "relay any recipient" during a migration, opens port 25 to satisfy a third-party scanner, or installs a default mail package on Ubuntu Server or AlmaLinux that ships with permissive transport rules. Australian hosts like Digital Pacific generally apply sensible outbound relay restrictions by default, but VPS customers running their own mail stack on Vultr, Linode, or the AWS Sydney region often have to tighten these settings themselves.

Manual Tests Versus Browser-Based Probes

The classic method is to open a telnet session to port 25 of the mail server, then manually type the SMTP conversation: HELO, MAIL FROM, RCPT TO. A closed relay returns "550 Relaying denied" after the uninvited RCPT TO; an open one returns "250 OK" and queues the message. This approach is still useful for engineers who want full visibility into the server's behaviour.

Browser-based testers remove the friction. They handle the dialog automatically, capture the transcript, and present a pass-or-fail verdict. Tools like MXToolbox and MailTester.com work without an account. The trade-off is limited visibility, so a careful admin runs more than one to catch edge cases. For organisations that operate a mix of mail and other network services, guides like how to create a simple online calculator for subnet masks earn their keep when planning IP ranges.

Step-by-Step Walkthrough of a Free Relay Test

The first step is identifying the mail exchange records for your domain. A quick dig MX yourdomain.com.au returns the hostname of your mail server. Copy that hostname, open a free SMTP tester, and enter it into the server field along with port 25. The tool will then initiate a fake transaction, posing as a stranger sending mail to another stranger outside your domain.

Within seconds the tester reports back with a clear verdict: relay open, closed, or partially restricted. Save the transcript, since it doubles as evidence of due diligence if a regulator or auditor ever asks how the server was validated. If the result is anything other than closed, jump straight to remediation before any further configuration work.

Reading SMTP Response Codes Correctly

Understanding the conversation matters as much as running the test. SMTP is a text-based protocol where every client command triggers a three-digit numeric reply. Codes in the 200 range mean success, 300 means accepted but pending more data, 400 is a transient failure, and 500 is a permanent rejection. During a relay test the key responses are 250 OK after RCPT TO (the server is willing to relay), 554 or 550 Relaying denied.

Any 250 reply during an unauthenticated session is a serious finding. A partial failure, where the server accepts the first recipient but rejects a second, often points to a misconfigured recipient whitelist or an overly broad transport map. A few patterns in the transcript reliably flag a server that needs attention.

Closing the Hole and Staying Closed

When a test returns an open relay verdict, isolate the server from production traffic to stop any active abuse. Then review the relay restrictions: in Postfix it lives in smtpd_relay_restrictions, in Exim it is the relay_from_hosts directive, and in Microsoft Exchange it sits on the Receive Connector permissions. Require SMTP authentication on port 587, force TLS for all submission traffic, and brush up on the fundamentals of TLS certificates so that expired certs do not cause submission ports to silently fall back to plaintext.

After tightening the configuration, retest from at least two external locations to confirm the fix holds. Australian businesses subject to the Spam Act 2003, enforced by the Australian Communications and Media Authority, can face substantial penalties if their infrastructure facilitates unsolicited commercial email. Routine quarterly testing, paired with blocklist monitoring, keeps the mail flow clean and the legal exposure minimal.