Evaluation guide

Evaluating MailThreatZero on your own mail

How to run a gateway evaluation that answers your question rather than ours: one domain, real traffic, every verdict recorded and none enforced until you decide the policy is right. This page covers what you need, which mail flows are supported, how to review verdicts, and how to switch on enforcement and back off again.

Every statement here describes the product as it runs today. Where a capability is not generally available, its status comes from the same feature registry as the rest of the site.

One-domain evaluation checklist

The whole evaluation on one page, in order. Each item links to the section that explains it. Print it, or copy it into your ticket.

Before you change anything

Pick one domain whose mail you can judge on sight, with at least two weeks of normal traffic ahead. Scope
Write down the current MX records and their TTL. They are your rollback. Rollback
Confirm you can edit the domain’s DNS and administer its mail platform. Prerequisites
Check the domain’s receiving addresses fit the allowance: aliases, shared and distribution addresses count. Calculator
Decide what you are testing for and write it down: false positives on a known sender, missed phishing, explainability, or the console workflow.

Cut-over

Add the domain in the console with Monitor Mode switched on, and its destination host. Monitor Mode
Create the inbound connector or inbound gateway from the generated settings. Mail flow
Let the wizard’s checks pass: DNS, destination, acceptance from the gateway.
Point MX at mt0.mailthreatzero.com, then send a test message from outside and find it in message tracking.

During Monitor Mode

Read every message that would have been held. Verdict review
Read a sample of what would have been delivered, starting with mail you expected to be stopped.
For each wrong verdict, find the deciding stage. Investigating a verdict
Keep the tally: held and correct, held and wrong, delivered and should not have been.
Tune per domain and watch the next days’ verdicts change. Policy tuning

Before enforcement

Agree the tally with whoever owns the domain. Enforcement
Restrict the mail platform to accept inbound mail only from the gateway.
Keep the reject threshold generous: held mail is recoverable, rejected mail is not.
Know both rollback levers: Monitor Mode back on (immediate), or MX back (TTL).

Not on this list because it is not part of an inbound evaluation: outbound routing, archiving and the capabilities under known beta limitations.

Scope and prerequisites

An evaluation is one protected domain, inbound mail, in Monitor Mode first. Start with a domain whose traffic you know well enough to judge a verdict on sight. A second domain is added the same way once the first one has told you what you wanted to know.

Control of the domain's DNS: Monitor Mode can only score mail it receives, so the domain's MX record has to point at mt0.mailthreatzero.com. Note the current MX value before you change it: it is your rollback.
Administrative access to the mail platform: To create the inbound connector or inbound gateway that accepts filtered mail, and later to restrict the platform to accepting mail only from the gateway.
A destination that accepts SMTP from the gateway: One destination host per domain. The gateway relays to it over SMTP with TLS.
Addresses: Up to 250 receiving addresses included per protected domain.
Optional: AI analysis: Optional AI analysis uses the provider and credentials selected for the deployment and can be enabled or disabled per domain. It runs on your own provider account and is not needed to evaluate the other scanning stages.
Time: Two weeks of normal traffic is a sensible minimum. Fewer days than that and the sample may simply not contain the mail you care about.

Supported mail-flow configurations

MailThreatZero is an MX gateway. Inbound mail arrives at the gateway, is scanned, and is relayed over SMTP to the domain's destination. There is no API-only or MX-less mode: mail does not travel through Microsoft Graph or the Gmail API.

Microsoft 365

An Exchange Online inbound connector accepts mail from the gateway. The console generates the connector, Enhanced Filtering and lock-down settings for your domain. Microsoft 365 details.

Google Workspace

A Google Workspace inbound gateway entry does the same job, generated for your domain. Google Workspace details.

Any other SMTP destination

On-premises Exchange, a hosted provider or a server you operate: anything that accepts SMTP from a smart host.

Current production status. Domains protected today relay to customer-operated SMTP servers. The Microsoft 365 and Google Workspace configurations follow each platform's documented inbound-connector settings; no production domain uses either path yet, so an evaluation on either platform comes with a cut-over we work through with you.

Outbound mail is a separate routing decision. Outbound inspection and DLP are available for mail routed through the MailThreatZero gateway. Coverage depends on mail-flow configuration. This guide covers inbound.

Monitor Mode

Monitor Mode records how MailThreatZero would classify messages while allowing normal delivery. Review decisions, tune policy, and validate the fit before enabling enforcement. It is a per-domain switch, and it is where every evaluation should start.

Every verdict is simulated. Every scoring verdict is simulated. A message that would be delivered, quarantined or rejected is delivered and recorded instead — including one a scanner confirmed as malware. Monitor Mode measures what the policy would do; it does not protect the domain while it runs.
SMTP-time blocklists are switched off. The SMTP-time DNS blocklist refusal is switched off for a domain in Monitor Mode. It has to be: that refusal happens at RCPT, before the message is ever scanned, so leaving it on would refuse mail the mode exists to observe.
What still applies. Controls that are not part of scoring still apply. Your mail server's own recipient, size and rate limits are unaffected, and if the scanner itself fails the message is deferred with a temporary error for the sender to retry, exactly as it would be outside Monitor Mode.
The message may still be modified. Delivery is unchanged; the message is not necessarily unchanged. Warning banners and link rewriting are separate per-domain settings and are not suspended by Monitor Mode, so if you have either enabled the body is still modified on the way through.
It needs your MX. It needs MX pointed at the gateway, because it can only score mail it receives. That is the one change you make to start; nothing else about delivery changes.

Reviewing a sample of verdicts

Message tracking records every message the gateway handled with the score from each stage that ran, the decision, and the reason for it in a sentence. In Monitor Mode that sentence says what scoring would have done: Delivered under monitor mode: this domain records verdicts without acting on them. Scoring would have chosen ‘quarantined’.

Read every message that would have been held. Quarantine and reject are where a wrong verdict costs something, and in most domains they are a small fraction of the traffic.
Read a sample of what would have been delivered. A filter is judged on what it missed as well as on what it stopped. Pick messages you would have expected to be stopped, and check how close they came.
Find the deciding stage. Each breakdown shows which stages scored and by how much. A stage that did not run, timed out, or could not read an artifact is reported as exactly that, never as a pass. An encrypted archive the gateway cannot open is a coverage gap, not a clean result.
Keep a tally. Held and correct, held and wrong (a false positive), delivered and should not have been (a miss). Those three numbers, from your own mail, are the result of the evaluation.

To see how a single verdict is presented before you have any of your own, the product demo walks through four synthetic examples, and investigating a false positive covers reading a breakdown in detail.

False-positive handling

In Monitor Mode a false positive costs nothing: the message was delivered, and the record shows it would not have been. That is the cheapest time to find one. Once enforcement is on:

Quarantined mail can be released: Release delivers the stored original message and bypasses re-scanning. Releases are recorded with who released the message and when.
Rejected mail cannot: A rejected message is discarded and recorded as rejected, with the deciding stage; the sender is not notified. That is why the reject threshold is the last thing to lower, not the first.
Fix the cause, not the message: The breakdown names the stage that decided. The fix is usually a per-domain threshold, a stage switched off for that domain, or an allow-list entry.
Allow-list entries are claims: An entry matches on an address or domain that a sender can write into a header. The gateway records whether each allow-list match was backed by passing, aligned SPF or DKIM, and requiring that is a per-domain option, off by default. Ask us to switch it on for a domain where allow-listing matters.
Tell us: A verdict you believe is wrong is the most useful thing an evaluator can send. We read the record behind it, not a summary of it.

Policy tuning

Everything below is set per domain, so one customer's tuning never moves another's. Monitor Mode shows the effect of a change against the traffic that domain actually receives, without anyone's mail being held while you find out.

Thresholds. A quarantine threshold and a higher reject threshold. Raising quarantine trades caught mail for fewer false positives; raising reject moves borderline mail into quarantine, where it can be recovered.
Which stages run. Content, header, phishing, attachment and QR analysis are switched on per domain, as is optional AI analysis.
Allow and block lists. Per domain, with the caveat above.
Banners and link rewriting. Separate per-domain settings that change the message body, independently of the verdict.

We publish the concepts rather than the exact weights, which are tuning detail an evader would find more useful than a buyer. Every message's per-stage scores are visible in your console.

Activating enforcement

Enforcement is switching Monitor Mode off for the domain. It takes effect for the next message; no DNS change is involved, because mail is already flowing through the gateway.

Agree the numbers first. The tally from your verdict review is what you are accepting. If the false positives in it are not ones you would tolerate in production, keep tuning.
Close the side door. Restrict the mail platform to accepting inbound mail only from the gateway. Until then, a sender who knows your previous MX can deliver around the filter.
Start with a generous reject threshold. Held mail is recoverable; rejected mail is not.
Watch the quarantine for the first days. Release anything that should have arrived, and correct the policy that held it.

Rollback and mail continuity

Rollback has two levers, and the faster one does not involve DNS.

1. Stop enforcing: switch the domain back to Monitor Mode: Scanning continues and nothing is blocked, immediately. Mail keeps flowing through the gateway, so there is no propagation wait. This is usually what is actually wanted.
2. Stop routing: point MX back at the previous host: The value you noted before the cut-over. The change is made in DNS you control, which the gateway does not hold, so it cannot be done from the console.
Wait out the TTL: Senders keep using the old MX until their cached copy expires, so mail continues to arrive at the gateway for up to the published TTL. It is scanned and relayed as normal.
Deal with held mail: Quarantined messages are not forwarded by a rollback and do not expire on their own. Release anything that should be delivered before you finish.
If the gateway's scanner fails: The message is deferred with a temporary error and the sending server retries it. Nothing is lost or silently passed.
If your mail server is down: The gateway queues mail rather than bouncing it. How the queue works and what it is not.

Known beta limitations

Every capability that is not generally available today, with its own caveat. This list is generated from the feature registry, so it changes when a status does. None of these is needed to evaluate inbound filtering.

Payment and invoice fraud detection Beta
Off by default, and monitor-only when first switched on. Payment language alone scores nothing, however much of it there is — a finance team discusses invoices, and that is what a finance team does. It scores only when the language is corroborated by something about the sender: no correspondence history with that person, a near-miss of a domain you really deal with, authentication that failed, a reply that would go somewhere else. One corroborating signal is a suspicion; two is a finding, and the finding names which. It is offered as beta until it has been validated against real payment-fraud traffic, and starting monitor-only lets you confirm it leaves ordinary finance mail alone before it acts.
Shared threat intelligence map Beta
Public map of senders and infrastructure seen attacking protected domains, from the gateway's own observations and published open feeds. Public now, and labelled beta while its data sources are broadened.
Account-takeover risk from Microsoft and Google identity telemetry Beta testing
Coverage depends on your provider licence - sign-in risk signals are not available on every plan. Not yet verified against a live customer tenant; the gateway-side sending-behavior detection it complements is generally available and is listed separately.
Mailbox lookup in Microsoft 365 and Google Workspace Beta testing
Built against the Microsoft Graph and Google Directory APIs and covered by tests, and in beta testing until it has run against a live customer tenant.
Post-delivery removal with verification Beta testing
Every action is a MOVE to a recoverable folder, never a delete, and the result is read back and reported per mailbox. Built and covered by tests against recorded Graph and Gmail behavior; not yet verified against a live customer tenant.
Bulk campaign remediation Beta testing
Joins campaign detection to per-message remediation so one confirmed run is handled once. Built and tested; not yet verified against a live customer tenant.
Journal ingestion (internal mail capture) Beta testing
Built and tested: a journal rule on Microsoft 365 or Google Workspace can deliver internal user-to-user mail to a per-tenant journal address, and the report is unwrapped to the original message and its true recipients. The journal endpoint is not yet switched on for customers, so internal mail is not captured today; until it is, the archive holds mail that passed through the gateway.
Webhooks and SIEM delivery Beta testing
Built and covered by tests; not yet delivered to a live customer SIEM, Slack or Teams endpoint.
Single sign-on (OpenID Connect) Beta testing
Built and covered by tests; not yet run against a live customer identity provider.
SCIM provisioning Beta testing
Built and covered by tests; not yet run against a live customer identity provider. An identity provider (Entra, Okta) creates and disables console accounts, so offboarding happens in one place. A tenant-scoped bearer token authenticates SCIM 2.0 at /scim/v2; what a provisioned user may do comes from the group-to-role mapping an operator configures, not from the directory. An unmapped group earns the lowest role and superadmin can never be granted this way. Tokens are issued and revoked from Settings > Directory Sync (SCIM), shown once, and stored hashed.
Google Postmaster Tools connection Beta testing
Built and covered by tests against Google's published v2 API schema; not yet run against a live Postmaster account. Google removed domain and IP reputation from Postmaster Tools, so this reports Google's sender-requirement compliance verdicts and daily Gmail statistics, in Google's words. It needs a Google Cloud OAuth client, a connected Google account, and each domain registered and DNS-verified in that account; only domains with enough mail to Gmail get data.
Behavioral attachment analysis (VirusTotal, Joe Sandbox, or a self-hosted CAPE host) Requires setup
MailThreatZero does not offer detonation as a hosted service - you supply the provider account or you operate the analysis host. A VirusTotal hash lookup is not a detonation. It returns verdicts from sandboxes VirusTotal has ALREADY run against that exact file, and returns nothing when no sandbox has ever run it - it is never filled in from the multi-engine antivirus result, which is a different stage answering a different question. Detonating a file nobody has analyzed requires uploading it (off by default on every domain), a Joe Sandbox account you hold, or a CAPE host you operate.
User-reported phishing triage Requires setup
Reporting, triage and campaign correlation work on the gateway alone. Removing matching copies from other mailboxes requires a connected tenant.
Spamhaus, SURBL and Google Web Risk for protected domain reputation Requires setup
Off until the platform operator supplies a Spamhaus DQS key, a licensed SURBL zone or a Google Web Risk API key. Spamhaus's and SURBL's free services exclude commercial filtering providers, and Google Safe Browsing is non-commercial only, so domain reputation monitoring queries none of them without one. Until then they are shown as unavailable and are never counted as checked.
Automatic post-delivery remediation execution (unattended removal) Not built
Not built, and deliberately so. The policy engine above decides; nothing acts on that decision without a person. Removal of delivered mail happens only through the mailbox integrations, one reviewed action at a time, and unattended execution will not be offered until it can be validated against a live tenant.
SAML 2.0 Not built
SAML 2.0 sign-in.

Start with one domain.

A walkthrough of the console first, then a domain in Monitor Mode on your own traffic.

Request a Demo Apply for the Beta