SMTP Production Debugging

Objective

Diagnose outbound SMTP incidents by evidence layer and unique message outcome. Do not infer failure rate from raw error-line counts.

Workflow

1. Identify the active MTA, runtime/container, effective configuration, persistent log source and current queue before changing anything.
2. Define an analysis window and correlate events by Postfix queue ID. Separate unique messages from delivery attempts/retries.
3. Classify each failure at the earliest proven layer: DNS resolution, TCP connectivity, SMTP session/banner, temporary SMTP policy, permanent SMTP policy/recipient, or sender authentication/reputation/content.
4. Group persistent failures by recipient domain, MX hostname and MX IP. Distinguish isolated/transient MX failure from a repeated destination cluster.
5. Build controls from the same source host/IP and time window: unrelated healthy MXes and, when possible, another MX in the same provider family.
6. For pre-banner failures, resolve MX/A/AAAA and perform spaced TCP probes. Capture whether the result is connected/banner, timeout, refusal or explicit network error. Do not send mail during a connectivity probe unless the task requires a controlled delivery test.
7. Use TCP traceroute/MTR only as supporting path evidence. Do not treat ICMP failure or intermediate-hop probe loss as proof of end-to-end forwarding loss.
8. Test local/provider hypotheses against persistence and controls. A documented rate limit is not automatically the cause of a TCP handshake timeout.
9. Preserve confirmed facts, strong hypotheses and unproven hypotheses separately. State what additional telemetry would be required for claims outside the host's visibility.
10. Recommend the smallest reversible remediation, then verify queue behavior and final delivery independently.