Delivery Events and Diagnostics

Every message an ESP handles passes through a sequence of states, and each state transition is an event the platform records and can stream to the customer. This event stream is the raw telemetry beneath every deliverability diagnosis: bounce/complaint rates, per-provider inbox estimates, delay analysis, and engagement scoring are all aggregations over it. This article documents the event taxonomy an ESP/MTA emits, using the SendGrid Event Webhook as the canonical, fully-specified example, then covers the reporting, delay-diagnosis, and engagement-scoring layers built on top.

For the thresholds these events feed and the corrective actions they trigger, see Metrics and Benchmarks; for the step-by-step diagnosis flows, see Delivery Troubleshooting Playbooks. This article is the instrumentation layer beneath both.

The event taxonomy

Events split into three families: delivery events (what the sending system and receiving servers did with the message), engagement events (what the recipient did), and account events (platform-level status changes). SendGrid's Event Webhook posts these as JSON; the names are vendor UI labels, but every sending infrastructure produces equivalent states.

Delivery events

Event	Meaning	What triggers it
processed	The platform accepted the message and can deliver it	Message injected, validated, and queued for the outbound MTA
deferred	Receiving server temporarily refused the message	4.x.x reply at the remote MTA (greylisting, rate-limit, "try again later"); the platform retries
delivered	Receiving server accepted the message (250 OK)	Remote MTA returned a 2.x.x success on DATA
bounce	Receiving server rejected the message	5.x.x permanent rejection (or a temporary failure that exhausted retries), returned at SMTP time or via async DSN
blocked	A soft bounce sub-type: temporarily rejected for reputation/content/technical reasons	Remote MTA rejected but the failure is not "address invalid"; represented as bounce type: "blocked"
dropped	The platform itself refused to send — the message never left	Recipient on a suppression list (prior bounce, unsubscribe, spam report, invalid address), spam-flagged content, invalid header/template, or quota exceeded

Key semantic distinctions an operator must internalize:

• deferred ≠ bounce. A deferral is a temporary refusal that the platform keeps retrying; it becomes a bounce only if retries are exhausted. Persistent deferrals are the classic signature of throttling/rate-limiting (see Delay and latency diagnosis).
• bounce vs. blocked. SendGrid encodes both under the bounce event and disambiguates with the type field: type: "bounce" = hard bounce (permanent, e.g. address doesn't exist); type: "blocked" = soft bounce (temporary reputation/content/technical rejection). In the UI's Bounces & Blocks reporting, a Bounce means an invalid email address (never existed or deactivated) while a Block means the message was rejected for "content and reputation issues or technical failures." A block is a receiver signal about you, not about the address — it can clear once the underlying reputation/content issue is fixed. See Blocklists and Spamhaus.
• dropped is self-inflicted / protective. The platform suppressed the send before transmission. Most drops are healthy (suppressing a known-bad address protects reputation); a spike in drops for unexpected reasons (invalid template, quota exceeded) signals a sending-side defect, not a receiver problem. Suppression scoping is covered in Suppression-List Architecture.
• delivered ≠ inbox. delivered is posted on the receiving server's 250 OK. At that point the receiver may still route the message to the inbox, queue it, file it to spam/junk, drop it silently, or (Gmail) sort it into a tab — and the sending platform gets no further signal about any of that. The only downstream evidence is recipient engagement (open/click). A high delivered rate with near-zero engagement is the fingerprint of spam-foldering. This is why inbox placement can only be estimated (seed tests, provider postmaster dashboards), never read directly off the delivery stream. See Tracking and Measurement Distortion and Reputation Monitoring.

Bounce classification

For bounce events SendGrid attaches a bounce_classification bucketing the SMTP failure into a cause an operator can act on:

Classification	Meaning / operator action
Invalid Address	Address doesn't exist / never existed — remove permanently (list hygiene)
Mailbox Unavailable	Mailbox full or temporarily unreachable — often soft; may recover
Technical	DNS, connection, protocol, or TLS failure — infrastructure-side
Content	Message content triggered rejection/filtering — fix the content/links
Reputation	Sending IP/domain reputation caused rejection — reputation remediation
Frequency/Volume	Sending rate/volume too high for the receiver — throttle (see MTA Delivery Tuning)
Unclassified	Could not be mapped

The Content and Reputation buckets are the ones that feed engagement-quality scoring (below) as an early-warning signal, because they indicate the receiver is judging you, not that the address is bad.

Engagement events

Event	Meaning	What triggers it
open	Recipient rendered the HTML message	Open-tracking pixel loaded (requires Open Tracking enabled)
click	Recipient clicked a tracked link	Wrapped link followed (requires Click Tracking enabled)
spamreport	Recipient marked the message as spam (a complaint)	Recipient's "mark as spam" action, relayed via the provider feedback loop
unsubscribe	Recipient clicked the global "opt out of all" link	Subscription Tracking enabled
group_unsubscribe	Recipient unsubscribed from one suppression group	Group unsubscribe link or preferences page
group_resubscribe	Recipient re-subscribed to a group	Preferences page (Subscription Tracking enabled)

Engagement caveats (critical for not being misled by the stream):

• Machine opens. SendGrid sets sg_machine_open: true when Apple Mail Privacy Protection (MPP) generated the open rather than a human. These must be filtered out of any real engagement measure. Gmail image prefetch similarly fires opens without a human. See Tracking and Measurement Distortion and Non-Human Interactions.
• spamreport is your complaint signal. It is the sender-facing half of the provider feedback loop; sustained spamreport rate is the single most damaging reputation input. See Complaint Feedback Loops.

Account events

SendGrid emits account_status_change when the platform changes an account's standing for compliance reasons (phishing, elevated spam rates, "other bad behavior"). The type field carries the action, escalating in severity:

type	Effect
compliance_suspend	Blocks delivery; queues messages, bounces them at delivery time
compliance_deactivate	Blocks delivery; rejects queues; deletes queued messages; bans the user after 48 hours
compliance_ban	Blocks delivery; rejects and deletes queues; removes console access; cancels billing; removes IP addresses
reactivate	Restores the account to active status

For the ESP-operator's side of this — detection, containment, and the enforcement ladder — see Compromised Accounts, Outbound Monitoring, and Multi-Tenant Architecture.

Event payload fields

The webhook JSON carries a common core plus event-specific fields. This is the diagnostic vocabulary — the fields you correlate to answer "what happened to this message."

Field	Type	Events	Meaning
email	string	all	Recipient address
timestamp	int (UNIX)	all	When the event occurred
event	string	all	Event type identifier
sg_event_id	string	all	Unique event ID (URL-safe, ≤100 chars) — dedupe key
sg_message_id	string	all	Unique message ID — join key across a message's events
smtp-id	string	delivery + bounce/spam/group	Message-ID assigned by the originating system
category	string / array	delivery + engagement	Custom organizational tags set by the sender
asm_group_id	int	most	Unsubscribe (suppression) group ID
marketing_campaign_id / _name	int / string	delivery + engagement	Campaign identifiers
pool	object {name, id}	processed	IP pool the message was sent from
ip	string	delivered, open, click, group unsub/resub	Sending IP (delivered) or recipient IP (engagement)
response	string	delivered, deferred	Full text of the receiving server's response
reason	string	bounce, deferred, dropped	Human-readable error/drop reason
status	string X.Y.Z	bounce, dropped	Enhanced status code (see Enhanced Status Codes)
attempt	int	deferred	Number of delivery attempts so far
type	string	bounce, account	bounce/blocked, or the account-status action
bounce_classification	string	bounce	Cause bucket (table above)
tls	bool	bounce, delivered	Whether the connection used TLS
url	string	click, group unsub/resub	The URL involved
url_offset	int	click	Zero-based index of the link in the HTML (duplicate URLs get distinct indexes)
useragent	string	open, click, group unsub/resub	Client program that generated the event
sg_machine_open	bool	open	True if Apple MPP generated the open
unique_args	object	delivery + engagement	Custom sender-supplied parameters

Operational note: SendGrid warns never to put PII in category / unique_args — the platform may consume these for internal operations, they cannot be redacted, and they may be retained long-term.

A typical diagnostic query joins all events sharing an sg_message_id and orders by timestamp to reconstruct one message's full history (processed → deferred×N → delivered → open), or aggregates by event + bounce_classification + receiving domain to find which provider is rejecting which category of mail.

Deliverability-insights reporting

Aggregating the event stream into a diagnostic dashboard is the "Deliverability Insights" pattern (SendGrid: Stats → Deliverability Insights). Data lags roughly 48 hours behind real time. It presents four views:

View	Shows
Overview	Processed ("how much mail you tried to send"), Delivered (% accepted by mailbox providers), Bounce & Blocked (% undelivered), Unique Opens (% of delivered that were opened) — trended over time
Mailbox Providers	The same metrics segmented per provider (Gmail, Yahoo, Microsoft, AOL, …) — delivered rate, open rate, bounce/block breakdown
Bounces & Blocks	Splits the two failure types: Bounce = invalid address; Block = rejected for content/reputation/technical reasons
Spam & Unsubscribes	Complaint and opt-out behavior, by provider

The decisive design choice is per-mailbox-provider segmentation. A blended 95% delivered rate can hide a total block at one provider; reputation is per-provider, so diagnosis and remediation must be too (see Reputation Monitoring and Per-Provider Tuning Baselines). SendGrid's published interpretation thresholds for this view:

Signal	Threshold
Block rate	Target ≤ 3% at each primary mailbox provider
Spam report rate	≥ 0.1% at any single provider is excessive
Bounce rate	consistently over 5% is concerning
Sunset trigger	Messages unopened/unclicked for 3 months → stop mailing that address

These reconcile with the cross-source thresholds in Metrics and Benchmarks (the ≤0.1% complaint / <5% bounce consensus, the 0.3% Gmail/Yahoo violation line).

Delay and latency diagnosis

"Delay" can mean two very different things, and conflating them misdirects diagnosis:

1. Receiver-side delay (deferrals / throttling). The message left the platform but the receiver is holding it back — greylisting, rate-limiting, or "try again later." This surfaces as deferred events with a rising attempt count and a 4.x.x response. The platform retries on a schedule; SendGrid retries deferred messages for up to 72 hours, after which the message is converted to a bounce. Persistent deferrals concentrated at one provider = you are exceeding that provider's rate/volume tolerance — the remedy is receiver-side shaping (reduce concurrency/rate, spread the send across more hours), not more retries. See MTA Delivery Tuning and Greylisting (RFC 6647).

2. Platform-side / injection delay (stuck in Processing). The message never reaches a receiver — it sits in processed state. SendGrid documents several causes:

• New dedicated-IP volume restriction — a volume cap is applied for the first 3 days on a new dedicated IP to prevent abuse; ramp with the IP warm-up process.
• Compliance hold — an account under review, or one where unusual activity was detected, has sending paused until Compliance confirms.
• MTA queue backpressure — very high sending volume can back messages up in the outbound MTA queue; pause sending or bring another IP online.
• Dormant-account suspension — accounts with no logins are auto-suspended.

Messages remain pending for up to 72 hours; if the underlying issue isn't resolved, they become hard bounces.

3. Transport latency (integration/network). The message is slow into the platform. SendGrid's guidance targets throughput and network path:

• Use official client libraries (C#, PHP, Ruby, Node.js, Python, Go, Java) rather than hand-rolled code.
• SMTP throughput: up to 5000 messages per connection, up to 1000 To: recipients per message (via the x-smtpapi header).
• Open additional concurrent connections — recommended maximum ~10 concurrent connections.
• Network diagnostics: hping / Test-NetConnection measure response time and TTL; anything exceeding ~150 ms response time is a first red flag. Traceroute mode exposes per-hop latency (watch for sudden jumps between hops). Header analyzers (e.g. Google's) show the message's per-MTA journey and dwell time at each hop; Wireshark captures the SMTP conversation for deep inspection.

Diagnostic order of operations: check event state first (processed → platform/compliance issue; deferred → receiver throttling; delivered but no engagement → spam-foldering, which is a placement problem not a delay). Only once the message is confirmed to be leaving the platform normally do transport-latency tools apply.

Engagement-quality scoring

The highest-level aggregation collapses the whole event stream into a single health score. SendGrid's Engagement Quality (SEQ) API is the canonical example: it returns a score from 1 to 5 (higher = better engagement and deliverability), meant to let an operator rank senders/subusers at scale and spot degradation before it becomes a reputation incident.

Requirements to generate a score: open tracking enabled and ≥ 1,000 messages sent in the previous 30 days. Scores are retained for a maximum of 90 days. Endpoints return scores at the account level and per subuser.

The score is composed of five weighted components — note these are exactly the event-derived signals above, distilled:

Component	What it measures
Engagement Recency	% of unique addresses sent to in the past 30 days that also engaged (opened or clicked) in the past 90 days
Unique Open Rate	Lowest rate across the last 7 and 30 days for the top 5 mailbox providers; excludes Apple machine opens and non-deliveries
Bounce Rate	Permanent-bounce % over the last 7 and 30 days; uses the highest (worst) of the two
Bounce Classification	Weighs bounces classified as Reputation or Content (the receiver-judging-you buckets)
Spam Rate	Recent spam complaints over a 7-day window

The design tells you what a platform actually treats as "quality": engaged recency (are you mailing people who still want it), real (non-machine) opens at the providers that matter, permanent bounces trending the wrong way, reputation/content bounces specifically, and complaints. An ESP can use these scores operationally to group subusers into IP pools, flag underperforming senders for outbound monitoring, and gate promotion/demotion between shared and dedicated infrastructure.

Related

• Delivery Troubleshooting Playbooks — the step-by-step diagnosis flows that consume this telemetry
• Metrics and Benchmarks — the thresholds and corrective actions these events feed
• Enhanced Status Codes (RFC 3463) — reading the status field
• Delivery Status Notifications (RFC 3464) — the async-bounce format behind bounce events
• Tracking and Measurement Distortion — why opens/clicks lie (MPP, prefetch, scanners)
• MTA Delivery Tuning — the shaping that resolves persistent deferrals
• Complaint Feedback Loops (RFC 6449) — the source of spamreport events
• Reputation Monitoring — turning per-provider event data into a placement estimate