Rethinking Email: a Developer‑First, Unified, and Trustworthy System
A simple paper about what we’re building, why it matters now, and why it will work.
1) The problem, from first principles
Job to be done: Help product teams compose, send, observe, and govern email—across both transactional flows (e.g., verification, receipts) and marketing campaigns—with deterministic outcomes: did it send, deliver, bounce, get clicked, or lead to an unsubscribe?
What’s broken today (deconstructed):
- Fragmentation: Different tools, UIs, and models for “transactional” vs “marketing” create duplicate work and inconsistent behavior.
- Glue code everywhere: Webhooks, retries, deduping, unsub handling, and “resend” logic are hand‑rolled repeatedly—and often differently—per team.
- Deliverability + compliance are bolted on: SPF/DKIM/DMARC, one‑click unsubscribe, and complaint budgets are treated as afterthoughts, not guardrails.
- Misleading metrics: Apple Mail Privacy Protection makes opens unreliable; teams still over‑optimize for them.
- Unsafe environments: Dev and staging accidentally hit real inboxes; logs can leak PII.
Constraints we accept:
- Email must be authenticated and compliant (SPF/DKIM/DMARC, one‑click unsub for bulk).
- Spam complaints are a hard budget; deliverability is everyone’s concern.
- Privacy realities (e.g., Apple MPP) mean clicks and conversions are the signals that matter.
2) The one simple idea
Email as a Deterministic Ledger
Instead of “send + hope + parse some webhooks,” we treat each email as a transaction with a lifecycle recorded in a canonical, append‑only ledger. This ledger becomes the single place to see, trust, and act on what happened—live.
We collapse the entire domain into a tiny, human‑sized algebra:
- Template – versioned content with typed data.
- Audience – a single person or a segment, with consent.
- Send – the intentional act (with an idempotency key); schedule, throttles.
- Message – the concrete rendering that was (or will be) delivered.
- Event – the truth stream: sent, delivered, deferred, bounced, complaint, click, unsubscribe.
And two guardrail objects:
- Policy – deliverability/compliance rails (auth must be green; bulk requires one‑click unsub; complaint budget; warm‑up; throughput).
- Suppression – addresses you never email (unsubscribed, bounced, complained).
What this unlocks:
- No more webhook glue in your app. The system normalizes events into the ledger automatically.
- Resend as a policy, not a button. A small state machine enforces cooldowns, TTLs, token rotation, and abuse caps by default.
- Compliance by construction. Bulk sends automatically include one‑click unsubscribe and will not go if auth or rules aren’t satisfied.
- Privacy‑sound analytics. We elevate clicks and conversions; opens become an “estimate,” not a KPI.
3) Why now
-
Mailbox rules tightened (e.g., one‑click unsubscribe for bulk) and privacy changed (opens are noisy). A new baseline is required: compliance and clicks first.
-
Modern building blocks exist:
- Convex brings reactive, live‑query data and scheduled jobs with minimal ops.
- Next.js makes it trivial to build a safe operations UI and public endpoints.
- AWS SES + SNS provide reliable delivery and a full event surface. These let us collapse the stack into a few primitives with correctness built in.
4) What we are building (in plain language)
-
A unified system where transactional and marketing share the same mental model—no separate stacks, no duplicated features.
-
A live timeline for any email, user, or campaign that answers “what happened?” without leaving the app.
-
Guardrails that refuse risky sends: unauthenticated domains, missing one‑click unsub for bulk, unhealthy complaint rates, unsafe dev/staging sends.
-
A developer experience that feels obvious:
- Define a template,
- Call
Send
with an idempotency key, - Watch the ledger update in real time,
- Use
Resend
as a policy (not a custom flow).
Our stack, at a glance (no deep detail):
- Frontend: Next.js – the operations UI (timeline, previews, audiences, safe edits) and public endpoints (one‑click unsubscribe, verification).
- Backend: Convex – the ledger (Templates, Sends, Messages, Events, Policies, Suppressions), live queries, scheduled jobs (link expiry, warm‑ups), and idempotent mutations.
- Delivery + Events: AWS SES for sending; SNS for reliable event notifications that are normalized into the ledger.
5) Assumptions we explicitly challenge
-
“Transactional and marketing must be separate systems.” No. One model handles both; policy decides the behavior and constraints.
-
“Open rate is the KPI.” No. It’s noisy. We prioritize clicks and conversions, and show opens as estimates.
-
“Webhooks are the only source of truth.” No. The ledger is the truth; webhooks are just a transport into it.
-
“Resend = send again.” No. Resend is a state machine with abuse controls and token rotation.
-
“Dev must send real emails.” No. Non‑prod uses a sandbox/allow‑list by default; no accidental real sends.
6) What teams get (outcomes that matter)
- Confidence & clarity: Every send has a trace; every outcome is visible; every policy is enforced.
- Speed: From “blank repo” to “first compliant, tracked email” in minutes, not days.
- Fewer incidents: Guardrails prevent unauthenticated bulk, missing unsub, dangerous complaint spikes, and dev-to-real inbox accidents.
- Less glue code: Idempotency, retries, resends, suppression, and one‑click unsub live in the platform.
- Better decisions: Analytics center on click‑through and conversion—signals that matter to product and growth.
7) Our core beliefs & values
-
Correctness by default The safest behavior should be the easiest path. If a send isn’t compliant or authenticated, it doesn’t go.
-
Transparency > mystery A single ledger and live timeline replace “check five dashboards” and guesswork.
-
Privacy‑respecting analytics Measure what’s real—clicks and conversions—and label the rest honestly.
-
Safety in every environment Dev/staging never hit real inboxes unless explicitly allowed. Logs don’t leak PII.
-
Everything in code, UI for the right edits Types, schemas, and flows live in code; a safe UI lets non‑engineers edit content within guardrails.
-
Small, composable primitives A tiny vocabulary (Template, Audience, Send, Message, Event, Policy, Suppression, Link) covers 80–90% of use cases.
-
Portability and no lock‑in We prefer standard tech (Next.js, Convex, AWS SES/SNS). Adapters keep provider choices open.
-
Operational empathy Deliverability isn’t “ops only.” Developers get proactive feedback (auth checks, complaint budgets, warm‑up) and can prevent issues before they happen.
8) Why this will work
- It aligns with the new reality. Compliance and privacy are non‑negotiable; our defaults enforce them.
- It removes accidental complexity. One ledger replaces custom webhook mappers, ad‑hoc suppressions, and brittle resend code.
- It fits modern teams. Next.js + Convex reduce ops toil; SES/SNS bring proven deliverability and event clarity.
- It scales in capability without growing in complexity. The same primitives serve single‑user flows and million‑recipient campaigns; policy adjusts the rails.
9) How we will measure ourselves
- Time to first compliant send (from init to delivered with auth + one‑click unsub where applicable).
- Zero duplicates across retries (idempotency).
- Complaint rate kept below budget by default (guardrails and throttles).
- Ops load reduction (no custom webhooks, fewer support escalations).
- Developer satisfaction (does the model feel obvious and trustworthy?).
10) The near‑term roadmap
- M0 – Send & Observe: Ledger, live timeline, idempotent sends, SES/SNS integration, one‑click unsub injection for bulk, non‑prod safety.
- M1 – Resend & Campaigns: Resend policies (TTL/cooldown/caps), scheduled sends, throughput controls, warm‑up, complaint guard.
- M2 – Multi‑channel (optional): Reuse the same ledger for SMS/push where it makes sense.
- M3 – Self‑host option: For teams that need full control and data residency.
11) The promise
We’re building the simplest possible way to send email the right way: one model, one ledger, live truth—with guardrails that keep you safe and signals that move your product forward.
This is email that respects developers, honors users, and meets the web where it is today.
Below is a “now‑or‑never” plan that distills the report into one radical, simple idea (like Convex/Autumn did in their spaces), plus a concrete stack blueprint for Next.js + Convex + AWS (SES/SNS). I’ll keep it first‑principles, then give you the exact tech/tools and a minimal reference implementation path.
1) The “never‑seen simple” innovation
Email as a Deterministic Ledger (not “an API you hope worked”)
Treat every email as a transaction with a lifecycle recorded in a canonical, append‑only ledger you query directly—no webhook glue in your app. This collapses templating, send, tracking, unsub, resend, and compliance into five primitives that are observable and correct by construction:
- Template (versioned, typed)
- Audience (single or segment, with consent/legal basis)
- Send (idempotency key, schedule, throttles)
- Message (concrete render + metadata)
- Event (normalized provider events + internal actions)
Then add two policy objects:
- Policy (deliverability/compliance rails: SPF/DKIM/DMARC green, one‑click unsub, throughput, warm‑up)
- Suppression (unsub, bounces, complaints)
Net effect: Developers never parse webhooks, never re‑implement resend logic, never forget one‑click unsubscribe, and never guess what happened—because the ledger is the source of truth, reactively updating the UI and code, like Convex queries auto‑react.
Why this is a genuine simplification
- Idempotent
Send
is the unit of work. A single call with a required idempotency key collapses retries/deduping. (Most ESPs don’t give you this; you bolt it on.) - Resend is a policy, not a button. A compact state machine (cool‑down/TTL/abuse caps) invalidates prior tokens and emits canonical events (Click→Verified), so you don’t write that state logic again.
- Compliance is “by construction.” For any
policy.kind === 'bulk'
, the platform auto‑injects RFC‑8058 one‑click headers and enforces the 48‑hour honor rule, satisfying Gmail/Yahoo 2024 bulk‑sender requirements out of the box. (Google Help, Senders, IETF Datatracker) - Privacy‑sound analytics. Treat clicks/conversions as the KPI; opens are displayed as “estimated” only (MPP inflates them), aligning with modern guidance. (Twilio)
2) Design tenets (first principles → your stack)
-
Correctness by default
- Every
Send
is idempotent; every outcome is an Event; every Event is queryable. - Policies gate sending (auth aligned, one‑click unsub present, complaint budget OK) before mail leaves. Gmail/Yahoo one‑click and low complaint rate are enforced centrally. (Google Help, Senders)
- Every
-
Reactive DX
- The ledger lives in Convex, so Next.js pages subscribe to live timelines (no polling/webhooks in-app). This mirrors Convex’s “reactive by default” model.
-
Collapse to a tiny algebra
- App code touches Template, Audience, Send, Events—nothing else. Everything else (warm‑up, unsub headers, suppression, retries) is policy.
-
Escape hatches, zero lock‑in
- Under the hood, we use SES/SNS for deliverability & events. Our model remains provider‑agnostic and would let you add other adapters later.
3) Reference architecture (Next.js + Convex + AWS SES/SNS)
Outbound path
-
Next.js (admin UI + public flows) calls Convex mutations/actions.
-
Convex action uses AWS SDK v3 SESv2
SendEmail
with:ConfigurationSetName
enabling event publishing (delivery, bounce, complaint, delay, open, click). (AWS Documentation)- Custom Headers:
List-Unsubscribe
(https POST URL) andList-Unsubscribe-Post: List-Unsubscribe=One-Click
(RFC‑8058). Supported in SESv2 viaHeaders
. (Boto3, Amazon Web Services, Inc.) - MessageTags to correlate provider events with your ledger rows.
Inbound event path (two options)
- Simple (Day‑1): SNS → HTTPS (Next.js route)
SNS posts to
/api/ses/sns
. The route verifies SNS signature, confirms subscriptions, maps SES notification → canonical Event, and writes to Convex. - Robust (Day‑2): SNS → SQS buffer → Lambda relay → Convex Guarantees delivery & retries at scale; Lambda transforms SES notifications into canonical Events and invokes a Convex HTTP endpoint. (SES docs: set an Event Destination to SNS, or SNS→SQS, via a Configuration Set.) (AWS Documentation)
Tracking domains
- Configure SES custom open/click tracking domains to keep URLs on your brand; or bypass “opens” entirely and rely on our own signed, privacy‑sound link tokens for clicks/verification. (AWS Documentation)
Compliance gates
- CI “doctor” script checks SPF/DKIM/DMARC and blocks bulk sends until green; rejects bulk sends without RFC‑8058 one‑click headers. Gmail/Yahoo rules baked in. (Google Help, Senders)
4) Key modern technologies & best tools (why each fits)
Core runtime & data
-
Convex: reactive state, live queries, scheduled jobs (link TTL expiry), idempotent mutations. The ledger sits here—no cron infra to maintain.
-
Next.js (App Router):
- Admin UI: Timeline debugger (live), template preview, audience builder.
- Public routes: POST one‑click unsubscribe endpoint and verification link landing. (Gmail/Yahoo require HTTPS one‑click endpoints per RFC‑8058.) (Google Help, IETF Datatracker)
-
AWS SESv2 + SNS (+ SQS + optional Lambda): battle‑tested deliverability, full event surface (deliveries, bounces, complaints, opens, clicks) with Configuration Sets and Event Destinations. (AWS Documentation)
Email templating
- React Email (or MJML) for predictable HTML, typed props, and great Next.js ergonomics; auto‑generate text part. (If you prefer zero external deps, plain TSX +
@react-email/render
or MJML CLI works.)
Schema & safety
- TypeScript everywhere + Zod schemas for Template data, Events, and Policies.
- dns/promises in a Node CI script to verify SPF/DKIM/DMARC; fail if misconfigured.
Security & tokens
- Opaque link tokens (crypto‑secure random; store SHA‑256 hash + TTL + scope in Convex). Stateless JWT looks tempting, but revocation matters for resend—so use server‑validated opaque tokens.
Deliverability tooling
- SES Easy DKIM, custom tracking domain, complaint/bounce suppression in our ledger; warm‑up scheduler powered by SES sending limits.
- Optional: register Gmail Postmaster and Yahoo FBL outside code; our UI can link to these. (One‑click unsub + <0.3% complaint is the new baseline.) (Google Help, Senders)
Local/dev safety
- Mailpit/MailHog docker in dev; non‑prod allow‑list to prevent real sends.
Event parsing
- aws-sns-message-validator (Node) to validate SNS signatures on the Next.js route.
5) Minimal API surface (what devs actually use)
// 1) Create a typed template (React Email or MJML-backed)
Template.define<'verify_email', { userName: string; link: string }>('verify_email')
// 2) One line to send with idempotency + policy
await Send.create({
template: 'verify_email',
to: user.email,
data: { userName: user.name, link: Link.issue('verify', user.id, { ttl: '24h' }) },
idempotencyKey: `verify:${user.id}`,
policy: 'transactional'
})
// 3) Reactively observe outcomes (Convex query)
const timeline = useQuery('events.timelineByUser', { userId })
Resend on policy, not a button
await Send.retry({ sendId, policy: 'verification_resend' }) // rotates token, enforces cooldown & caps
6) Glue you do not write anymore
- Webhook consumers & bespoke mappers → gone (SNS → one canonical Event insert).
- RFC‑8058 one‑click wiring & 48‑hour SLA tracking → platform‑level. (Google Help, Mailgun)
- Dedup & idempotency → mandatory key on
Send
. - Resend state machine (TTL, caps, replay protection) → built‑in.
- Deliverability warm‑up, throttles, complaint guard → Policy with sane defaults.
7) SES/SNS wiring (concrete)
A. Outbound (Convex action using SESv2)
// convex/actions/sendEmail.ts
import { SESv2Client, SendEmailCommand } from '@aws-sdk/client-sesv2'
const ses = new SESv2Client({ region: process.env.AWS_REGION })
export async function sendEmail({ to, subject, html, text, headers, configSet, tags }:{
to:string; subject:string; html:string; text:string;
headers:{Name:string; Value:string}[]; configSet:string; tags:{Name:string;Value:string}[];
}) {
await ses.send(new SendEmailCommand({
Destination: { ToAddresses: [to] },
FromEmailAddress: 'MyApp <noreply@myapp.com>',
ConfigurationSetName: configSet, // enables events to SNS
EmailTags: tags, // correlate later
Content: {
Simple: {
Subject: { Data: subject },
Body: { Html: { Data: html }, Text: { Data: text } },
Headers: headers, // include RFC‑8058 headers below
}
}
}))
}
RFC‑8058 headers (bulk only):
const headers = [
{ Name: 'List-Unsubscribe', Value: `<https://app.example.com/api/unsub?tok=${tok}>` },
{ Name: 'List-Unsubscribe-Post', Value: 'List-Unsubscribe=One-Click' }
]
SESv2 supports custom Headers on SendEmail
; pair with a Configuration Set that publishes deliveries, bounces, complaints, delays, opens, clicks to SNS/SQS. (Boto3, AWS Documentation)
B. Inbound (SNS → Next.js)
// app/api/ses/sns/route.ts (Next.js App Router)
import { NextRequest, NextResponse } from 'next/server'
import { MessageValidator } from 'aws-sns-message-validator'
import { serverActionIngest } from '@/convex/_server' // call a Convex action/mutation
const validator = new MessageValidator()
export async function POST(req: NextRequest) {
const msg = await req.json()
await new Promise<void>((resolve, reject) =>
validator.validate(msg, (err) => err ? reject(err) : resolve())
)
// Handle subscription confirmation
if (msg.Type === 'SubscriptionConfirmation') {
await fetch(msg.SubscribeURL) // confirm
return NextResponse.json({ ok: true })
}
// Notification → map SES JSON to canonical Event and insert into Convex
const sesEvent = JSON.parse(msg.Message) // deliveries/bounces/complaints/opens/clicks/...
await serverActionIngest(sesEvent) // write to ledger, update suppressions, etc.
return NextResponse.json({ ok: true })
}
When you want bullet‑proof buffering & retries: switch to SNS → SQS → Lambda → Convex, using the same mapper. (AWS Documentation)
8) The “deliverability autopilot” you get for free
- One‑click unsub headers auto‑injected for any bulk send, meeting Gmail/Yahoo rules (and our endpoint honors within 48h). (Google Help, Mailgun)
- Auth guard blocks bulk sends until SPF/DKIM/DMARC are aligned (“green”).
- Complaint budget (target <0.3%) with auto‑throttle/pause on spikes. (Senders)
- Warm‑up scheduler respects SES limits; throughput scales as reputation builds.
- Clicks as truth: dashboards rank by CTR/conversion, not opens (MPP). (Twilio)
9) Tooling checklist (pick these and you’re done)
-
Frontend (Next.js)
- App Router, Server Components for admin UI
- @react-email/components +
@react-email/render
(or MJML CLI) for templates - aws-sns-message-validator on
/api/ses/sns
-
Convex
- Tables:
templates
,audiences
,sends
,messages
,events
,links
,policies
,suppressions
- Mutations:
send.create
,send.retry
,suppression.add
- Actions:
ses.sendEmail
,dns.checkAuth
,warmup.schedule
- Scheduled jobs: link TTL expiry, warm‑up ramping, complaint guard
- Tables:
-
AWS
- SESv2: Easy DKIM, Configuration Set, custom open/click domain,
SendEmail
with Headers and EmailTags (AWS Documentation, Boto3) - SNS: Event destination target for configuration set (AWS Documentation)
- (Optional) SQS + Lambda bridge for resilient event ingestion
- SESv2: Easy DKIM, Configuration Set, custom open/click domain,
-
CI / Safety
- Node script using
dns/promises
to verify SPF, DKIM, DMARC records - Non‑prod allowlist + Mailpit/MailHog docker
- Lint: ensure RFC‑8058 headers on bulk sends
- Node script using
10) Why this will feel like Convex/Autumn–level simplicity
- One mental model:
Template → Send → Events
, everything else is Policy. - Reactive correctness: open the timeline and watch truth flow in, because SNS→ledger→UI is one path, not “10 webhooks” you stitched manually.
- By‑construction compliance: you can’t forget one‑click unsub or auth alignment; the platform won’t send non‑compliant bulk mail. (Google Help)
- Clicks first analytics matches modern reality (MPP makes opens noisy). (Twilio)
11) Immediate next steps (concrete)
- Create SES Configuration Set (+ SNS destination) with events: Delivery, Bounce, Complaint, Delay, Open, Click. (AWS Documentation)
- Add custom tracking domain for SES open/click (or skip opens; keep clicks). (AWS Documentation)
- Stand up
/api/ses/sns
(Next.js) with signature validation; map to Convexevents.ingest
. - Ship the 5 primitives in Convex (schema + basic mutations); require
idempotencyKey
. - Implement RFC‑8058 autopilot: inject
List-Unsubscribe
+List-Unsubscribe-Post
forpolicy.kind === 'bulk'
. (SESv2Headers
supports this.) (Boto3) - Wire the resend policy (cooldown/TTL/cap) and link tokens (opaque, hashed, expiring).
- Add the CI doctor (DNS/auth + “no bulk without one‑click”).
- Timeline UI in Next.js using Convex live queries.
This gives you a production‑safe MVP with deterministic outcomes and radically less glue—an innovation that’s simple enough to explain in one breath and strong enough to carry the platform for years.
If you want, I can draft the Convex schema and the two or three core mutations/actions next, following this exact blueprint.