Skip to main content
← Back to blog

Staged Deploys Without Rollbacks: Publish Only After Synthetic Checks Pass

Cowpin

Cowpin

10/18/2026

#devops #vercel #deployment #testing #saas #reliability
Staged Deploys Without Rollbacks: Publish Only After Synthetic Checks Pass

Rollback is a trap once production code has already applied a database change. Old binaries against new columns (or the reverse) can be worse than the bug you are trying to undo.

Cowpin’s answer is simpler: bad builds never get traffic. We combine:

  1. Compile + unit gate inside the Vercel build (CI in the build).
  2. Staged release — production builds succeed but do not auto-assign www.cowpin.com.
  3. Post-build synthetic checks on the new deployment URL, then manual promote via API when green.

There is no rollback button in this design on purpose.

Step 0: Vercel auto-publish is off

With autoAssignCustomDomains: false, a green production deploy gets a URL like https://cowpin-xxx.vercel.app while customers stay on the previous build.

git push means “candidate release,” not “live in three minutes.”

Step 1: Webhook → AWS synthetic monitor

On deployment.succeeded, Vercel calls our Lambda (same family as the 6‑hour prod monitor). The handler answers HTTP immediately and finishes work in a background run.

Step 2: Twelve core checks on the new URL

Against the hidden deployment (with Vercel’s protection-bypass header), we run flows that only make sense on real infrastructure + real Postgres:

  • Marketing pages and 404 behavior
  • Pinboard v1 auth walls (401, not accidental 500)
  • MCP initialize + tools/list
  • Signed-in read via Pinboard API
  • Real save → delete (write path + latency)
  • Privacy default — saves without shared stay private

The list lives in code as STAGED_CHECKS in monitoring/synthetic-monitor/staged.mjs.

Pass → promote deployment to production domains, wait until www serves it, run the wider suite on www.

Fail → email “deploy held back” with failing check names; www unchanged.

Step 3: Newer deploy always wins

An older build that finishes checking late never overwrites a newer one. Scheduled monitor runs also alarm if a build waited unpublished > 30 minutes (release pipeline stuck).

Manual promote (escape hatch)

node scripts/promote-deploy.mjs              # list recent production builds
node scripts/promote-deploy.mjs <dpl_id>     # publish one (needs VERCEL_TOKEN)

Use only when automation is stuck — the happy path is push → checks → self-publish.

What this does not replace

  • Local Cypress for long mocked flows — still before big UI changes.
  • Operator setup — webhook secret, VERCEL_TOKEN, bypass secret, SNS email for hold alerts (see internal runbooks; no secrets in this post).

Why write this up

If you run a solo SaaS on Vercel + Supabase, the cheap win is not faster rollback — it is never assigning broken HTML/API to your apex domain. Staged promote + real write-path smoke beats a green unit test suite that never touched auth or Postgres.

Cowpin product tie-in

We ship bookmark memory, Pinboard API compat, and MCP for agents. That only matters if production stays up — the staged pipeline is how we keep import and API promises honest while shipping several times a day.