examples gallery

The leaf gallery, on one page

The shipped examples, one per tab, every widget live: boards drag, options take a pick, diagrams render. Every project, name, and number here is invented. Switch tabs freely (that's your view alone); select any text to comment on the example it belongs to.

Design decisionIncident reportPR walkthroughStatus reportLive progressTriage boardCommand hubRelease notesParallel workstreams

Where sessions live

The monolith split leaves session state homeless: today it rides the app server's memory and dies with it. Three candidates, one recommendation — Redis with a signed-cookie fallback. Click an option — or Tab to its mark and press Enter — to decide; your pick reaches the agent directly.

Constraints

  • Logout must revoke immediately — support runs "log out all devices" during account-takeover response.
  • Session writes happen on every request (rolling expiry), so the store sees full request volume.
  • The team already operates Redis for rate limiting; nobody runs Dynamo-style infrastructure today.

Settled last week

How the session id travels was decided before this page and the client change is merged, so it reads as one line — open it if you want the alternatives.

effort: lowrisk: low Host-only cookie, SameSite=Lax Set by the auth origin, sent on top-level navigations; nothing to store client-side and nothing for a script to read. chosen effort: lowrisk: med SameSite=Strict Tighter, but a session started by following a link from email arrives logged out — support saw this on the last trial. choose one effort: highrisk: med Bearer header from local storage Works for the mobile client without a cookie jar; puts the id somewhere every script on the page can read. choose one
Send

How payments were asked

Payments settled the same question in March. Their page put it in two cards with the operating cost on the front of each, which is a framing worth copying. Their answer travels less well, since they had a tenth of our write volume and nothing that had to survive the store going down.

payments, March already on call for it Redis Revocation is a delete, and the idempotency keys live there already. ledger primary Postgres table One table, but every request writes to the instance the ledger is on. Payments, March
Writes
a tenth of ours
Store down
queue and retry

A payment is seconds long, so a Redis blip is a spinner and a retry, and nothing about their answer had to outlive the store.

Sessions, today
Writes
every request
Store down
readers stay signed in

Rolling expiry writes on every request, and an outage must not sign the fleet out — the constraint their page never had to argue.

Options

effort: lowrisk: high Stateless JWT
Revoke
build a denylist
Outage
unaffected
New ops
none, at first

Sessions become signed tokens; no store at all. Revocation requires a denylist, which quietly reintroduces the store — with the hard parts (replication, expiry) still attached, and the account-takeover response waiting on them.

choose one
effort: medrisk: low Redis, cookie fallback
Revoke
delete the key
Outage
reads ride the cookie
New ops
what we already do

Sessions in the Redis we already run, keyed by an opaque id; a short-lived signed cookie covers Redis outages for reads, so a blip doesn't log everyone out. Revocation is a delete — which is the shape support's "log out all devices" needs.

hit

miss

Redis

handle

cookie

choose one
effort: lowrisk: med Postgres table
Revoke
delete the row
Outage
shares the database's fate
New ops
vacuum pressure

One table, no new moving parts. Every request writes a row (rolling expiry), so at our volume that's the primary's headroom spent on expiry bookkeeping — and a session outage becomes a database outage.

choose one
Send

The two cookies

revocablenothing readable in itSession cookie An opaque id; the store says everything else. outlives a revocationsurvives a Redis blipFallback cookie A signed snapshot; outlives a blip, never a revocation.

One revision is waiting on you below: accept or reject it with the controls in the margin, and the next version carries whichever you chose.

What I pick up next

Whichever store wins, three jobs stand behind it, and each is argued somewhere above rather than in a card of its own. Pick the ones worth starting — click a row, or Tab to its mark and press Enter — and the box takes anything the rows don't cover, including "none of these".

A revocation drill at support's volume§ constraintschoose any Key rotation for the fallback cookie§ flowchoose any A write load test at request rate§ optionschoose any
Send

Export queue backlog, 12 June

A retry storm from one malformed workspace held the export queue for 94 minutes. No exports were lost; the oldest was delayed 81 minutes.

Detected
09:14, queue-depth alert
Resolved
10:48, poison job quarantined
Blast radius
exports only; imports and sync unaffected
1,204exports delayed+1,204 81 minoldest delay 0exports lost

Timeline

09:14 Queue-depth alert fires Depth crosses 500; normal peak is 60. 09:26 On-call confirms a single hot job The same export id retrying at the head of the queue, every 45 seconds. 09:51 First fix doesn't hold Raising worker count clears depth briefly; the retry storm refills it because the head job still fails first. 10:32 Poison job quarantined The workspace's export is parked in a dead-letter table; the queue drains at normal rate. 10:48 Backlog cleared Depth back under 60; delayed exports delivered.

Root cause

The export worker treats any failure as retryable. One workspace carried an attachment with a declared size of −1, which fails serialization every time; with retries capped by attempt count but not by queue position, the job returned to the head on each attempt and starved everything behind it.

serialize fails

starved

queue head

worker

retry in 45s

1,204 jobs behind

Follow-ups

  1. Classify worker failures as permanent vs. retryable; dead-letter permanents on first failure.
  2. Re-enqueue retries at the tail.
  3. Alert on a single job id exceeding five attempts, not only on queue depth.

Per-token rate limits

Moves rate limiting from per-IP to per-token, so one office NAT can't exhaust the budget for every client behind it. Three files, one new bucket table, no behavior change for untokened requests.

Shape of the change

  • gateway/
    • limits.py+38-9
    • middleware.py+6-2
    • migrations/
      • 0042_token_buckets.sql+11

The interesting part is gateway/limits.py:41: the bucket key changes from the remote address to the token id when one is present. Everything else is plumbing that key through.

The ceilings themselves

None of these numbers move. What moves is what they are counted against: a plan's ceiling used to be spent by every client behind one address, and is now spent per token.

Ceilings per plan, and what each is counted against after this change.
Plan A minute Burst Counted against
Free 60 120 the token
Team 600 1,200 the token
Enterprise 6,000 12,000 the token, per environment
Untokened 60 60 the remote address, as before

The core diff

Why the prefixes matter

def bucket_key(self, request):
    if request.token:
        return f"tok:{request.token.id}"
Unprefixed, a token id that happens to look like an IP would share a bucket with that address. The prefixes make the two namespaces disjoint.
return f"ip:{request.remote_addr}"

Testing

  • Unit: bucket key for tokened, untokened, and both-present requests.
  • Integration: two tokens behind one address each get a full budget; two addresses on one token share one.

Both suites run under the rate-limit marker:

# the integration half needs the migration applied first
cd gateway && alembic upgrade head
pytest tests/ -m ratelimit --maxfail=1

Search relaunch, week 6

Indexing is done and dark-launched; relevance is the open question. The shadow comparison puts the new engine ahead on head queries and behind on long-tail, which is the expected trade at this stage.

118 msp95 latency-64 0.71ndcg@10, head+0.05 0.58ndcg@10, tail-0.03 99.4%index coverage

Milestones

Reindex on the new schema
weeks 1-4
Full corpus, verified against the old index by document count and spot checks.
Shadow traffic comparison
weeks 5-7relevance
Every production query runs on both engines; judgments collected on the diffs.
Long-tail recovery
week 7relevancedata
Blocked on the synonym table export from the old engine — owner found, ETA Tuesday.
Cutover behind a flag
week 9
5% of traffic, then doubling daily if guardrails hold.

The long-tail gap

The old engine's synonym table carries fifteen years of manual curation; the new engine currently ships without it. On tail queries where a synonym was the match, we lose the document entirely, which is most of the −0.03.

Evidence: 61% of tail regressions disappear with the table patched in

A one-hour experiment loading the March export into the new engine's synonym slot recovered 214 of 349 regressed queries. The remainder split between stemming differences and genuine ranking changes.

Next week

  1. Import the synonym table export when it lands Tuesday.
  2. Re-run the shadow comparison on the tail set.
  3. Draft the cutover guardrails (error rate, zero-result rate, p95).

Checkout cutover rehearsal

The agent republishes this page as each check finishes. The browser is following the newest version, so statuses and counts change without a refresh. The rollback drill is running now.

14 of 18checks complete+3 2running now 1blocked 31 minelapsed

Right now

Work

Capture the baseline
14:02
Error rate, p95, order count, and queue depth recorded before the rehearsal.
Shadow production traffic
14:08
New and old services agreed on 2,000 sampled checkouts.
Cut traffic to the new service
14:19
Guardrails held for ten minutes at 100% traffic.
Prove rollback
runningrollback
Old service is live again; order-count comparison is still running.
Publish the rehearsal report
next
Attach timings and name the one blocked follow-up.

Blocked

Finance export reconciliation needs a fixture with a partially refunded order. The rest of the rehearsal does not depend on it; the agent left the check open and moved on.

Latest events

14:19 Cutover complete New checkout took 100% of traffic. 14:29 Guardrails held Error rate 0.08%; p95 181 ms. 14:31 Rollback started Traffic returned to the old service in 42 seconds.

Release triage

Everything open against the v2.4 release. Drag cards to re-triage — or Tab to a grip and press Enter, then arrows: each move reaches the agent as an action, and the next version of this page ships with the board as you left it.

Migration reruns on every deploy The version stamp never lands, so 0041 replays; idempotent today, not once 0042 ships. Logout 500s with an expired session Double-delete on the session key; needs a guard, one line. Digest email uses server timezone Cosmetic for most, wrong-day for UTC+13. Search ignores archived items toggle Filter dropped in the query builder rewrite. CSV export quotes numerics Breaks one customer's downstream import. Layout breaks on IE mode Out of support matrix; documented instead.

Triage notes

The migration stamp bug is the only one with data risk; it blocks the release even if the fix slips a day. The logout fix is trivial but touches the auth path, so it rides the same release train rather than a hotfix.

Atlas importer rewrite

The standing view of the rewrite: what each agent holds, where every task stands, and the decisions waiting on you. The orchestrator republishes as work moves, so the page keeps up on its own — what it needs from you is the one section below the numbers.

4 of 10tasks done+2 3agents live 2decisions waiting 0reverts this week

Needs you

Two items, one decision each. The tree's ambers are the milder tier — work finished and readable whenever you sit down; they queue, they don't interrupt. Everything else on the page is news, not a request.

Land the schema migration?

The migration adds the two nullable columns the reconciliation work writes into. It touches schema/atlas.sql, which is in the overlap zone, so it holds for your word rather than landing on green tests — file-disjoint tasks still meet in this schema, which is why it is listed. Landing it unblocks the variance report; holding it keeps main untouched but idles finch after today. Click an option to decide.

Land it nowchoose one Hold until the backfill is writtenchoose one Walk me through it first§ t-schemachoose one
Send

The reconciliation fixture is yours

Reconcile ledger exports is blocked on a fixture with a partially refunded order, which only you can pull from the production replica. One command, pasted in a comment here, unblocks it.

Work

The plan as it stands, counted by its leaves. Amber means finished by its agent and waiting on your review; red is stuck, and says on what.

Groundwork
weeks 1-23/3 done
Snapshot the import corpus
wren
Replay harness
junco
Old and new importer over the same corpus, diffed.
Schema draft
finch
Replace the XML parser
wrenweek 31/3 done
The streaming parser is in; the two format edge cases remain. Streaming core
wren
Constant-memory parse of the 4 GB nightly export.
CDATA edge cases
wrenatlas-cdata
Diff is small and green; awaiting your read.
Encoding declarations Starts once CDATA lands.
Schema migration
finchatlas-schema
Written and tested; gated above because it touches the overlap zone.
Ledger reconciliation
juncoweek 3-40/2 done
Reconcile ledger exports
junco
Needs the fixture named above.
Nightly variance report Needs the schema migration landed.
Cut over the nightly job
week 5
Shadow first, then swap.

Agents

Where each agent is working, and its last report's age — read from the log, never from the agent.
Agent Task Branch Last report
wren CDATA edge cases atlas-cdata 12 min ago
finch Schema migration atlas-schema 1 h ago
junco Ledger reconciliation atlas-ledger 3 h ago

Feed

thu 14:12 CDATA branch went green wren moved the task to review. thu 13:40 Schema migration held at the gate Overlap-zone file; waiting on you above. thu 11:03 Streaming core merged Serialized onto main after the merge-slot test run.

Standing policy

Tasks are scoped not to share files, branch from the same base commit, and merge one at a time after a test run against current main. Mechanical changes land on green; anything touching schema/, auth/, or a public contract holds for your word, however green the tests. The grant is yours, recorded below — click to tighten or revoke it any time — and any landed change is a revert away for as long as you care to look.

Two-tier gates as stated chosen Everything waits for my wordchoose one
Send

Release notes, drafted

Four notes cover everything user-visible in 3.2. Each is a draft you own: double-click one (or its ✎) and rewrite it until it reads right — the exact wording reaches me, and the next version of this page carries it. The framing lines are mine, and one of them is a rewrite I'm proposing: accept or reject it in the margin, or select it and comment if it needs discussing first. When the set reads well, sign off; if a note shouldn't ship at all, say so in a comment.

CLI

Adds --dry-run to every mutating command. Running without it is unchanged, so existing scripts keep working.

API

Kept deliberately dry — the Sunset header does the announcing, and the migration guide carries the detail.

Deprecates the v1 auth header. It keeps working until the next major, and the server now sets a Sunset header on every v1 response.

Console

The run list grew a status column, so a failure now reads off the list instead of costing a click each. It is the release's only visual change.

before: the run list, before and after the status columnbefore
after: the run list, before and after the status columnafter
Run status now shows on the run list itself. Filtering by it lands in 3.3.

Operations

The minute figure is from staging (1.1M rows) — rounded rather than promised.

The minute figure is from staging (1.1M rows) on a warm cache — rounded rather than promised. If operators will book a window around it, the note should give a range instead.

Run the schema migration before deploying. It is online and takes about a minute on a million rows.

Left out on purpose

The dependency bumps and the flaky-test fixes stay in the changelog only — nothing a user would change behavior over. If one of them deserves a note after all, say so in a comment and a draft for it arrives in the next version.

Aviary projects, week 3

Three mini-projects run side by side; each tab below is one project's full context — status, board, decisions. Which tab is open is your view alone: switching isn't a comment and doesn't reach the agent.

FeedersBird bathCamera
2 of 4feeders mounted+1 312daily visits+41 2.1/wkrefills+0.4

The south pair is up and drawing traffic; the north pair waits on brackets. Drag cards to reprioritize — your edits reach the agent directly.

Steel brackets Order for the north pair. Squirrel baffle Heated perch Wire the south feeder. South mounts

The bracket order goes in on Friday and there is room in it. These are the extras the south pair turned out to want.

£9 the pairmore cleaning Seed tray Catches the spill under the south pair, where the grass has thinned to bare earth. choose any £15 eachfits both pairs Weather dome Keeps the seed dry; the south feeders clogged twice in the wet week. choose any £24 fitteduses the spare camera Second camera mount Puts the spare on the north post once the feeders are up. choose any
Send

The basin is level and holding water; the open question is how to keep it liquid through January. Click an option to decide.

£18a cord across the lawnliquid all winter Immersion heater Drops into the basin; needs a cord run across the lawn. choose one £70swap the basinliquid but the hardest frosts Solar basin Replace the basin with the heated model; no cord, reversible. choose one £0nothing to installliquid on thaw days Let it freeze Refill on thaw days only. choose one
Send

The feed has been stable since the battery swap; one open follow-up on storage.

Mon 09:14 Camera offline Battery alert from the north post. Mon 09:40 Battery swapped Mon 09:41 Feed resumed Stable since.

Next: rotate clips to the shed NAS before the card fills, likely week 4.

✓ Accept✗ Reject✓ Accept✗ Reject