pr walkthrough

Per-token rate limits

Moves rate limiting from per-IP to per-token, so one office NAT can't exhaust the budget for every client behind it. Four files, one new bucket table, no behavior change for untokened requests.

Shape of the change

gateway/
  limits.py  +38 -9
  middleware.py  +2 -1
  ratelimit.conf  +1 -1
  migrations/
    0042_token_buckets.sql  +6

The interesting part is gateway/limits.py:41 on GitHub: the bucket key changes from the remote address to the token id when one is present. Everything else is plumbing that key through.

The ceilings themselves

None of these numbers move. What moves is what they are counted against: a plan's ceiling used to be spent by every client behind one address, and is now spent per token.

Ceilings per plan, and what each is counted against after this change.
Plan	A minute	Burst	Counted against
Free	60	120	the token
Team	600	1,200	the token
Enterprise	6,000	12,000	the token, per environment
Untokened	60	60	the remote address, as before

The reviewed rate-limit configuration

This is the exact branch snapshot the walkthrough was checked against, not a second rendering of the diff.

The core diff

diff --git a/gateway/limits.py b/gateway/limits.py
--- a/gateway/limits.py
+++ b/gateway/limits.py
@@ -38,6 +38,8 @@ class Limiter:
-    def bucket_key(self, request):
-        return request.remote_addr
+    def bucket_key(self, request):
+        if request.token:
+            return f"tok:{request.token.id}"
+        return f"ip:{request.remote_addr}"
 
     def allow(self, request):
         key = self.bucket_key(request)
         return self.buckets[key].take()
diff --git a/gateway/middleware.py b/gateway/middleware.py
--- a/gateway/middleware.py
+++ b/gateway/middleware.py
@@ -12,3 +12,4 @@ def attach(app):
     def wrap(request):
-        limiter.check(request.remote_addr)
+        limiter.check(request)
+        request.limit_key = limiter.bucket_key(request)
         return request
diff --git a/gateway/ratelimit.conf b/gateway/ratelimit.conf
--- a/gateway/ratelimit.conf
+++ b/gateway/ratelimit.conf
@@ -1,2 +1,2 @@
 burst_multiplier = 2
-bucket_ttl_minutes = 10
+bucket_ttl_minutes = 30
\ No newline at end of file
diff --git a/gateway/migrations/0042_token_buckets.sql b/gateway/migrations/0042_token_buckets.sql
--- /dev/null
+++ b/gateway/migrations/0042_token_buckets.sql
@@ -0,0 +1,6 @@
+create table token_buckets (
+  key text primary key,
+  tokens integer not null,
+  refilled_at timestamptz not null
+);
+create index token_buckets_refilled_at_cleanup_idx on token_buckets (refilled_at) where tokens = 0;

Prefixes keep token and address buckets disjoint

def bucket_key(self, request):
    if request.token:
        return f"tok:{request.token.id}"
    return f"ip:{request.remote_addr}"

Unprefixed, a token id that happens to look like an IP would share a bucket with that address. The prefixes make the two namespaces disjoint.

Testing

What each half of the change proves, and the test that holds it.
Suite	What it proves	Held by
Unit	The bucket key for tokened, untokened, and both-present requests.	`test_the_bucket_key_prefers_the_token_and_falls_back_to_the_address`
Integration	Two tokens behind one address each get a full budget; two addresses on one token share one.	`test_two_tokens_behind_one_address_each_get_a_full_budget`, `test_two_addresses_on_one_token_share_one_budget`

Both suites run under the rate-limit marker:

# the integration half needs the migration applied first
cd gateway && alembic upgrade head
pytest tests/ -m ratelimit --maxfail=1

The two-token case's transcript, from the branch as pushed:

$ pytest tests/ratelimit/test_bucket_identity.py::test_two_tokens_behind_one_address_each_get_a_full_budget -q
..                                                                 [100%]
2 passed in 1.84s