Apigee · Traffic Management

Spike Arrest:
how the bucket actually works

What a 429 means and how clients should react to one — then the mechanism that produces it: rate normalization, the derived burst capacity, and a play-by-play of the token bucket under a real spike.

There is no <MaxBurst> element in Apigee. Burst capacity is derived from your rate — which is exactly why 30ps and 1800pm are not the same policy, despite being the same throughput.

← → or SPACE to navigate  ·  simulations auto-play, or step them manually

429 Too Many Requests

before the mechanism — what the client is actually being told
What it means
  • "Not right now" — not "your request is wrong"
  • The request never reached the backend. Nothing ran, nothing changed
  • Retryable, unlike 400 / 401 / 403 — those fail identically forever
  • The server is protecting itself: reject 10% cheaply, or collapse under 100%
  • May carry Retry-After — seconds, or an HTTP date
  • The window is often sub-second. A short wait is usually enough
  • The very next request from the same client may well succeed
What it does not mean
  • Not a bug in your payload, auth, or URL
  • Not permanent — the rate window keeps moving
  • Not an invitation to retry instantly (that's what causes the next one)
  • Not a reason to surface a hard failure to the user — a correct retry usually succeeds
  • Not necessarily an incident. A steady trickle of 429s means the limiter is working
  • Not rare. At scale, 429s are normal. Designing as if they never happen is the actual bug

A 429 is a signal, not a failure. It tells you the server has a rate model and you're currently outside it. The rest of this deck is what that model looks like inside Apigee — and why the same "requests per second" can produce wildly different 429 behaviour.

Handling 429s: the client's job

retry · backoff · jitter
1 · Retry
  • Honor Retry-After first — it beats any formula you invent
  • Cap attempts (~5) and total elapsed time
  • Safe to retry reads; for writes use an idempotency key
  • Never retry immediately — the bucket is still empty
2 · Backoff
  • Exponential: 100 ms → 200 → 400 → 800 → 1600
  • Cap the delay (~30 s) so it can't grow unbounded
  • Backoff gives the bucket time to refill — that's the whole mechanism
  • Fixed-interval retry just re-clumps your traffic
3 · Jitter
  • Without it, every rejected client retries in lockstep
  • N clients 429'd at once → N retries at once → 429 again
  • Full jitter spreads them across the window
  • This is the single highest-value line of code here
FULL JITTER
delay = random( 0, min(cap, base × 2n) )

Randomising the full range beats adding a small random nudge to a fixed delay — it decorrelates clients fastest and empirically minimises total work under contention.

Beyond retrying Rate-limit yourself — don't send what you know will be rejected Circuit-break after repeated 429s; pause, then probe Queue and smooth your own bursts before they leave the client
Step 1 of 3

The rate is normalized into an interval

Apigee never thinks in "per minute". It converts your rate into a drip interval — one token every interval.

Rate = 30ps
1 000 ms  ÷  30
33.3ms per request
Rate = 1800pm
60 000 ms  ÷  1800
33.3ms per request

both produce the identical drip →

Identical. Same interval, same 30 requests/second average. If the interval were the whole story, 30ps and 1800pm would be interchangeable. They are not — and the next slide is why.

Step 2 of 3

The bucket capacity is 10% of the rate

"The bucket size is always configured to be 10% of the messagesPerPeriod."

— Apigee SpikeArrest policy reference

3
30ps
10% of 30
×60
180
1800pm
10% of 1800
RatemessagesPerPeriodIntervalAverageBucket
30ps30 per 1 s 33.3 ms30/s3
1800pm1800 per 60 s 33.3 ms30/s180

Capacity is derived from messagesPerPeriodnot from the rate you mean. Express 30/s per second and that number is 30. Express it per minute and it's 1800. Same throughput. Sixty times the burst tolerance.

Step 3 of 3

The whole algorithm is two rules

1

Tokens drip in, continuously

One token every interval — for both our configs, one every 33.3 ms (30/second). The bucket fills whether or not anyone is calling.

tokens = min( capacity, tokens + Δt × 30 )

The min() is the point: idle time buys burst capacity, but only up to the cap. Idle an hour and 30ps still gives you 3.

2

Each request takes a token

Enough in the bucket? It passes. Not enough? 429 — immediately.

if tokens ≥ 1: tokens −= 1  → 200
else: tokens unchanged  → 429

A rejected request consumes nothing and is never buffered or retried server-side. It is simply gone.

Spike Arrest sheds load.
It never delays it.

There is no queue, no waiting room, no smoothing buffer. Every request is decided the instant it arrives — pass or 429. Which is why the client's backoff is the only thing that recovers the traffic.

Play-by-play: 1800pm under a spike

capacity 180 · 1 token / 33.3 ms · slow-mo during the action
Act 1 183 pass / 57 × 429

240 requests at once. The full bucket empties in 120 ms — everything after that is shed.

Act 2 30 pass / 0 × 429

A polite client: one request every 33.3 ms — exactly the drip. Level holds flat. Steady state.

Act 3 88 pass / 62 × 429

2 s lull, then 150 at once. The lull refilled only ~61 tokens. Burst tolerance regrows at the drip rate.

▶ Play
Step ▸
↺ Reset
SPEED
0.5×
t = 0.00 s · tokens 180.0

30ps vs 1800pm — same traffic, same second

60 requests arriving 1 ms apart · both buckets start full
30ps
capacity 3
drip 33.3 ms
1800pm
capacity 180
drip 33.3 ms
▶ Fire the same 60 requests at both
↺ Reset
30ps: — · 1800pm: —

Both policies are "30 requests per second." Identical traffic in. 1800pm absorbs the entire spike without a single 429. 30ps passes 4 of 60 — its bucket of 3, plus the one token that dripped in during the burst. Nothing in the config hints at this; the difference lives entirely in the units.

pm buys you 60× the burst

for any throughput
Rate = Rps
0.1 R
×60 →
Rate = (60R)pm
6 R
Throughputas psbucketas pmbucket
10/s10ps1600pm60
30/s30ps31800pm180
100/s100ps106000pm600

Pick ps for hard smoothing — a fragile backend that must never see a clump.

Pick pm when real clients arrive in clumps — apps waking up, cron jobs, a page firing six calls at once.

With no <MaxBurst>, the units of <Rate> are your only burst control — and they move it in 60× steps. At 30 req/s there is nothing between a bucket of 3 and a bucket of 180.

One bucket per client

<Identifier ref="client_id"/>

Each distinct client_id gets its own independent token bucket. One app's spike stays that app's problem.

<SpikeArrest name="SA-Protect">
  <Rate>1800pm</Rate>
  <Identifier ref="client_id"/>
</SpikeArrest>
  • The rate is now per client, not per proxy
  • A misbehaving app can't 429 your well-behaved partners
  • 429s become diagnostic — they name the client that caused them

Independent buckets

app-A
spiking · 429s
app-B
unaffected
app-C
unaffected

Capacity planning: your worst case is no longer Rate — it's Rate × active clients. Size for one caller, then confirm the backend survives N of them bursting at once.

Tuning it, in practice

Do

  • Set the rate near your measured backend capacity
  • Choose the units deliberatelypm vs ps is your only burst control
  • Always set <Identifier ref="client_id"/>
  • Put the policy early in the PreFlow, before any expensive step
  • Return Retry-After on the 429 so clients can back off correctly

Don't

  • Don't treat 30ps and 1800pm as interchangeable — 60× apart
  • Don't expect exactly N requests per window — expect smoothing plus a burst
  • Don't let clients retry immediately; the bucket refills only at the drip rate
  • Don't expect requests to be queued — they're shed, not delayed
  • Don't look for <MaxBurst>. It does not exist.

Sizing shortcut. To pass a spike of B straight through, set Rate = (B × 10)pm — which simultaneously pins your average at B/6 per second. The two are not independently selectable.

And the other half of the job is on the client. A correctly sized bucket still produces 429s during a spike — exponential backoff with full jitter is what turns those into successful retries instead of a second spike.

Apigee · Spike Arrest