Web push looks like the easiest channel you can build. There's no telco in the path, no Meta approval queue, no carrier throttling you've never heard of. It's an HTTPS request to an endpoint the browser handed you. That apparent simplicity is exactly why it scales so treacherously: the hard parts are invisible until the volume finds them for you.
Between 2018 and 2021 I helped take one web push pipeline from 1.3 billion notifications a month to 24 billion — roughly 18x in three years. We did it with a small team: two to five engineers and one to three QA at any given time, the numbers drifting with the roadmap. This is what actually broke on the way up, and what we built to get past it.
The baseline: a polyglot pipeline
The system that had to grow wasn't one language or one datastore. It was polyglot on purpose, because different parts of a send pipeline have genuinely different shapes.
The services ran across PHP, Node.js, and Go, with the workers built as an asynchronous, multi-cluster fleet — fan-out is embarrassingly parallel, and the architecture leaned into that. Storage was spread by access pattern rather than forced into one engine: Aurora and MySQL for relational data, DynamoDB where we needed key-value throughput that didn't flinch, and Elasticsearch for the query shapes the others were bad at. The queueing layer was the same story — SQS, Kinesis, NSQ, and Redis lists each did the job they were best at rather than one broker pretending to be all of them.
None of that is exotic. The point is that "the stack" was never a single decision; it was a set of local decisions about where each kind of load belonged. At 1.3B that felt like over-engineering. At 24B it was the only reason the thing stayed standing.
The first wall: providers don't publish the limits that matter
The first hard ceiling wasn't ours — it was the provider's, and it wasn't in the docs.
Early on we were on GCM, and scaling into it meant hitting a stack of limits nobody advertises together: Google API quotas, AWS Lambda concurrency limits, SQS throughput ceilings. Each one looked like a different problem and every one of them was the same problem wearing a different badge — the pipeline was faster than something upstream was willing to accept. We spent a real part of 2018–2021 finding those ceilings and engineering underneath them.
Then came the migration to FCM, and with it a different class of challenge: pushing toward roughly ten million sends per second. At that rate, "just call the API" stops being a strategy. Everything about how you batch, how you hold connections, and how you absorb the provider's own variability becomes the actual product. We got there, but the migration also handed me the most memorable bug of the whole era — more on that below.
The queue rewrite: no tenant waits for another
The failure mode that scales worst is one big customer starving everyone else. If a single tenant's campaign can monopolize the pipeline, then your p99 isn't a number about your system — it's a number about your noisiest customer's Tuesday.
So the queue architecture was rebuilt around tenant parallelism. Instead of one shared path where a huge send blocks the queue behind it, the redesign ran all tenants concurrently across multiple queues, with no tenant waiting on another. The organizing principle was simple to say and hard to hold: at any instant t, push as much concurrent work through as the providers and the workers can absorb, and never let one campaign's size become another campaign's latency. Fairness stopped being a nice-to-have and became the thing the topology was shaped around.
Retries under peak: budgets you can move at runtime
At 24B a month, retries are not a detail — they're a multiplier. A naive retry policy turns a provider hiccup into a self-inflicted traffic spike, and you take yourself down faster than the provider ever would have.
We ran retries through the queue with a circuit breaker: allow up to X attempts within a window, then trip and stop hammering something that's already hurting. The part that mattered most was that these numbers weren't static. Retry budgets and breaker thresholds were dynamic, and during peak events — Black Friday and the live-traffic seasons — they were overridden on the fly to match the day's reality. A threshold that's correct on an average Tuesday is wrong on the biggest sending day of the year, and the system had to let us say so at runtime.
The war story: a caching bug inside the vendor's own library
Every scaling era leaves you with one bug you'll retell for years. Mine was the FCM migration, in 2021.
We started seeing errors and, reasonably, trusted Google's native FCM Node package to be the stable thing in the stack — it's the vendor's own client, after all. It wasn't the stable thing. There was a caching problem inside the package itself: under our conditions the FCM key wasn't being refreshed the way it should have been. We eventually cornered it by hand, resetting the FCM key variable manually to force the state the library wouldn't reach on its own. An official fix landed in the package later — but by then we'd already learned the lesson the hard way: at scale, "it's the vendor's official library" is a hypothesis, not a guarantee. The most expensive bugs live in the layer you decided you didn't have to check.
Where 24B actually sits
Here's the honest shape of the story, because the number gets quoted without its timeline. 24 billion a month was a peak we reached in 2021, not a plateau I've been sitting on since. We hit it with that small team, and then two things happened: I left web push to build WhatsApp and SMS as products from zero, and Google subsequently dialed web push back as a channel. The 24B is a real high-water mark for what that pipeline did — it just isn't a present-tense throughput number, and I'd rather say that plainly than let a big figure imply something it doesn't.
What it comes down to
Web push punishes you for believing it's simple. The lesson that survived from 1.3B to 24B isn't a specific queue or a specific datastore — it's that at every order of magnitude the bottleneck moves somewhere you weren't looking: into a provider's undocumented quota, into one tenant's campaign, into a retry policy that was fine yesterday, into a vendor library you trusted because trusting it was convenient. Scaling wasn't one big rewrite. It was refusing to assume any layer was safe just because it hadn't broken yet.
Salimcan Venedik is a Staff Software Engineer and Technical Lead at Insider, where he works on messaging infrastructure and AI product engineering. — About / Contact