Scaling from 10 users to 10 million — the decisions that matter at each stage

The most expensive engineering mistake a startup can make is solving a scaling problem it doesn't have yet. The second most expensive is ignoring a scaling problem until it's an outage.

Scaling isn't one problem. It's a sequence of different problems that appear at different stages, require different solutions, and — critically — require different solutions at different times. The architecture that's right for 10 users is wrong for 10 million. The architecture that's right for 10 million is catastrophically over-engineered for 10 users.

This is our map of what actually matters at each stage, what the real bottlenecks are, and the decisions that are worth making early versus the ones that are worth deferring.

Stage 1: 10 to 1,000 users — ship and learn

At this stage, your scaling problem is not a scaling problem. It's a product problem. The question isn't "can this handle load?" — it's "is this the right thing to build?"

What the architecture should optimise for: speed of iteration. The ability to change the data model, the API shape, the business logic, the user flow — quickly, without ceremony. Every hour spent on infrastructure at this stage is an hour not spent learning whether the product is right.

What's actually fine at this stage:

• A single server. One Heroku dyno, one Railway instance, one EC2 t3.small. It will handle thousands of users without breaking a sweat.
• A single database. Postgres on the same server, or a managed Postgres instance. No read replicas, no sharding, no caching layer.
• Synchronous everything. No message queues, no background job infrastructure, no event bus. If something takes 200ms, it takes 200ms.
• No CDN for dynamic content. A CDN for static assets (images, JS, CSS) is worth it from day one. A CDN for API responses is not.

What's worth doing from day one:

• Indexes on the columns you query. The single most impactful performance decision at any stage. An unindexed query on a 10,000-row table is fast. On a 10-million-row table, it's an outage. Add indexes when you add the query, not when the table is already large.
• Structured logging. Not because you need it now, but because retrofitting it later is painful. Log request IDs, user IDs, operation names. When something goes wrong at 100,000 users, you'll want to be able to trace a request.
• A deployment pipeline. Even a simple one. The ability to deploy in under 5 minutes, with a rollback path, is worth more than any infrastructure optimisation at this stage.

What to ignore: horizontal scaling, caching layers, read replicas, message queues, microservices, CDN for API responses, database connection pooling (until you need it).

Stage 2: 1,000 to 50,000 users — the first real bottlenecks

At this stage, you have real users, real load, and the first real bottlenecks. The bottlenecks are almost always in the same places.

The database is usually the first bottleneck. Not because Postgres can't handle the load — it can handle far more than most people think — but because the queries that were fine at 1,000 rows are slow at 1,000,000 rows, and the indexes that were added for the obvious queries weren't added for the queries that turned out to be common.

The fix is almost never "add a read replica" or "switch to a different database." It's almost always "add the missing index" or "rewrite the slow query." Run EXPLAIN ANALYZE on your slow queries before you add infrastructure.

Background jobs become necessary. Sending emails synchronously in a request handler was fine at 100 users. At 10,000 users, a slow email provider causes request timeouts. Move anything that doesn't need to be synchronous — emails, notifications, report generation, webhook delivery — to a background job queue. Redis + a job queue library (Sidekiq, BullMQ, Celery) is the standard pattern.

Caching becomes worth it for specific things. Not everything — caching adds complexity and cache invalidation is genuinely hard. But for data that's expensive to compute and changes infrequently — dashboard aggregates, recommendation lists, configuration data — a cache layer pays off. Start with in-process caching (a simple Map with a TTL) before reaching for Redis.

What to add at this stage:

• Database indexes (if you haven't already — and you probably haven't for all the right columns)
• Background job queue for async work
• Application-level caching for expensive, infrequently-changing data
• A proper APM tool (Datadog, New Relic, or even just Sentry's performance monitoring) — you need to see where the time is going

What to still ignore: read replicas (unless your read/write ratio is genuinely extreme), microservices, database sharding, a CDN for API responses.

Stage 3: 50,000 to 500,000 users — horizontal scaling and the database ceiling

At this stage, a single application server is probably the bottleneck, and the database is approaching its limits on write throughput or connection count.

Horizontal scaling for the application tier. Add more application servers behind a load balancer. This is straightforward if your application is stateless — if session state lives in the database or Redis, not in memory. If your application has in-memory state, you need to fix that before you can scale horizontally.

The practical step: make sure sessions are stored in Redis or the database, not in the application process. Make sure any in-memory caches are either acceptable to be per-instance (each instance has its own cache, slightly inconsistent) or moved to Redis.

Read replicas for the database. If your read/write ratio is high (most web apps are 80–90% reads), a read replica takes significant load off the primary. Route read queries to the replica, write queries to the primary. Most ORMs support this with a configuration change.

Connection pooling. At this scale, the number of database connections from multiple application servers can exceed the database's connection limit. PgBouncer (for Postgres) or a managed connection pooler solves this. Add it before you hit the limit, not after.

CDN for static assets and cacheable responses. If you have pages or API responses that are the same for all users (or all users in a segment), a CDN can serve them without hitting your application servers at all. This is high-leverage for content-heavy products.

What to add at this stage:

• Load balancer + multiple application servers (stateless)
• Database read replica
• Connection pooler (PgBouncer or managed equivalent)
• CDN for static assets and cacheable responses
• Redis for sessions and shared caching

What to still defer: database sharding, microservices (unless you have a specific team-ownership or independent-scaling problem), event sourcing.

Stage 4: 500,000 to 10 million users — the hard problems

At this stage, the easy wins are gone. The bottlenecks are in the places that require real architectural changes.

Database write throughput. A single Postgres primary can handle a lot of writes — but not unlimited writes. The solutions, in order of complexity:

1. Optimise the writes. Batch inserts instead of individual inserts. Async writes for non-critical data. Denormalise where the join cost is high.
2. Vertical scaling. A larger database instance handles more writes. This is often the right answer before sharding.
3. Functional partitioning. Split the database by domain — billing on one database, orders on another, users on a third. This is the modular monolith / bounded context idea applied to the data layer.
4. Sharding. Horizontal partitioning of a single table across multiple database instances. The most complex option, with real operational overhead. Apply it when functional partitioning isn't enough.

Hot spots. At scale, certain rows get hit far more than others — a popular product, a viral post, a high-traffic user. These hot spots can saturate a single database row's lock contention. Solutions: application-level caching for the hot data, counter sharding for high-write counters, read-through caches.

Async-first architecture. At this scale, synchronous request-response for everything is a liability. Operations that can be async should be async — not just emails and notifications, but order processing, inventory updates, analytics events. A message queue (Kafka, SQS, RabbitMQ) becomes the backbone of the system.

Service extraction for genuine scaling needs. If one part of the system genuinely needs to scale independently — a search service, an image processing pipeline, a recommendation engine — extract it. Not because microservices are fashionable, but because the scaling profile is genuinely different and co-scaling is expensive.

The mistakes that cost the most

Premature optimisation. Solving stage 4 problems at stage 1. The most common form: building microservices before you understand the domain, adding a message queue before you have async work, sharding before you've hit the database ceiling. The cost is engineering time that could have been spent on the product.

Ignoring indexes. The most common form of the opposite mistake. Indexes are cheap to add and expensive to retrofit. Add them when you add the query.

Stateful application servers. In-memory session state, in-memory caches that can't be shared — these make horizontal scaling hard. Keep application servers stateless from the beginning.

Not measuring. Scaling decisions made without data are usually wrong. Add APM early. Know where the time is going before you add infrastructure.

Rewriting instead of optimising. "Our Rails app can't scale" is almost never true. "Our Rails app has slow queries and no caching" is almost always the real problem. Optimise before you rewrite.

The decisions worth making early

A few decisions are cheap to make early and expensive to retrofit:

• Stateless application servers. Sessions in Redis or the database, not in memory.
• Indexes on queried columns. Add them when you add the query.
• Structured logging with request IDs. Retrofitting this is painful.
• Async for anything that doesn't need to be synchronous. Email, notifications, webhooks — from the beginning.
• A deployment pipeline with rollback. The ability to deploy and roll back quickly is worth more than any infrastructure optimisation.

Everything else — read replicas, CDN, connection pooling, sharding, microservices — can wait until the problem is real.

TL;DR

At 10 users: optimise for iteration speed. One server, one database, no infrastructure overhead.

At 10,000 users: add indexes, move async work to a job queue, add caching for expensive data.

At 100,000 users: horizontal scaling for the app tier, read replica for the database, connection pooling, CDN.

At 1,000,000+ users: functional database partitioning, async-first architecture, service extraction for genuine independent scaling needs.

The pattern that holds across every stage: measure first, then optimise. The bottleneck is almost never where you think it is before you look.

If you're designing the infrastructure for a new product or hitting scaling limits on an existing one, let's talk.