Scalability: How a System Goes From Falling Over to Getting Sturdier as It Grows

Say you've built a website. Normally it gets a few hundred visitors a day and the server barely breaks a sweat. Then one day the product gets covered in the news, traffic jumps a hundredfold, and the site just goes down. Your first instinct is probably "I should've beefed up the server." But would a bigger machine actually have fixed it? Let's break down what scalability really means, and what your actual options are when a system needs to absorb traffic growth.

What scalability actually means

Scalability describes whether a system can maintain its performance as load increases, by adding resources. A system that runs smoothly today but grinds to a halt — or dies outright — the moment traffic grows tenfold isn't scalable, no matter how well it performs right now.

This is easy to confuse with performance. Performance asks "is it fast right now?" Scalability asks "can it stay fast once traffic grows?" A system can perform great under low traffic and still have zero scalability, because its architecture simply has no way to absorb growth by throwing more resources at it. These are two different questions.

Option one: vertical scaling — make the machine bigger

The most intuitive fix is vertical scaling: keep the architecture as-is and just upgrade the hardware. Bump up the CPU, memory, and disk on a single machine. If a 4-core, 8GB box can't keep up, swap it for a 16-core, 64GB one.

The appeal is simplicity — no code changes, no architectural rework. But it runs into two hard limits:

• There's a ceiling. Hardware specs can only go so far, and the higher you go, the worse the price-to-performance ratio gets — costs stop scaling linearly and start climbing fast.
• Single point of failure. All traffic hits one machine. If that machine goes down, the whole service goes down with it. No failover.

Option two: horizontal scaling — add more machines

Horizontal scaling takes a different approach: instead of growing one machine, you run several machines of the same spec and spread traffic across them. In theory you can keep adding machines indefinitely — ten not enough, add twenty — without hitting the hardware ceiling vertical scaling runs into. And if one machine dies, the others keep serving traffic.

But horizontal scaling doesn't work for free. It has a precondition: these machines need to share the load fairly, and users need the same experience no matter which one they land on. That precondition comes down to two things: making the service stateless, and having something in front to route traffic.

Stateless: so any machine can pick up the request

Say your login system stores each user's session directly in server memory:

// Stateful approach — session lives in a single server's memory
const sessions = new Map()
 
function login(userId, res) {
  const sessionId = crypto.randomUUID()
  sessions.set(sessionId, { userId })
  res.cookie('sessionId', sessionId)
}
 
function getUser(sessionId) {
  return sessions.get(sessionId) // only this server remembers it
}

That's fine as long as you're running a single machine. But scale out to three machines and you've got a problem: a user logs in on machine A, their session lives in machine A's memory, and their next request gets routed to machine B — which has never heard of that session. The user gets treated as logged out and kicked back to the login screen.

This is exactly why horizontal scaling pushes you toward stateless services: the server itself shouldn't hold any state tied to a specific user. That state belongs somewhere every server can reach — Redis, for example:

// Stateless approach — session lives in Redis, reachable by every server
async function login(userId, res, redis) {
  const sessionId = crypto.randomUUID()
  await redis.set(sessionId, JSON.stringify({ userId }))
  res.cookie('sessionId', sessionId)
}
 
async function getUser(sessionId, redis) {
  const data = await redis.get(sessionId) // any server can fetch this
  return data ? JSON.parse(data) : null
}

Now it doesn't matter which server picks up the request — everyone reads the same session data, and the machines become truly interchangeable.

Load balancer: deciding where traffic goes

Once you've got a pool of stateless servers, you need something in front deciding which machine handles each incoming request. That's the load balancer's job. Common strategies include round robin, routing to whichever machine currently has the lowest load, or routing to the machine closest to the user geographically.

From the caller's perspective, there's just one entry point — no need to know there's a whole fleet of machines taking turns behind it.

The database is where horizontal scaling actually gets hard

Making the application layer stateless isn't too bad. The real difficulty is usually the database — it's literally the thing that holds state, so you can't just declare it stateless and move on. Two common approaches:

• Read replicas: keep several read-only copies of the database, and route read traffic — usually the majority of requests — to those copies, while writes still go to a single primary. This spreads out read load dramatically, but the replicas lag slightly behind, so a read might return slightly stale data.
• Sharding: split your data by some rule (like user ID) across multiple databases, where each shard only holds a slice of the data. This spreads out both read and write load, but at the cost of much higher complexity — something as simple as "fetch all of a user's orders," which used to be a single query, now might require first figuring out which shard that user lives on.

Caching: keep requests from hitting the database at all

Besides making the database handle more load, the other angle is reducing how many requests actually reach it in the first place — that's what caching is for. Frequently-read, rarely-changed data — a product category list, a user's basic profile — gets stashed somewhere much faster to read from, like memory or Redis, so the next identical query skips the database entirely.

Done well, caching can take a huge amount of pressure off the database. But it introduces its own problem: when the underlying data changes, how does the stale copy in the cache get updated or invalidated? That's complexity the caching layer itself has to handle — adding a cache doesn't mean you're done.

More scalability isn't automatically better

At this point it might seem like every system should launch fully stateless, pre-sharded, and cached everywhere. That's over-engineering. Horizontal scaling, sharding, and cache invalidation each add a layer of complexity, and every layer adds to your operational burden and makes debugging harder.

A site with a few hundred daily users can probably fix its traffic problem in an afternoon just by bumping up the server spec — there's no need to pre-build all that architectural complexity. How much you invest in scalability should track how fast and how big your traffic is actually growing, not "we might need it eventually."

Wrapping up

Scalability is about whether a system can hold its performance as load grows by throwing more resources at it — not about how fast it runs today. Vertical scaling is simple but hits a hardware ceiling and carries single-point-of-failure risk. Horizontal scaling can, in theory, keep growing indefinitely, but it needs a stateless service design, a load balancer to route traffic, and — at the database layer — read replicas or sharding paired with caching to actually hold up. When designing a system, instead of reaching for every technique on day one, it's worth first understanding how fast your traffic is actually growing, and deciding what to invest in at each stage from there.