Distributed Systems from Scratch: Building a Scalable Task Queue

Built by

hieumessideptrai

Computer Science

Distributed Systems from Scratch: Building a Scalable Task Queue

Scale your applications beyond a single server. Learn the core patterns of distributed architectures by building, containerizing, and scaling an event-driven task processing system with an API Gateway, background workers, and shared caching.

5 modules15 lessonsComputer Science

What you learn by building this

Build it yourself, get guided when you are stuck, and leave with proof you can actually show.

Learning Journey

From Monolith to Microservices: API Gateway & Worker Isolation

3 lessons

Deconstruct a single server into dedicated services to handle high traffic and isolated workloads safely.

Asynchronous Decoupling with Message Queues

4 lessons

Transition from blocking HTTP calls to a non-blocking, event-driven architecture using message queues.

Containerization and Horizontal Scaling

3 lessons

Package your distributed services and scale them horizontally to handle massive parallel workloads.

Distributed Resilience: Caching and Fault Tolerance

3 lessons

Protect your system against failure, handle network partitions, and optimize reads with shared caches.

Deployment: Sharing Your Project

2 lessons

Deploy your distributed task queue to a free-tier platform and share a live URL others can try.

Public lesson

Designing a Multi-Service Architecture

Right now, our backend is a single block. If a user asks our server to do a heavy, slow task, the entire server freezes up and other users can't even load the homepage.

Did You Know? Netflix's open‑source API gateway, Zuul, routes billions of requests each day and can handle over 100 million requests per minute. It powers Netflix's edge layer, demonstrating how a well‑designed gateway can scale to massive traffic while keeping services isolated.

Real-World Case Study: Netflix's Multi-Service Architecture

To understand why we split monolithic systems, let's look at a giant that pioneered this pattern: Netflix.

In its early days, Netflix ran on a monolithic architecture. As millions of users streamed videos worldwide, a single bug in the billing code could take down the video playback system, or a spike in users browsing the movie catalog could crash the authentication service.

To solve this, Netflix broke their monolith apart into hundreds of microservices. Here is how key services interact in their ecosystem:

How They Communicate

The API Gateway: A single entrance point. When you open Netflix, your app makes requests to the Gateway. The Gateway authenticates the request, performs rate limiting, and routes traffic to the correct downstream services.
Synchronous Communication:
- When you log in, the Gateway makes a synchronous call to the Authentication Service.
- When you search for a title, the Gateway calls the Catalog Service to fetch metadata.
- When you click "Play", the Gateway contacts the Video Streaming Service to locate the media files.
- When you update your payment details, the Gateway contacts the Billing Service.
Asynchronous Communication & Queues:
- Every time you watch a movie or browse a genre, the Video Streaming and Catalog services publish tracking events to an asynchronous queue (such as Kafka or RabbitMQ).
- The Recommendation Engine processes these events in the background from the queue. This prevents heavy, complex recommendation calculations from slowing down your live streaming experience.

Why Break Them Apart?

Each of these services has different characteristics, requirements, and risk profiles:

Authentication must be highly secure and extremely fast.
Video Streaming requires immense bandwidth and optimized network delivery.
Billing requires high data consistency and transactional safety, but is rarely accessed compared to the video player.
Recommendation Engine performs heavy mathematical data crunching.

If they were bundled together, an issue or heavy load in one (e.g., millions of recommendations being calculated) would impact the availability of others (e.g., blocking logins or streaming).

Key Benefits of Independent Scaling & Deployment

Independent Scaling: On a Friday night, Netflix sees a massive surge in people watching videos. Netflix can scale up the Video Streaming Service and Catalog Service to run on thousands of server instances, while keeping the Billing Service running on just a few instances (since billing cycles are distributed throughout the month). This saves enormous infrastructure costs.
Independent Deployment: Engineers working on the Recommendation Engine can deploy updates multiple times a day without risking a disruption to the core Video Streaming Service or Authentication Service.
Fault Isolation: If the Recommendation Engine experiences a memory leak and crashes, users can still log in, browse the catalog, and watch movies—they just won't see personalized recommendations temporarily.

Netflix's Open-Source Microservices Ecosystem

Netflix is famous not just for adopting microservices, but for sharing their technology with the entire world. They have open-sourced a massive portion of their cloud infrastructure tools, allowing thousands of other companies to build and run highly resilient microservices.

Here are the key open-source tools pioneered by Netflix:

Tool Name	Core Capability	What It Does in Simple Terms
Eureka	Service Discovery	Keeps a dynamic, real-time "phone book" of where every microservice instance is running (IP addresses and ports) so services can find each other.
Zuul	API Gateway	Acts as the front entrance, routing all incoming client requests to the appropriate downstream microservices while handling security and rate limiting.
Hystrix	Circuit Breaker	Stops cascading failures. If one service is slow or crashing, Hystrix "trips" a circuit breaker to stop sending traffic to it, returning a safe fallback response instead of freezing the system.
Chaos Monkey	Resiliency Testing	Randomly terminates virtual machine instances and services in production to force engineers to build self-healing, highly resilient software.
Ribbon	Client-Side Load Balancing	Helps microservices distribute outgoing requests evenly across multiple healthy instances of another service (working closely with Eureka).
Spinnaker	Continuous Delivery	A multi-cloud deployment platform used to release and manage software updates quickly, safely, and automatically.

How These Tools Work Together

In a real production environment, these tools act as a coordinated symphony:

A user opens Netflix on their TV. The app sends a request to the Zuul API Gateway.
Zuul doesn't hardcode where the services live. Instead, it queries Eureka to find the network location of the Video Streaming Service.
Ribbon selects one of the healthy Video Streaming Service instances to handle the request.
During playback, the Video Streaming Service calls the Recommendation Engine. Because this call is wrapped with Hystrix, if the recommendation service goes down, the player doesn't freeze—it just displays a default "Popular Titles" list instead of custom suggestions.
All of these services are constantly deployed, scaled, and managed using Spinnaker.
At any moment, Chaos Monkey might kill an instance of the Video Streaming Service, proving that the system can instantly reroute requests to another healthy instance without the viewer noticing a thing!

Why Netflix Open Sources Their Architecture

You might wonder: why would a multi-billion-dollar company share their proprietary cloud architecture for free?

Setting Industry Standards: By making their tools public, Netflix established them as the industry standard. This means new hires already know how Netflix's architecture works before their first day on the job.
Crowdsourced Improvements: Thousands of developers from external companies find bugs, suggest optimizations, and submit code contributions, improving the tools at no cost to Netflix.
Ecosystem Resilience: Strengthening the public cloud and microservices ecosystem ensures that cloud providers (like AWS) optimize for these patterns, making the entire internet more stable.

From Theory to Your Project

You just saw how Netflix splits responsibilities across services to prevent a slow or failing component from dragging down the entire platform. Your project faces the same challenge: right now, all HTTP requests flow through a single backend service. If an incoming request triggers a heavy, blocking operation (like a large data transformation or a slow third-party API call), every other request on that service blocks and waits—which means users see slow or unresponsive pages even for lightweight tasks.

To fix this, we're going to apply the same pattern: split your existing backend so that request handling and heavy work live in separate services.

Here's what we'll change:

Gateway (port 5000, based on your existing backend): Remains the public entry point. It validates incoming requests quickly and routes them—either returning light responses immediately or delegating heavy work to the Processor.
Processor (port 5001, a new service): Handles heavy operations asynchronously. The Gateway calls it for anything that's slow or CPU-intensive, keeping the gateway responsive for other users.

Because your monorepo is ready, we can run these two services side-by-side. Let's build the Processor and wire them together.

Step 1: Set up the Processor (Port 5001)

This service will live in apps/processor/src/index.ts. Its only job is to receive raw data, process it, and return the result.

Tasks

Open apps/processor/src/index.ts and set up the Express route. You need to write the logic that takes a string from the request body, converts it to uppercase, and sends it back.

import express from 'express';

const app = express();
const PORT = 5001;

app.use(express.json());

// This endpoint receives the raw data to process
app.post('/jobs', (req, res) => {
  const { data } = req.body;

  if (!data) {
    return res.status(400).json({ error: 'No data provided' });
  }

  // TODO: Transform the 'data' string to uppercase
  const processedData = // ... your code here ...

  res.json({
    status: 'completed',
    result: processedData,
    processedAt: new Date().toISOString()
  });
});

app.listen(PORT, () => {
  console.log(`Processor service running on port ${PORT}`);
});

import express from 'express';

const app = express();
const PORT = 5001;

app.use(express.json());

// This endpoint receives the raw data to process
app.post('/jobs', (req, res) => {
  const { data } = req.body;

  if (!data) {
    return res.status(400).json({ error: 'No data provided' });
  }

  // TODO: Transform the 'data' string to uppercase
  const processedData = // ... your code here ...

  res.json({
    status: 'completed',
    result: processedData,
    processedAt: new Date().toISOString()
  });
});

app.listen(PORT, () => {
  console.log(`Processor service running on port ${PORT}`);
});

Step 2: Set up the Gateway (Port 5000)

Now let’s build the Gateway in apps/gateway/src/index.ts.

The Gateway will accept a client request at /work, extract the payload, and use fetch to pass it off to the Processor service on port 5001. Once the Processor responds, the Gateway sends that result back to the user.

Tasks

Complete the fetch call below to connect the two services.

import express from 'express';

const app = express();
const PORT = 5000;

app.use(express.json());

app.post('/work', async (req, res) => {
  const { payload } = req.body;

  if (!payload) {
    return res.status(400).json({ error: 'No payload provided' });
  }

  try {
    // TODO: Use native fetch to send a POST request to the Processor's /jobs endpoint.
    // The Processor is running on port 5001.
    const response = await fetch('/* fill in the processor URL */', {
      method: 'POST',
      headers: {
        'Content-Type': 'application/json',
      },
      body: JSON.stringify({ data: payload }),
    });

    if (!response.ok) {
      throw new Error(`Processor returned status ${response.status}`);
    }

    // TODO: Parse the JSON response from the Processor
    const data = await // ... your code here ...

    // Send the processed result back to the client
    res.json({
      message: 'Gateway successfully routed the job',
      processorResponse: data,
    });
  } catch (error) {
    console.error('Error contacting processor:', error);
    res.status(502).json({ error: 'Processor service is unreachable' });
  }
});

app.listen(PORT, () => {
  console.log(`Gateway service running on port ${PORT}`);
});

import express from 'express';

const app = express();
const PORT = 5000;

app.use(express.json());

app.post('/work', async (req, res) => {
  const { payload } = req.body;

  if (!payload) {
    return res.status(400).json({ error: 'No payload provided' });
  }

  try {
    // TODO: Use native fetch to send a POST request to the Processor's /jobs endpoint.
    // The Processor is running on port 5001.
    const response = await fetch('/* fill in the processor URL */', {
      method: 'POST',
      headers: {
        'Content-Type': 'application/json',
      },
      body: JSON.stringify({ data: payload }),
    });

    if (!response.ok) {
      throw new Error(`Processor returned status ${response.status}`);
    }

    // TODO: Parse the JSON response from the Processor
    const data = await // ... your code here ...

    // Send the processed result back to the client
    res.json({
      message: 'Gateway successfully routed the job',
      processorResponse: data,
    });
  } catch (error) {
    console.error('Error contacting processor:', error);
    res.status(502).json({ error: 'Processor service is unreachable' });
  }
});

app.listen(PORT, () => {
  console.log(`Gateway service running on port ${PORT}`);
});

Tasks

Step 3: Test the Connection

To verify that your services are talking to each other, we need to run both at the same time.

Open two terminal windows.
In the first terminal, start the Processor:
```
npm run dev --workspace=processor
```
```
npm run dev --workspace=processor
```
(Or your monorepo's equivalent run command, ensuring it runs on port 5001)
In the second terminal, start the Gateway:
```
npm run dev --workspace=gateway
```
```
npm run dev --workspace=gateway
```
(Ensuring it runs on port 5000)

Tasks

Now, send a request to the Gateway (port 5000) using curl in a third terminal, or your preferred API client (like Postman/Thunder Client):

curl -X POST http://localhost:5000/work \
     -H "Content-Type: application/json" \
     -d '{"payload": "hello multi-service world"}'

curl -X POST http://localhost:5000/work \
     -H "Content-Type: application/json" \
     -d '{"payload": "hello multi-service world"}'

What to look for:

If everything is wired correctly, you should receive a response back from port 5000 containing the uppercase version of your payload, generated by the service on port 5001:

{
  "message": "Gateway successfully routed the job",
  "processorResponse": {
    "status": "completed",
    "result": "HELLO MULTI-SERVICE WORLD",
    "processedAt": "2026-02-05..."
  }
}

{
  "message": "Gateway successfully routed the job",
  "processorResponse": {
    "status": "completed",
    "result": "HELLO MULTI-SERVICE WORLD",
    "processedAt": "2026-02-05..."
  }
}

If you get a 502 error, double-check that your Gateway's fetch URL is pointing exactly to http://localhost:5001/jobs.

Course Outline

5 modules · 15 lessons

From Monolith to Microservices: API Gateway & Worker Isolation

Asynchronous Decoupling with Message Queues

Containerization and Horizontal Scaling

Distributed Resilience: Caching and Fault Tolerance

Deployment: Sharing Your Project

Learn by building your own version.

Remix this public project to open the workspace, follow the guided build, and let the AI mentor teach you through the work instead of doing it for you.