Infrastructure

Building x8 Reverse Proxy From Scratch

Understanding routing, configs, HTTP forwarding, and the infrastructure decisions behind building a production-grade reverse proxy.

2026-06-12·12 min·advanced

#networking#node#proxy#http

650 words · 170 lines

Xonoxc

Problem

Most developers treat reverse proxies as magic black boxes. You point nginx at a port and hope it works. I wanted to understand exactly what happens between a request arriving at port 80 and reaching an upstream server.

Implementation

The core of x8 is built around a layered architecture:

Architecture

Client → Listener → Router → Middleware Chain → Upstream Pool → Backend
                             ↓
                        Load Balancer
                             ↓
                     Health Checker

Each layer has a single responsibility. The router matches host/path patterns. The middleware chain handles headers, rate limiting, logging. The upstream pool manages connections.

Connection Handling

Node.js http.Server gives us the raw request. The key insight is that you should not pipe req directly to an upstream socket. Instead:

code

class UpstreamConnection {
  private socket: net.Socket
  private requestQueue: PendingRequest[]

  async forward(req: IncomingMessage): Promise<ServerResponse> {
    const upstreamReq = http.request({
      hostname: this.host,
      port: this.port,
      path: req.url,
      method: req.method,
      headers: this.rewriteHeaders(req.headers),
    })

    req.pipe(upstreamReq)

    return new Promise((resolve, reject) => {
      upstreamReq.on("response", resolve)
      upstreamReq.on("error", reject)
    })
  }
}

This looks simple, but the devil is in connection reuse.

Problems Encountered

Connection Stickiness

The first version created a new TCP connection for every request. This worked but was incredibly slow — TLS handshakes alone added 50-100ms per request.

Pitfall

I used agent: false thinking "fresh connections are safer."

The fix was implementing a proper http.Agent with keep-alive and max connection limits. This alone reduced p99 latency by 60%.

const agent = new http.Agent({
  keepAlive: true,
  keepAliveMsecs: 1000,
  maxSockets: 256,
  maxFreeSockets: 256,
  scheduling: "lifo",
})

LIFO scheduling matters because recently used sockets are less likely to have timed out.

Header Leaking

When forwarding requests, internal headers like x-forwarded-for require careful handling. The proxy needs to append, not overwrite.

Key Insight

The x-forwarded-for chain is a ledger. Each proxy appends the client IP. Never rewrite — always append.

function rewriteHeaders(headers: IncomingHttpHeaders): OutgoingHttpHeaders {
  const forwarded = headers["x-forwarded-for"]
  const clientIp = headers["x-real-ip"]

  return {
    ...headers,
    "x-forwarded-for": forwarded
      ? `${forwarded}, ${clientIp}`
      : clientIp,
    "x-real-ip": undefined, // consumed, not forwarded
    host: this.upstreamHost, // must rewrite
  }
}

Tradeoffs

Tradeoffs.md

Strategy	Latency	Memory	Throughput	Complexity
Direct pipe (per-request connection)	High	Low	Low	Minimal
Keep-alive pool	Low	Medium	High	Medium
Multiplexed (HTTP/2)	Lowest	High	Highest	Complex
Socket pre-warming	Low	High	High	High

I chose keep-alive pooling with LIFO scheduling as the best balance of complexity and performance for a single-node deployment.

Final Mental Model

A reverse proxy is not a "connection forwarder." It is a protocol translator, load distributor, and failure isolator.

The mental model that finally clicked:

Improvements

If I were to rebuild this:

HTTP/2 multiplexing for true request pipelining
Circuit breaker pattern for failing upstreams
Dynamic weight adjustment based on response times
Connection coalescing for HTTP/2 to the same origin
Better backpressure handling for slow clients