Infrastructure

Building x8 Reverse Proxy From Scratch

Understanding routing, configs, HTTP forwarding, and the infrastructure decisions behind building a production-grade reverse proxy.

2026-06-1212 minadvanced

#networking#node#proxy#http

EST CONFIG|No response

650 words · 170 lines

Xonoxc⚏

Problem

Most developers treat reverse proxies as magic black boxes. You point nginx at a port and hope it works. I wanted to understand exactly what happens between a request arriving at port 80 and reaching an upstream server.

[!] Pitfall

My initial assumption was that reverse proxying is "just forwarding TCP connections." I thought it would be a thin wrapper around net.Socket piping.

In reality, a production reverse proxy needs to handle:

Connection pooling to upstreams
Keep-alive management
Request/response header rewriting
Buffering strategies
Error recovery
Graceful shutdown
TLS termination
Load balancing algorithms
Health checking

Each of these is a non-trivial engineering problem.

Implementation

The core of x8 is built around a layered architecture:

Client → Listener → Router → Middleware Chain → Upstream Pool → Backend
                             ↓
                        Load Balancer
                             ↓
                     Health Checker

Each layer has a single responsibility. The router matches host/path patterns. The middleware chain handles headers, rate limiting, logging. The upstream pool manages connections.

Connection Handling

Node.js http.Server gives us the raw request. The key insight is that you should not pipe req directly to an upstream socket. Instead:

class UpstreamConnection {
  private socket: net.Socket
  private requestQueue: PendingRequest[]

  async forward(req: IncomingMessage): Promise<ServerResponse> {
    const upstreamReq = http.request({
      hostname: this.host,
      port: this.port,
      path: req.url,
      method: req.method,
      headers: this.rewriteHeaders(req.headers),
    })

    req.pipe(upstreamReq)

    return new Promise((resolve, reject) => {
      upstreamReq.on("response", resolve)
      upstreamReq.on("error", reject)
    })
  }
}

This looks simple, but the devil is in connection reuse.

Problems Encountered

Connection Stickiness

The first version created a new TCP connection for every request. This worked but was incredibly slow — TLS handshakes alone added 50-100ms per request.

[!] Pitfall

I used agent: false thinking "fresh connections are safer."

The fix was implementing a proper http.Agent with keep-alive and max connection limits. This alone reduced p99 latency by 60%.

const agent = new http.Agent({
  keepAlive: true,
  keepAliveMsecs: 1000,
  maxSockets: 256,
  maxFreeSockets: 256,
  scheduling: "lifo",
})

LIFO scheduling matters because recently used sockets are less likely to have timed out.

Header Leaking

When forwarding requests, internal headers like x-forwarded-for require careful handling. The proxy needs to append, not overwrite.

> Key Insight

The x-forwarded-for chain is a ledger. Each proxy appends the client IP. Never rewrite — always append.

function rewriteHeaders(headers: IncomingHttpHeaders): OutgoingHttpHeaders {
  const forwarded = headers["x-forwarded-for"]
  const clientIp = headers["x-real-ip"]

  return {
    ...headers,
    "x-forwarded-for": forwarded
      ? `${forwarded}, ${clientIp}`
      : clientIp,
    "x-real-ip": undefined, // consumed, not forwarded
    host: this.upstreamHost, // must rewrite
  }
}

Tradeoffs

[ Tradedoffs.md ]

Strategy	Latency	Memory	Throughput	Complexity
Direct pipe (per-request connection)	High	Low	Low	Minimal
Keep-alive pool	Low	Medium	High	Medium
Multiplexed (HTTP/2)	Lowest	High	Highest	Complex
Socket pre-warming	Low	High	High	High

I chose keep-alive pooling with LIFO scheduling as the best balance of complexity and performance for a single-node deployment.

Final Mental Model

A reverse proxy is not a "connection forwarder." It is a protocol translator, load distributor, and failure isolator.

The mental model that finally clicked:

> Key Insight

Think of each upstream as a "worker pool." The proxy is a dispatcher that:

Accepts connections from the outside world
Decides which worker pool can handle the work
Manages the health and capacity of each pool
Gracefully degrades when pools are overloaded

This shifts the focus from "how do I forward bytes" to "how do I manage distributed work."

Improvements

If I were to rebuild this:

HTTP/2 multiplexing for true request pipelining
Circuit breaker pattern for failing upstreams
Dynamic weight adjustment based on response times
Connection coalescing for HTTP/2 to the same origin
Better backpressure handling for slow clients