Skip to content
Data Rocksdocs

HTTPS Proxy Flow & Certificate Architecture

This document describes the complete flow of an HTTPS request through the MITM proxy, including all certificates involved, how they are generated, and how TLS is terminated and re-established.

Certificates Overview

Section titled “Certificates Overview”

There are four certificate/key artifacts involved in a proxied HTTPS connection:

# Artifact Who holds it Purpose
1 MITM CA Certificate (ca.crt) Proxy + all clients (trust store) Root of trust for intercepted connections
2 MITM CA Private Key (ca.key) Proxy only Signs per-domain certificates on the fly
3 Per-domain certificate Generated in memory by proxy Presented to the client, impersonates the real server
4 Upstream server’s real certificate Origin server Verified by the proxy using system CAs (+ optional outgoing_ca_bundle, outgoing_ca, outgoing_truststore_*)

Certificate Generation

Section titled “Certificate Generation”

The MITM CA is generated once and distributed ahead of time. The built-in gencert subcommand handles this without requiring OpenSSL:

Terminal window
# Simple root CA (self-signed, ECDSA P-256, 10-year validity)
mitm-proxy gencert --type root \
  --cn "Internal-MITM-CA" --org "ProxyCorp" --country US \
  --out-cert certs/ca.crt --out-key certs/ca.key


# Production setup with intermediate CA
mitm-proxy gencert --type root --cn "Corp Root CA" --org "Corp" \
  --out-cert certs/root-ca.crt --out-key certs/root-ca.key


mitm-proxy gencert --type intermediate \
  --signing-cert certs/root-ca.crt --signing-key certs/root-ca.key \
  --cn "MITM Proxy CA" --org "Corp" --max-path-len 0 --validity 365 \
  --out-cert certs/mitm-ca.crt --out-key certs/mitm-ca.key \
  --out-chain certs/mitm-chain.crt


# Generate client trust bundles for distribution
mitm-proxy gencert --type root --cn "Corp Root CA" \
  --out-cert certs/root-ca.crt --out-key certs/root-ca.key \
  --out-client-bundle certs/trust.pem \
  --out-client-p12 certs/truststore.p12 --client-p12-password changeit

Note: The gencert command always sets CA:TRUE in BasicConstraints and KeyUsageCertSign | KeyUsageCRLSign in KeyUsage. The proxy validates this at startup and will refuse to start if the MITM certificate is not a CA.

Alternatively, you can use OpenSSL (scripts/gen-ca.sh) or provide the CA as a PKCS#12 keystore (.p12) instead of separate PEM files. The gencert command can also output PKCS#12 keystores via --out-p12.

Deployment requirements:

  • ca.crt + ca.key (or .p12) must be available to the proxy at startup.
  • ca.crt must be installed as a Trusted Root CA on every client machine (OS trust store, browser, container, etc.). Without this, clients will reject the proxy-generated certificates.

Per-Domain Certificate (On-the-Fly)

Section titled “Per-Domain Certificate (On-the-Fly)”

When the proxy intercepts a CONNECT tunnel, goproxy dynamically generates a TLS certificate for the requested domain:

  1. Creates a new X.509 certificate with the target hostname as Subject/SAN.
  2. Signs it with the MITM CA private key.
  3. This certificate lives only in memory for the duration of the connection.

Full HTTPS Connection Flow

Section titled “Full HTTPS Connection Flow”

Step-by-step

Section titled “Step-by-step”

  1. Client initiates CONNECT – The client sends an HTTP CONNECT host:443 request to the proxy over plaintext HTTP.

  2. Proxy evaluates the CONNECT ACL – Before performing MITM, DecideConnect inspects the host.

    • Matches acl.passthrough and acl.blacklist: the CONNECT is rejected with 403 Policy Blocked written to the client connection, and the request is recorded in the blocked-request log. A passthrough tunnel is never inspected, so a denied host must not receive one. This is the only case refused at CONNECT time.
    • Matches acl.passthrough only: the proxy responds with 200 Connection Established and creates a plain TCP tunnel — no TLS interception occurs, the client talks directly to the upstream server through the tunnel. The PASSTHROUGH action is logged and recorded in metrics. Steps 3–9 are skipped.
    • Otherwise: interception proceeds, including for blacklisted hosts. Nothing reaches the upstream before the request-stage ACL runs, and the client receives a readable 403.

  3. Proxy accepts the tunnel (MITM) – For non-passthrough hosts, the proxy responds with 200 Connection Established. The TCP tunnel is now open.

  4. Client-side TLS handshake (MITM) – The proxy acts as the TLS server. goproxy generates a certificate for host signed by the MITM CA and presents it. The client verifies it against its trust store (which includes the MITM CA) and the handshake completes.

  5. Client sends HTTP request – Over the now-encrypted tunnel, the client sends the actual HTTP request (e.g., GET /api/data).

  6. Proxy evaluates policy – The handleRequest function processes the request:

    • Generates and injects X-Request-ID header
    • Checks the blacklist first; a match blocks with 403 and the rewrite table is never consulted
    • Otherwise checks rewrite rules (exact match, then wildcard patterns)
    • If no rewrite matched: checks whitelist -> default policy
    • If blocked (BLACK-LISTED or BLOCKED): returns 403 Forbidden immediately; no upstream connection is made.

  7. Proxy dials upstream – For allowed/rewritten requests, the custom DialContext resolves the destination:

    • Normal case: dials the original host:443 via DNS
    • Rewrite case (split-brain DNS): dials target_ip:443 instead, bypassing DNS entirely. TLS SNI still uses the original hostname, so the upstream server’s certificate is verified against the original domain.

  8. Upstream TLS handshake – The proxy acts as a TLS client toward the origin server. It verifies the server’s real certificate against:

    • System CA trust store
    • Optional outgoing_ca_bundle (PEM bundle file)
    • Optional outgoing_ca (list of individual PEM cert files)
    • Optional outgoing_truststore_* (PKCS#12 truststore)

  9. Request forwarded – The proxy forwards the (potentially modified) request to the upstream server. For rewritten domains, custom headers are injected.

  10. Response relayed – The upstream response flows back through both TLS tunnels to the client. Response metrics are recorded.

Two Independent TLS Sessions

Section titled “Two Independent TLS Sessions”

The proxy maintains two separate TLS sessions simultaneously:

graph LR
    C["Client"] -- "TLS Session #1<br/>MITM cert (forged for domain)<br/>Proxy = TLS server" --> P["MITM Proxy<br/>(plaintext access)"]
    P -- "TLS Session #2<br/>Real server cert<br/>Proxy = TLS client" --> U["Upstream Server"]

    style C fill:#dceeff,stroke:#4a90d9
    style P fill:#e8f5e9,stroke:#66bb6a,font-weight:bold
    style U fill:#fff3e0,stroke:#e6a23c

The proxy has access to the plaintext HTTP request/response between these two TLS sessions, which is what enables inspection, policy enforcement, header injection, and logging.

Sequence Diagram

Section titled “Sequence Diagram”
sequenceDiagram
    autonumber
    participant C as Client
    participant P as MITM Proxy
    participant U as Upstream Server

    note over C,P: Plaintext HTTP (port 8080)

    C->>P: CONNECT api.example.com:443 HTTP/1.1

    note right of P: Check acl.passthrough patterns<br/>against hostname

    alt PASSTHROUGH (host matches acl.passthrough)
        P->>C: HTTP/1.1 200 Connection Established
        note over C,U: Plain TCP tunnel (no MITM)<br/>Proxy cannot inspect traffic
        C->>U: TLS Handshake directly with upstream
        U->>C: Server certificate (real cert)
        C->>U: Application data
        U->>C: Application data
    else Normal flow (MITM)
    P->>C: HTTP/1.1 200 Connection Established

    note over C,P: TLS Handshake #1 (Client ↔ Proxy)
    rect rgb(220, 240, 255)
        C->>P: ClientHello (SNI: api.example.com)
        note right of P: goproxy generates cert for<br/>api.example.com signed by MITM CA
        P->>C: ServerHello + Certificate (signed by MITM CA)
        note left of C: Client verifies cert against<br/>trust store (MITM CA installed)
        C->>P: Finished
        P->>C: Finished
    end

    note over C,P: Encrypted tunnel established

    C->>P: GET /api/data HTTP/1.1<br/>Host: api.example.com
    note right of P: handleRequest() evaluates policy:<br/>1. Inject X-Request-ID<br/>2. Check blacklist (short-circuits rewrites)<br/>3. Check rewrites (exact → wildcard)<br/>4. Check whitelist<br/>5. Apply default policy

    alt BLOCKED or BLACK-LISTED
        P->>C: HTTP/1.1 403 Forbidden<br/>"Policy Blocked"
    else REWRITTEN
        note right of P: Custom DialContext routes<br/>TCP to target_ip instead of DNS
        note over P,U: TLS Handshake #2 (Proxy ↔ Upstream)
        rect rgb(255, 240, 220)
            P->>U: TCP connect to target_ip:443
            P->>U: ClientHello (SNI: api.example.com)
            U->>P: ServerHello + Certificate (real cert)
            note left of P: Proxy verifies real cert against<br/>system CAs + custom CA config
            P->>U: Finished
            U->>P: Finished
        end
        note right of P: Inject custom headers<br/>(e.g. X-Proxy-Source)
        P->>U: GET /api/data HTTP/1.1<br/>Host: api.example.com<br/>X-Request-ID: abc123<br/>X-Proxy-Source: egress-gateway
        U->>P: HTTP/1.1 200 OK + body
        P->>C: HTTP/1.1 200 OK + body
    else WHITE-LISTED or ALLOWED-BY-DEFAULT
        note over P,U: TLS Handshake #2 (Proxy ↔ Upstream)
        rect rgb(220, 255, 220)
            P->>U: TCP connect to api.example.com:443 (via DNS)
            P->>U: ClientHello (SNI: api.example.com)
            U->>P: ServerHello + Certificate (real cert)
            note left of P: Proxy verifies real cert against<br/>system CAs + custom CA config
            P->>U: Finished
            U->>P: Finished
        end
        P->>U: GET /api/data HTTP/1.1<br/>Host: api.example.com<br/>X-Request-ID: abc123
        U->>P: HTTP/1.1 200 OK + body
        P->>C: HTTP/1.1 200 OK + body
    end
    end

Split-Brain DNS Detail

Section titled “Split-Brain DNS Detail”

In the rewrite case, the proxy achieves split-brain DNS at the TCP dial layer, not at DNS level. The upstream server at the target_ip must present a valid certificate for the original domain. This is the typical pattern for routing traffic to an internal load balancer or service mesh endpoint that terminates TLS for the original domain.

graph LR
    subgraph dns["Public DNS"]
        R["api.example.com<br/>→ 203.0.113.50"]
    end

    subgraph config["Proxy Rewrite Rule"]
        RW["domain: api.example.com<br/>target_ip: 10.20.30.40"]
    end

    C["Client"] -- "CONNECT api.example.com:443" --> P["MITM Proxy"]

    P -. "DNS would resolve to<br/>203.0.113.50 (ignored)" .-x R

    P -- "TCP dial → 10.20.30.40:443<br/>TLS SNI: api.example.com" --> U["Internal Server<br/>10.20.30.40"]

    U -- "presents valid cert for<br/>api.example.com ✅" --> P

    style dns fill:#f5f5f5,stroke:#999,stroke-dasharray: 5 5
    style config fill:#e8f5e9,stroke:#66bb6a
    style P fill:#dceeff,stroke:#4a90d9
    style U fill:#fff3e0,stroke:#e6a23c

Trust Chain Summary

Section titled “Trust Chain Summary”

Two independent trust chains are at play. On the client side, the MITM CA must be installed as a trusted root so clients accept the per-domain certificates the proxy generates on the fly. On the proxy side, upstream server certificates are verified against the system CA store plus any custom CAs provided via outgoing_ca_bundle, outgoing_ca, or outgoing_truststore_*. These two chains are completely separate — compromising or changing one does not affect the other.

The gencert subcommand can produce client trust bundles in both PEM (--out-client-bundle) and PKCS#12 (--out-client-p12) formats to simplify distribution. The PKCS#12 truststore can be used directly by Java applications (-Djavax.net.ssl.trustStore=...) or imported into existing keystores via keytool.

graph TB
    subgraph client_store["Client Trust Store"]
        SysCA1["System CAs<br/>(DigiCert, Let's Encrypt, ...)"]
        MITMCA["MITM CA (ca.crt)"]
    end

    subgraph proxy_store["Proxy Trust Store"]
        SysCA2["System CAs<br/>(DigiCert, Let's Encrypt, ...)"]
        CustomCA["outgoing_ca_bundle / outgoing_ca<br/>outgoing_truststore_*<br/>(optional, for internal CAs)"]
    end

    subgraph tls1["TLS Session #1: Client ↔ Proxy"]
        GenCert["Per-domain certificate<br/>CN: api.example.com<br/>(generated on the fly)"]
    end

    subgraph tls2["TLS Session #2: Proxy ↔ Upstream"]
        RealCert["Real server certificate<br/>CN: api.example.com<br/>(issued by public/corporate CA)"]
    end

    MITMCA -- "signs" --> GenCert
    MITMCA -. "trusted by client" .-> client_store
    GenCert -- "presented to Client<br/>Client verifies ✅" --> client_store

    SysCA2 -. "verifies public upstreams" .-> RealCert
    CustomCA -. "verifies internal upstreams" .-> RealCert
    RealCert -- "presented to Proxy<br/>Proxy verifies ✅" --> proxy_store

    style client_store fill:#dceeff,stroke:#4a90d9
    style proxy_store fill:#fff3e0,stroke:#e6a23c
    style tls1 fill:#e8f5e9,stroke:#66bb6a
    style tls2 fill:#fce4ec,stroke:#ef5350
    style MITMCA fill:#bbdefb,stroke:#1976d2,font-weight:bold

Docs for v4.1.1 · built from the release tag