10. HTTP Engine#

HTTP Engine is Link1's Layer 7 processing module for HTTP/HTTPS debugging, compatibility fixes, request/response rewriting, Mock, Capture, and Replay. It is not required for ordinary proxying. Use it only when you explicitly need to inspect or modify HTTP content.

Processing Position#

Inbound connection
  -> Routing/outbound selection
  -> HTTP Engine decides whether to take over the HTTP stream
  -> URL/Header/Body/JSON/JQ/Script/Mock/Route rules
  -> Capture records
  -> Upstream or client

HTTPS content is encrypted by default. To rewrite HTTPS requests/responses, MITM is required:

1. Generate or import a CA.

1. Trust the CA on the client.

1. Match the target domain in http-engine.mitm.hosts.

1. Route the connection through Link1.

Minimal Enablement#

http-engine:
  enabled: true
  defaults:
    body-max-size: 1 MiB
    on-error: fail-open

Field effects:

defaults#

http-engine:
  defaults:
    body-max-size: 1 MiB
    jq-timeout: 100ms
    script-timeout: 50ms
    script-memory-limit: 8 MiB
    on-error: fail-open

MITM#

Using the CA Automatically Managed by the App#

managed: ca-managed means using the local CA created and saved by the Link1 App. Ordinary users do not need to write PEM manually. Generate/trust the CA in the App, then reference this CA ID in the configuration.

http-engine:
  enabled: true
  mitm:
    enabled: true
    ca-cert:
      managed: ca-managed
    ca-key:
      managed: ca-managed
    hosts:
      - service.example.com
      - '*.debug.example.com'
    h2: true
    leaf-cache-max-entries: 1024

Using PEM Files#

http-engine:
  enabled: true
  mitm:
    enabled: true
    ca-cert:
      file: ./certs/ca.pem
    ca-key:
      file: ./certs/ca.key
    hosts:
      - service.example.com

Using Inline PEM#

ca-cert:
  inline-pem: |
    -----BEGIN CERTIFICATE-----
    ...
    -----END CERTIFICATE-----
ca-key:
  inline-pem: |
    -----BEGIN PRIVATE KEY-----
    ...
    -----END PRIVATE KEY-----

Field effects:

Security note: MITM only the domains you need to debug. Do not enable it for the entire internet.

Capture#

http-engine:
  enabled: true
  capture:
    enabled: true
    max-flows: 1000
    body-preview-bytes: 4 KiB
    store-full-body: true
    full-body-max-bytes: 1 MiB
    spool-dir: ./http-capture

Field effects:

Shown in the Link1 App:

• Flow list: summary of recently captured requests/responses.

• Flow details: method, host, path, status code, matched rules, and duration.

• Body view: view the original request, rewritten request, original response, and rewritten response.

• Disk storage note: when the body exceeds the preview size or full saving is enabled, the App shows the saved location or truncation status.

These contents appear only when Capture is enabled and the traffic matches HTTP Engine.

Match Conditions#

All HTTP Engine rules have match. Fields:

Example:

match:
  scheme: [https]
  host: service.example.com
  path-regex: '^/old/'
  method: [GET, POST]
  header:
    X-Client: app

URL rewrite#

http-engine:
  rules:
    url-rewrite:
      - name: rewrite-service
        match:
          host: old.example.com
          path: /old/items
        action:
          type: rewrite-url
          replacement: https://new.example.com/new/items

Action types:

Header rewrite#

http-engine:
  rules:
    header-rewrite:
      - name: add-debug-header
        direction: request
        match:
          host: service.example.com
        operations:
          - op: set
            key: X-Debug
            value: '1'
          - op: del
            key: X-Remove-Me

Direction: request or response.

Operations:

Body rewrite#

http-engine:
  rules:
    body-rewrite:
      - name: patch-text
        direction: response
        match:
          host: service.example.com
          content-type: [text/plain]
        require-body: true
        max-size: 512 KiB
        on-error: fail-open
        operations:
          - op: replace
            from: old
            to: new
          - op: replace-regex
            pattern: 'token=[^&]+'
            replacement: 'token=redacted'

Field effects:

JSON transform#

http-engine:
  rules:
    json-transform:
      - name: normalize-json
        direction: response
        match:
          host: service.example.com
          content-type: [application/json]
        require-body: true
        max-size: 1 MiB
        when:
          - path: $.ok
            eq: true
        ops:
          - op: set
            path: $.source
            value: link1
          - op: set-if-missing
            path: $.items
            value: []
          - op: del
            path: $.debug

Rule fields:

when supports: path, eq, neq, in, not-in, exists.

Supported operations:

The array filter condition where supports field, eq, neq, and all.

JQ#

http-engine:
  rules:
    jq:
      - name: jq-response
        direction: response
        match:
          host: service.example.com
          content-type: [application/json]
        require-body: true
        max-size: 1 MiB
        timeout: 100ms
        on-error: fail-open
        expression: '.items |= map(. + {source: $source})'
        variables:
          source: link1

Field effects:

QuickJS script#

QuickJS script is suitable for HTTP rewrite logic that cannot be expressed by built-in rules but does not warrant a separate service, such as dynamically adding fields based on response JSON, directly mocking certain requests, or temporarily marking routes based on headers. It runs inside the Link1 kernel. It is not a browser plugin or a Node.js environment.

Script source#

Declare the script source in http-engine.scripts first, then reference it from rules. Script names must be unique.

http-engine:
  scripts:
    - name: patcher
      source:
        file: ./http-engine/scripts/patcher.js

Scripts can also be inline, which is suitable for very short rules:

http-engine:
  scripts:
    - name: inline-script
      source:
        inline: |
          function onResponse(payload) {
            return {
              headers: {
                "X-Link1": "1"
              }
            };
          }

Note: script functions should be declared directly as global functions, such as function onRequest(...) / function onResponse(...). Do not write export function, and do not assume a module system exists.

Referencing a script from a rule#

http-engine:
  rules:
    script:
      - name: run-patcher
        direction: response
        match:
          host: service.example.com
        engine: quickjs
        script: patcher
        require-body: true
        max-size: 1 MiB
        timeout: 50ms
        memory-limit: 8 MiB
        on-error: fail-open
        arguments:
          env: prod

Field effects:

Hook and payload context#

direction determines which hook is called:

function onRequest(payload) {
  // Handles only the request phase
}

function onResponse(payload) {
  // Can see both request and response
}

payload is an object constructed by Link1 for the current HTTP flow. Modifying payload itself in the script does not take effect automatically; you should return a rewrite result envelope. Do not return payload to "return as is"; otherwise, during the response phase, the current response may be treated as a synthesized response and rewritten again.

Body fields are not always present:

• When there is no body, body, bodyText, and bodyBytes are not present.

• When require-body: false and the body is streaming, Link1 does not force the stream to be fully read for the script, so the script cannot see the body.

• When the body exceeds max-size, require-body: false lets the script continue running without a body; require-body: true triggers the error policy.

• In non-binary mode, JSON bodies go into body; non-JSON text goes into bodyText.

• In binary mode, the body goes into bodyBytes, and the script sees a Uint8Array.

Return value envelope#

The script return value determines whether to rewrite the current flow.

Output headers should use string values. Input headers may be arrays of strings, but output arrays are formatted as ordinary values and are not suitable for representing multi-value headers.

Example: add a header by environment#

http-engine:
  scripts:
    - name: add-env-header
      source:
        inline: |
          function onRequest(payload) {
            return {
              headers: {
                "X-Link1-Env": payload.arguments.env || "dev"
              }
            };
          }
  rules:
    script:
      - name: add-env-header
        direction: request
        match:
          host: api.example.com
        script: add-env-header
        arguments:
          env: prod

Effect: requests sent to api.example.com are given X-Link1-Env: prod. This example does not need to read the body, so do not write require-body: true.

Example: modify a JSON response#

http-engine:
  scripts:
    - name: normalize-profile
      source:
        inline: |
          function onResponse(payload) {
            const body = payload.response.body;
            if (!body || typeof body !== "object") return;
            body.source = "link1";
            if (!Array.isArray(body.items)) body.items = [];
            delete body.debug;
            return {
              headers: {
                "Content-Type": "application/json"
              },
              body: body
            };
          }
  rules:
    script:
      - name: normalize-profile
        direction: response
        match:
          host: api.example.com
          path: /profile
          content-type: [application/json]
        script: normalize-profile
        require-body: true
        max-size: 1 MiB
        timeout: 50ms
        on-error: fail-open

Effect: the response is modified only when its body can be read as JSON. After you return body, Link1 serializes it as JSON. If you want to preserve the original indentation or non-JSON text, use bodyBytes.

Example: return a synthesized response#

http-engine:
  scripts:
    - name: mock-maintenance
      source:
        inline: |
          function onRequest(payload) {
            if (payload.arguments.maintenance !== "on") return;
            return {
              response: {
                status: 503,
                headers: {
                  "Content-Type": "application/json",
                  "Retry-After": "60"
                },
                body: {
                  ok: false,
                  message: "maintenance"
                }
              }
            };
          }
  rules:
    script:
      - name: maintenance-page
        direction: request
        match:
          host: api.example.com
          path: /v1/order
        script: mock-maintenance
        arguments:
          maintenance: "on"
        on-error: fail-closed

Effect: after a match, Link1 returns 503 directly to the client and no longer connects to api.example.com. This type of rule is suitable for testing and temporary degradation, but not as a long-term replacement for server-side logic.

Example: Binary body#

If you need to output exact text bytes or handle images, archives, or protobuf, enable binary-body-mode. The QuickJS environment does not provide TextEncoder. If you only need to debug ASCII text, you can use a small function to generate a byte array:

http-engine:
  scripts:
    - name: binary-debug
      source:
        inline: |
          function ascii(text) {
            const out = new Uint8Array(text.length);
            for (let i = 0; i < text.length; i++) out[i] = text.charCodeAt(i) & 255;
            return out;
          }

          function onResponse(payload) {
            const input = payload.response.bodyBytes;
            if (!(input instanceof Uint8Array)) return;
            return {
              headers: {
                "Content-Type": "text/plain"
              },
              bodyBytes: ascii("bytes=" + input.length)
            };
          }
  rules:
    script:
      - name: binary-debug
        direction: response
        match:
          host: download.example.com
        script: binary-debug
        binary-body-mode: true
        require-body: true
        max-size: 64 KiB

Effect: the response body is replaced with the raw text bytes bytes=<length>. When processing real binary data, keep match and max-size as narrow as possible to avoid reading large files into the script.

Debugging methods#

When debugging QuickJS scripts, follow this order:

1. Narrow match first: Match only one test domain, path, and method to avoid affecting real traffic.

1. Start without body: Confirm that Header rewriting works before enabling require-body.

1. Use fail-closed while debugging: If the script throws an error, the connection fails, and the App logs or connection error can show the rule name and error message. After confirming it works, switch back to fail-open.

1. Use throw new Error(...) for assertions: For example, if (!payload.response.body) throw new Error("missing json body").

1. Output state with temporary Headers: For example, return headers: {"X-Debug-Status": String(payload.response.status)}, then check it in Capture or browser developer tools.

1. Output debug information with synthetic responses: In the request stage, you can temporarily return {response: {status: 200, body: {...}}} to confirm exactly what is in the payload.

1. Enable Capture to compare before and after: Compare the original request/response with the rewritten request/response to determine whether match did not hit, the body was not read, or the returned envelope is incorrect.

The QuickJS environment does not provide console.log. If you write console.log(...) in a script, you will get ReferenceError: 'console' is not defined. Do not rely on fetch, the file system, DOM, browser objects, Node.js modules, setTimeout, TextEncoder, or an asynchronous event loop.

Runtime constraints and security boundaries#

• Each script execution uses an independent VM. Do not rely on global variables to preserve state across requests.

• Scripts run synchronously. Do not write logic that waits for networks, timers, or asynchronous callbacks.

• The body must be read into memory before entering the script. Use max-size to limit large bodies.

• memory-limit limits the script VM memory, not all overhead of the entire Link1 process.

• timeout is not a performance optimization tool; it is a fallback protection. The script itself should still be kept simple.

• QuickJS rules run at the proxy layer and can see and rewrite HTTP content. Do not retain passwords, Tokens, Cookies, private keys, or enterprise data in scripts or Capture for long periods.

Mock#

http-engine:
  rules:
    mock:
      - name: mock-health
        match:
          host: service.example.com
          path: /health
        response:
          status: 200
          headers:
            Content-Type: application/json
          body: '{"ok":true}'

Response body sources:

Route rules#

http-engine:
  rules:
    route:
      - name: mark-service
        match:
          host: service.example.com
        outbound: PROXY

HTTP Engine route is used to mark the service outbound for the current HTTP flow during layer-7 processing. For example, you can mark a request to PROXY based on the HTTP path, Header, or JSON/JQ/QuickJS result. It only affects HTTP flows that have entered HTTP Engine. For global, general outbound routing, it is still recommended to configure top-level rules, so DNS, TUN, non-HTTP traffic, and normal connections are handled consistently.

Replay#

Replay is used to replay captured HTTP flows for testing. Typical workflow:

1. Select a flow in the App's HTTP capture list.

1. Review the original request and the rewritten request to ensure no sensitive content is leaked.

1. Select the replay target: use the original target, or temporarily change it to a test target.

1. Run the replay and observe the status code, response headers, response body, and matched HTTP Engine rules.

Replay is suitable for validating URL rewrite, Header rewrite, JSON/JQ/QuickJS rewriting, and Mock. It may access real services again. Do not replay requests containing login state or business data casually.

Recommendations#

• Enable MITM only for hosts that need debugging.

• Set reasonable max-size and timeout values for body/JQ/script.

• The default on-error: fail-open is more suitable for production proxies, avoiding site-wide outages caused by rewrite rule failures.

• fail-closed is suitable for strict policy or test scenarios.

• Capturing full bodies in Capture consumes disk space. In production, limit max-flows and full-body-max-bytes.