Console logging and error messages in n8n sometimes include very little useful information for troubleshooting http requests that a workflow sends to one of the integrated services. Fortunately, most (if not all) of the http requests n8n sends, use Axios, and Axios can be configured with environment variables to use a proxy for http and/or https. Then, mitmproxy can be run in a docker container to forward those requests and capture/display the details in an easy-to-use web UI.

Prerequisites

  • n8n running in Docker, preferably started using docker-compose

Goal

Set up mitmproxy to run in Docker, and configure n8n to use mitmproxy for https/http requests.

Steps (overview)

  • Set up mitmproxy to run in docker (using docker compose)
  • Adjust the runtime configuration (docker-compose.yml) of n8n to include environment variables that instruct Axios to use mitmproxy.

mitmproxy docker-compose.yml

This docker-compose.yml file was adapted from this example, but...

  • The DNS parts were removed
  • The networks were adjusted to actual RFC 1918 private networks.
    • 172.201.. and 172.202.. overlap public, "real" address ranges and are blocked by default in mitmproxy.
    • Reported here
  • Some other adjustments (e.g. tty: true) were made so that the current Python-based mitmproxy code will start correctly in the docker container.
  • A default password is specified for the mitmproxy web UI so it won't be necessary to watch the console for a generated password.

docker-compose.yaml

services:
  mitmproxy:
    image: mitmproxy/mitmproxy
    hostname: mitmproxy
    restart: always
    tty: true
    volumes:
      - ./mitmproxy_data:/home/mitmproxy/.mitmproxy
      # - './cert:/root/.mitmproxy'
    entrypoint: mitmweb
    command: '--web-host 0.0.0.0 -m regular@80 -m regular@443 --set web_password=changeme123'
    networks:
      mitm_internal:
        ipv4_address: 192.168.115.80
      mitm_external:
        ipv4_address: 192.168.116.80
    ports:
      - '38080:80/tcp'
      - '38443:443/tcp'
      - '38081:8081/tcp'

networks:
  mitm_internal:
    name: mitm_internal
    internal: true
    ipam:
      driver: default
      config:
        - subnet: 192.168.115.0/24
          gateway: 192.168.115.1
  mitm_external:
    name: mitm_external
    ipam:
      driver: default
      config:
        - subnet: 192.168.116.0/24
          gateway: 192.168.116.1

Adjustments to the n8n docker-compose.yml

  • Add the docker network to the n8n service/container 
services:
  n8n:
    ...
    networks:
      - mitm_external
...
networks:
  mitm_external:
    name: mitm_external
    external: true
  • Add environment variables to tell Axios to use mitmproxy
    • Note: These are not n8n environment variables, but are used directly by the [Axios] npm library/module.
    • Note: https requests could also be routed through the https listener on mitmproxy, but that potentially introduces certificate related issues. 
services:
  n8n:
    ...
    environment:
      ...
      - "HTTPS_PROXY=http://mitmproxy:80"
      - "HTTP_PROXY=http://mitmproxy:80"

Using mitmproxy with n8n

After starting the mitmproxy container and restarting the n8n container to pick up changes to networks and environment variables...

  • Open the n8n editor to a workflow which sends out http/s requests.
  • Open the mitmproxy web UI in another browser or browser-tab.
    • http://localhost:38081
    • Log in with the web_password specified in the mitmproxy docker-compose.yml ("changeme123" if you didn't modify it).
  • Execute the workflow (or http based workflow step), and the mitmproxy web UI should begin showing the requests as they are sent.
  • Click on one of the request items to see details about the request, response, connection, etc.

Alternative to "Global" Proxy Config

Instead of setting the HTTPS_PROXY and/or HTTP_PROXY environment variables, which sends all Axios traffic through the proxy, some n8n nodes, like the HTTP Request node, have a Proxy option that can be explicitly routed to mitmproxy. For nodes that support this:

  • Add the Proxy option in the node configuration
  • Set the target proxy URL to http://mitmproxy

Links