home-lab/docker/ingress.md

Ingress

cloudflared

To set up zero-inbound traffic, we'll use Cloudflare tunnels which will also let us handle some security aspects (WAF, DDoS absorption etc.), things like caching on Cloudflare's CDN and SSL termination. SSL termination is particularly handy as it lets Cloudflare handle all HTTPS traffic, in turn letting us do all internal server routing over HTTP and not setup things like Let's Encrypt or Certbot. This tunnel will be our only public facing ingress point also reducing attack surface. To set this up, we'll use a cloudflared container to connect our tunnel to Cloudflare. Note: Tunnels can only handle a few protocols. cloudflared will then redirect all traffic to a reverse proxy (nginx in our case) to handle all routing and load balancing. With Cloudflare tunnels, we have 2 choices to manage them: through Cloudflare's web dashboard or through a local config.yml file. We'll opt for the local config file to keep everything in line as much as possible with IaC guidelines and for maximum transparency.

CLI

Installation

To manage the tunnel locally, we'll need to install the cloudflared CLI tool.

1. Make a directory (if not existent) for GPG keys: sudo mkdir -p --mode=0755 /etc/apt/keyrings.
2. Download the key for cloudflared: curl -fsSL https://pkg.cloudflare.com/cloudflare-main.gpg | sudo tee /etc/apt/keyrings cloudflare-main.gpg >/dev/null.
3. Add cloudflare to list of apt sources: echo "deb [signed-by=/etc/apt/keyrings/cloudflare-main.gpg] https://pkg.cloudflare.com/cloudflared noble main" | sudo tee /etc/apt/sources.list.d/cloudflared.list.
4. Install with apt: sudo apt update && sudo apt install -y cloudflared.
5. Last, log in : cloudflared tunnel login.

Tunnel creation

To create a tunnel simply run:

cloudflared tunnel create mytunnel

This should give you a tunnel ID and generate a credentials file at ~/.cloudflared/<TUNNEL_ID>.json. Next, we'll move those credentials to our new folder dedicated to our cloudflared container (cf. docker/notes.md for more information about our filesystem):

mkdir -p /opt/appdata/cloudflared
mv ~/.cloudflared/<TUNNEL_ID>.json /opt/appdata/cloudflared/

DNS updates

For the tunnel to work, we need to update Cloudflare's DNS to point our domain (example.com) to our tunnel:

cloudflared tunnel route dns mytunnel "*.example.com" #for wildcard subdomains
cloudflared tunnel route dns mytunnel "example.com" #for root domain

Container

Config

Like mentioned before, we want our config to be completely local. For this, we create a local config file:

nano /opt/appdata/cloudflared/config.yml

The config file looks something like this:

tunnel: TUNNEL_ID
credentials-file: /etc/cloudflared/TUNNEL_ID.json

ingress:
  - hostname: "*.example.com"
    service: http://nginx:80

  - hostname: "example.com"
    service: http://nginx:80

  - service: http_status:404

Note: the credentials-file is /etc/cloudflared/TUNNEL_ID.json and not /opt/appdata/cloudflared/TUNNEL_ID.json, the reason is the config file is based on where it is mounted inside the container. This will makes sense in the next section. Here the routing is done to nginx but can be to whatever we want.

Compose file

The compose file for our cloudflared container looks something like this:

services:
  cloudflared:
    image: cloudflare/cloudflared:latest
    container_name: cloudflared
    restart: unless-stopped
    command: tunnel --config /etc/cloudflared/config.yml run
    depends_on:
      - nginx
    volumes:
      - /opt/appdata/cloudflared:/etc/cloudflared:ro
    networks:
      - default

nginx

While we could let the cloudflared container do our routing directly, we'll use a dedicated reverse proxy for good practice. That reverse proxy will also provide LAN access to our services if needed and provide clean log generation. Common reverse proxies are: Caddy, nginx, nginx proxy manager (NPM) and Traefik. I didn't do super in depth research on each but here we'll go with nginx for now. Because NPM is a bit resource heavy (with the GUI + sqlite DB) and doesn't integrate well with CrowdSec we won't be using here. Traefik doesn't always offer the best performance so we'll forget it too for now. Now Caddy and nginx are both lightweight, have great integration with CrowdSec and performance metrics. We go with nginx because it is more customisable and has native integration with CrowdSec. Like I said I did not do in depth research and comparison and this may be a mistake but for now, only time will tell.

Compose file

To get our nginx container running, we use the following compose.yml file:

services:
  nginx:
    image: nginx:alpine
    container_name: nginx
    restart: unless-stopped
    volumes:
      - /opt/appdata/ingress/nginx/conf.d:/etc/nginx/conf.d:ro
    networks:
      - default
      - frontend-net

networks:
  frontend-net:
    external: true

Now before, running this we need to create the network frontend-net. This network will host our reverse proxy and all our other frontend, public facing apps such as gitea. For this reason we have to specify external: true.

Config file

As you can see in the above config file, nginx relies on config files located in the nginx/conf.d directory. Then, let's create it:

mkdir -p /opt/appdata/ingress/nginx/conf.d

Now we add a default config file:

nano /opt/appdata/ingress/nginx/conf.d/default.conf

The following config file relies on an already running gitea container that is attached to the frontend-net network. The config is:

upstream gitea_frontend {
    server gitea:3000; # <docker_container_name>:<internal_port>
    keepalive 32;
}

server {
    listen 80;
    server_name gitea.example.com; # for me this would gitea.pvdm.dev

    # Increase payload limit for large Git pushes (Nginx default is 1M)
    client_max_body_size 512M;

    # Custom log format per vhost
    access_log /var/log/nginx/gitea_access.log;
    error_log  /var/log/nginx/gitea_error.log warn;

    location / {
        # Proxy to upstream block
        proxy_pass http://gitea_frontend;

        # Force HTTP/1.1 for connection reuse and WebSockets
        proxy_http_version 1.1;

        # Preserve HTTP request headers for backend visibility
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;

        # Pass WebSocket handshake headers dynamically
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";

        # Proxy timeouts
        proxy_connect_timeout 60s;
        proxy_read_timeout 3600s; # Long timeout for SSH-over-HTTP / large git syncs
        proxy_send_timeout 3600s;
    }
}