Reverse Proxies

Aug 26, 2020

A reverse proxy is a server application that sits in front of one or more web servers and forwards client requests to those web servers. Reverse proxies can be used for (but are not limited to):

  • offloading web server TLS encryption
  • reducing server loads by caching static and dynamic content
  • optimizing web content with compression
  • adding basic HTTP access authentication to web servers
  • allowing traffic to multiple web servers though a single IP address

For this post I will set up a reverse proxy on an Ubuntu 20.04 server using a Nginx Docker container with TLS encryption provided by a self-signed certificate. Nginx is a web server that serves a large chunk of websites on the internet and can also be used as a reverse proxy. Make sure to change example.com to your preferred server name and server_IP to your own server’s IP address. I will use Jellyfin as an example of an application to self host. See previous posts on Docker and Docker Compose for an installation and configuration refresher.

Note: self-signed certificates are not sufficient for services open to the public internet. Use this reverse proxy setup only for local networks.

Setup

I have created a github repo for this project as there are a fair number of configuration files and directories involved. First download the repo and generate a self-signed certificate using OpenSSL. For an in depth explanation, see the OpenSSL post:

$ cd ~/ && git clone https://github.com/petebuffon/reverse_proxy.git
$ cd reverse_proxy/nginx && mkdir keys
$ openssl req \
  -x509 -nodes -days 365 \
  -subj '/C=US/ST=State/L=City/CN=www.example.com' \
  -newkey rsa:2048 -keyout ./keys/mycert.pem -out ./keys/mycert.pem

The default Nginx container image will automatically run as root and this is a bad idea. If you need further convincing, RedHat has a great write up on container security.

The default.conf and nginx.conf config files require some alterations to allow Nginx be run as a non root user. The default listening port needs to be changed from 80 to 8080 in default.conf as unprivileged users can’t use ports under 1000. The directory paths in nginx.conf need to be changed from /var/run to /tmp/nginx.

Nginx logo

Nginx uses a text-based configuration file written in a specific format. By default this file is named nginx.conf and is located in the /etc/nginx directory. The configuration file consists of directives separated by whitespace from their parameters. Single-line directives each end with a semicolon. Multi-line directives, called blocks, are enclosed in curly braces.

By default nginx.conf adds all files ending with .conf in the /etc/nginx/conf.d directory to its configuration. This is done with the following line in nginx.conf:

include /etc/nginx/conf.d/*.conf;

There are a few top-level directives, called contexts, that each refer to a different traffic type. In this project we will only be dealing with the http context. Within a http context, one or more server blocks are included that define virtual servers. Here is a simple example of a Nginx configuration file, example.com.conf, that serves static content:

server {
  listen 127.0.0.1:80;
  server_name example.com;

  root /www/data;

  location / { 
  }
}

In this example, the Nginx web server is serving traffic on the local host IP (127.0.0.1) on port 80, clients access the web server at the domain example.com, the root of the web server is /www/data, and the root of /www/data is served.

In general, contexts are inherited from one level to another, parent to child. Some directives can appear in multiple contexts, and in this case you can override the parent context by also including the directive in a child context.

Compression

Compression can easily be enabled in nginx.conf to reduce the size of transmitted data. Here we are enabling gzip compression for specific file types when they are over 1024 bytes in size. See the official Nginx documentation on compression for more info:

...
gzip on;
gzip_disable msie6;
gzip_proxied no-cache no-store private expired auth;
gzip_types text/plain text/css application/x-javascript application/javascript text/xml application/xmapplication/xml+rss text/javascript image/x-icon image/bmp image/svg+xml;
gzip_min_length 1024;
gzip_vary on;
gunzip on;
...

Dockerfile

In the custom dockerfile we are simply copying the custom config files into a new container image, adding a new user nginx with uid 2000, and assigning the proper permissions to the new nginx user:

FROM nginx

COPY ./nginx.conf /etc/nginx/nginx.conf
COPY ./conf.d /etc/nginx/conf.d
COPY ./keys /etc/nginx/keys

RUN userdel nginx \
    && mkdir /tmp/nginx \
    && useradd -u 2000 nginx \
    && chown -R nginx:nginx /etc/nginx /var/cache/nginx /tmp/nginx

EXPOSE 8080

STOPSIGNAL SIGTERM

USER nginx:nginx

CMD ["nginx", "-g", "daemon off;"]

Configs

If we take a look at our project’s conf.d directory, there are four config files: defalt.conf, ssl.conf, proxy.conf, and jellyfin.conf. Separating out config files by role makes it easier to make changes down the line. Additionally, config options in ssl.conf and proxy.conf will not need to be rewritten for every virtual server.

As the config files are copied into the container image at build time, changes made to config files will not be seen until the container image is rebuilt and recreated.

ssl.conf

The lines from ssl.conf were copied almost verbatim from Mozilla’s SSL Configuration Generator using the modern configuration option:

ssl_certificate /etc/nginx/keys/mycert.pem;
ssl_certificate_key /etc/nginx/keys/mycert.pem;

ssl_session_timeout 1d;
ssl_session_cache shared:SSL:10m;
ssl_session_tickets off;

ssl_protocols TLSv1.3;
ssl_prefer_server_ciphers off;

add_header Strict-Transport-Security "max-age=63072000" always;

Note: OCSP Stapling is not needed as a self-signed certificate is being used.

proxy.conf

The proxy.conf file sets some good defaults for proxying connections. The proxy_set_header directive alters headers seen by the client. Setting Host to the host variable ensures the correct domain name is set for the server. Setting X-Real-IP to the variable remote_addr and setting X-Forwarded-For to the proxy_add_x_forwarded_for variable allows the upstream server to see the original IP address of a connecting client:

proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

Note: If any proxy settings are altered in an application config file, all proxy directives will need to be redefined as seen in jellyfin.conf.

Jellyfin logo

Jellyfin is a free and open source media server alternative to Emby and Plex. I use linuxserver.io’s image for Jellyfin, but Jellyfin also provides its own image. The jellyfin.conf file was derived from Jellyfin’s documentation.

jellyfin.conf contains two locations, a root location and a websocket location with additional proxy headers required for websockets. Port 8080 traffic is redirected to the TLS encrypted 8443 port, which is upgraded to http version 2.0.

In both locations the variable upstream is set to the Jellyfin’s container name at port 8096. Setting the resolver to Docker’s DNS IP, 127.0.0.11, ensures Nginx is able to find the Jellyfin server. The proxy is then set with proxy_pass to the upstream variable. If application container went down, the reverse proxy would go down as well without this workaround:

server {
  listen 8080;
  server_name jellyfin.example.com;
  return 301 https://$host$request_uri;
}

server {
  listen 8443 ssl http2;
  server_name jellyfin.example.com;
  resolver 127.0.0.11 valid=30s;

  location / {
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_set_header X-Forwarded-Protocol $scheme;
    proxy_set_header X-Forwarded-Host $http_host;
    set $upstream http://jellyfin:8096;
    proxy_pass $upstream;
  }

  location /socket {
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_set_header X-Forwarded-Protocol $scheme;
    proxy_set_header X-Forwarded-Host $http_host;
    set $upstream http://jellyfin:8096/socket;
    proxy_pass $upstream;
  }
}

Docker Compose

Here is the docker-compose.yml file for the project:

version: '2'
services:
  reverse_proxy:
    build:
      context: ./nginx
    container_name: reverse_proxy
    environment:
      # change to your timezine
      - TZ=America/Los_Angeles
    ports:
      - 80:8080
      - 443:8443
    networks:
      - default
      - public
    restart: unless-stopped
    
  jellyfin:
    image: linuxserver/jellyfin
    container_name: jellyfin
    environment:
      # change to owner of media files
      - PUID=4000
      - GUID=4000
      # change to your timezone
      - TZ=America/Los_Angeles
    ports:
      # discovery port
      - 1900:1900
    volumes:
      - jellyfin_config:/config
      #/pathtomedia:/data
    # https://jellyfin.org/docs/general/administration/hardware-acceleration.html
    # devices:
      # - /dev/dri:/dev/dri
    restart: unless-stopped

volumes:
  jellyfin_config:

networks:
  default:
  public:

Now let’s finally get everything up:

docker-compose build --pull && docker-compose pull && docker-compose up -d

Name Resolution

Lastly we’ll have to add a line to a client’s /etc/hosts file in order to reach the reverse proxy by server name:

...
jellyfin.example.com    server_IP
...

Editing /etc/hosts on every client computer is your only option unless you’re hosting your own DNS server.

I use pfSense for my router at home so I just had to add a host override. First navigate to the pfSense web GUI. From the top menu select Services / DNS Resolver General Settings. Go to the host override section. Enter example.com in Domain and server_IP in IP Address. In Additional Names for this host, enter jellyfin in Host and example.com in Domain. Click Save and Apply Changes.

You should now be able to immediately navigate to jellyfin.example.com. You will have to accept the self-signed certificate.

pfSense host override

There you have it. Using one application, Jellyfin as an example, it is easy to generate Nginx config files for many other applications. Most simple applications will require a few proxy header directives and possibly some additional proxy header directives for websockets. A lot of projects will also include documentation for using reverse proxies with their product.