How To Setup Caddy As A Reverse Proxy

In the video below, we show you how to install and configure caddy as a reverse proxy

Containers are an extremely useful and efficient way of deploying applications

But you could end up with a quite a lot of them, each managed through their own web interface

In some cases the web session will have no encryption, otherwise a self-signed certificate will be provided

And this is where Caddy can help

It has several purposes but you can use it as a reverse proxy for your containers

With an encrypted session to Caddy itself, your web browser only needs to trust one certificate

So in this video we’ll go over how to install Caddy as a container and how to configure it as a reverse proxy for other containers

Useful links:
https://caddyserver.com/docs/running#docker%20compose
https://caddyserver.com/docs/quick-starts/reverse-proxy
https://caddyserver.com/docs/caddyfile
https://caddyserver.com/docs/caddyfile/directives/reverse_proxy
https://caddyserver.com/docs/caddyfile/directives/tls
https://caddyserver.com/docs/automatic-https#local-https

Assumptions:
Now because this video is about setting up Caddy as a reverse proxy, I’m going to assume you already have Docker installed, or you at least know how to install it

If not then I do have another video available that shows you how to install Docker in a VM running in Proxmox VE for instance

Overview:
Now even if you’re only interested in how to use Caddy, I do feel it’s very important to provide this overview of how reverse proxies work because they pose a security risk

In a simple world, when you’re connecting to a web server, your computer will connect directly to that server

Assuming there’s encryption being used, you’ll have what is known as end-to-end encryption

In other words, only your computer and the server sees the data being exchanged

Everything in between just sees encrypted information

But not all containers support encryption, and those that do will require certificate management, because it’s not advisable to use self-signed certificates

Why?

Because it puts you at greater risk of a man-in-the-middle attack

With a reverse proxy, you’ll connect to the reverse proxy and that will create its own connection to the web server and it can provide connectivity to many servers

So at a basic level, you’ll have a secure connection to the reverse proxy and so only one certificate to trust

After those connections are established, it proxies the data between the web browser and the web server

Now this is the same thing that happens with a man-in-the-middle attack, except you control the device in the middle, or at least you should

So you have to put a lot of trust in the developer of that reverse proxy not to leak or steal your data

Your web browser will encrypt the data it sends to the web server but the reverse proxy will then decrypt it

That’s because you aren’t directly connected to the web server

And depending on how it connects to the web server it may or may not re-encrypt that data before sending it on

So do bear in mind that the reverse proxy will see usernames, passwords, etc. as they pass between your computer and the server you intended to connect to

Another thing to bear in mind is a reverse proxy can give you a false sense of security

Let’s say you have a physical network switch and it doesn’t support encryption for management through a web interface

If you use Caddy to access that, while your connection to Caddy will be secure, the security risk remains the same, i.e. the traffic to and from that switch is still unencrypted, but that’s now between Caddy and the switch rather than your computer and the switch

So while your web browser is telling you the connection is secure, anybody that can monitor the traffic between Caddy and the switch will be able to see all the data being exchanged, including usernames and passwords

In that case you’d want to be more strategic as to where you deploy a reverse proxy as you’ll want it as close to the end device as possible and ideally to give it direct access or find some means to block other devices being able to see the unencrypted data

So do plan accordingly

But assuming you trust the developer, I feel that a reverse proxy is very useful for containers

Caddy for instance can be deployed as a container, so as long as the other containers you connect to are on the same host, a lack of encryption between Caddy and another container may not concern you too much because that traffic won’t leave the host

Caddy Container:
The first thing to do is to create a container for Caddy

I like to use Docker Compose, so we’ll edit an existing compose file

nano docker-compose.yml

volumes:
  caddy_data: {}
  caddy_config: {}

services:
  caddy:
    image: caddy:latest
    container_name: caddy
    restart: unless-stopped
    ports:
      - "443:443"
    volumes:
      - ./caddy/Caddyfile:/etc/caddy/Caddyfile
      - caddy_data:/data
      - caddy_config:/config

Now save and exit

As with most containers, Caddy needs some information to persist a reboot for instance, so we’re instructing Docker Compose to create some volumes

As usual, we want the latest version of the image for this container

Providing a name for the container is also useful as it makes it easier to reference this and identify it from the command line

And we want it to be always running, unless we manually stop it

As with most secure web services, the standard port is 443, so we’ll have the host expose this

Now there is one slight catch here. You can’t have two applications using the same IP address and port

Because a reverse proxy can simplify our lives, if you do have other containers listening on port 443, it’s better to try and change them or access them through Caddy

There is an alternative option, which involves putting Caddy or the other container on a different IP address to the host, but it would be more complicated

Another thing to bear in mind, is if you’re running Podman for instance, port 443 is a privileged port that normally requires root access. So there would be additional work to run this in rootless mode

The configuration file needs to survive a reboot, so we’ll map that to a file that will be stored on the host

We then map its data and config folders to the volumes we’re creating

Reverse Proxy:
Now in this video we want Caddy to act as a reverse proxy, specifically for containers on the same host

First though we’ll create our Caddyfile to tell Caddy about a web server we want it to proxy

We’ll create the data folder for this container

mkdir caddy

We then need a configuration file, referred to as the Caddyfile, and what you put in that depends on the use case

nano caddy/Caddyfile

whoami.homelab.lan {
	tls internal
	reverse_proxy whoami:80
}

Now save and exit

NOTE: The formatting is sensitive and it requires tabs not spaces

In this example, we want to connect to a web server that we’ll reference in the browser as whoami.homelab.lan, in other words it’s Fully Qualified Domain name (FQDN)

NOTE: Your computer will need to be able to resolve this to an IP address either through DNS or its hosts file

We want the session to Caddy to be secure, so we’ll use TLS and we want Caddy to provide the certificate using its own internal Certificate Authority (CA)

The container we’ll create will be called whoami and it will be listening on port 80

TIP: Container platforms like Docker provide their own DHCP and DNS service so usually you can reference a container by its name

TIP: After you make a change to the Caddyfile you can either restart the container or instruct it to reload the config instead

docker compose exec -w /etc/caddy caddy caddy reload

Next, we’ll edit our compose file and we’ll create our web server container at the end of the services section

nano docker-compose.yml

services:
...
  whoami:
    image: traefik/whoami
    container_name: whoami
    restart: unless-stopped

Now save and exit

We’ll use an image that is actually from the developers of another popular reverse proxy, Traefik Proxy

We haven’t exposed any ports for this container because we want users to access it via Caddy

So the application in the container is still listening on port 80, but the host isn’t making this available to the rest of the network

We’ll then start the whoami container

docker compose up -d whoami

Then we’ll check the container is working

docker ps -l

Next, we’ll start Caddy

docker compose up -d caddy

And then we’ll check that container is working

docker ps -l

Now our web browser needs to trust TLS certificates that are signed by Caddy’s Intermediate CA

That has a certificate signed by its own Root CA, so we need a copy of the root certificate to import into our web browser

To obtain this we’ll copy it from the container to the host

docker compose cp caddy:/data/caddy/pki/authorities/local/root.crt caddy/root.crt

The challenge then is how to transfer this file to your client computer

For me, I just use the clipboard to copy and paste the contents into a new file I create on the other computer

Browsers typically don’t use the root store anymore, so you’ll likely need to import this file into your browser as a trusted CA

How you do that depends on the browser but typically there will be a Privacy and Security option and in there maybe a Security or Certificates option. Further down the rabbit hole you’ll eventually find an option to import an Authority and at that point you can import this file

If we now browse to https://whoami.homelab.lan we should see a secure connection because the web browser trusts the certificate

So what we have here is a secure session to Caddy, which in turn has an insecure and unencrypted session to the web server

Now this makes administration much simpler, because Caddy will automatically create a new certificate for each web server we ask it to proxy, and it will keep these up to date

In addition, our web browser will trust these certificates because it trusts the Root CA

And this will save us a lot of time

Backend Encryption:
Now there is one slight problem that I see with Caddy and this it do with the backend connection

While we’re improving security for containers on the same host that don’t support encryption, what about containers that do support TLS?

Well there’s a problem because Caddy won’t trust a self-signed certificate and the connection will fail without intervention

One option might be to disable security on the backend container and use HTTP instead, but that doesn’t seem ideal

Another is to disable certificate verification on Caddy, which is similar to ignoring the warning in a web browser when it tells you it doesn’t trust a certificate

So for example, lets say we have a server that supports HTTPS on port 443. In that case, we could have an entry in the Caddyfile a bit like this

apache.homelab.lan {
        tls internal
        reverse_proxy https://apache {
                transport http {
                        tls_insecure_skip_verify
                }
        }
}

It’s similar to what we did before, but this time we want to use HTTPS for the connection from Caddy to the web server, but we won’t validate the certificate

Although this works, it isn’t ideal either because there is a risk of a man-in-the middle attack

Now as unlikely as that may seem on an internal network, it is still feasible, particularly if we use Caddy as a reverse proxy for applications on other hosts or for stand alone devices

As part of best practice, if a web server supports HTTPS, we should give it a certificate that Caddy trusts

Now it is possible for Caddy to act as an ACME server, so it could provision the certificates itself, but that’s out of scope for this video, and as ever it depends

Not all devices support ACME and even those that do, may be limited to only working with Public solutions like Let’s Encrypt

It may also need additional software installing, but that may not be possible on standalone devices like network switches for instance

For me it’s something that needs further thought

So for now, I’ll only be using Caddy for containers on the same host

The traffic between the client and Caddy will be encrypted, and the traffic between Caddy and the other container will be on the host

Realistically, if a container or the host is compromised, it’s game over anyway

Networking:
Now there is one last thing to do for completeness and that is to create a separate network

While the whoami container doesn’t have any ports exposed to the external network, we can improve security a bit more

Docker creates virtual networks and by default, all containers will be attached to a default bridge created for the user

As a result, all containers will have direct access to each other

The goal here is to access containers like this via Caddy, so we’ll create a separate network

In which case, we’ll edit the compose file

nano docker-compose.yml

  caddy:
    ...
    networks:
      proxy:

  whoami:
    ...
    networks:
      proxy:

networks:
  proxy:

Now save and exit

First we update the two containers with a network entry which will connect them to a network we’ll call proxy, but you can call this whatever you like

So when the caddy container talks to the one we’ve called whomai, it will now do that over this new network

Then we’ll tell Docker Compose to create this new network

Now you can customise a network more, but for this video we’ll keep things simple

For the changes to take effect we’ll have to stop the containers

docker stop whoami && docker stop caddy

Then a new network needs to be created and the containers rebuilt

docker compose up -d

The we’ll check things are working

docker ps

TIP: You can check what containers are attached to a bridge/network by running a command like this

docker network inspect dockermgr_proxy

NOTE: The network name includes the user name you’ve logged in as and isn’t just what you called it

Now you can and probably should create other separate networks for your containers and you can join Caddy to those as well because a container isn’t limited to one network

For instance, I have a lot of containers based around Prometheus

They all need access to Prometheus or vice versa, so it makes sense to put them in their own network

But any other containers on the host would be in a different network

Similarly, if you have a layered application, you would have a web container that connects to a web network and an application network, an application container connected to the application and database networks, and a database container connected to the database network

So users connect to the web server, that in turn accesses the application on another network, which in turn accesses the database on another network

All of those network layers exist to make it harder to get unauthorised access to the data

Summary:
As long you’re dealing with containers running on the same host as Caddy then I think this can save you a lot of time and effort

When containers are on separate hosts though, things get a bit tricky

So you could install Caddy on multiple container platforms, and for security reasons I do run separate Docker instances in different network segments

This provides you with a secure session to each host and the backend traffic remains on the host

From what I’ve seen, you can have other instances of Caddy obtain their CA certificate from a master one. This way your web browser only needs to trust one root certificate

But for me, the jury’s still out when it comes to encryption between Caddy and separate devices

If a standalone device doesn’t support encryption, you’ll still have unencrypted traffic going over the network

It’s better than no encryption at all, but you could be lulled into a false sense of security if the unencrypted traffic goes over a path that can be monitored by a 3rd party

On the other hand, if the end device does support encryption their could be a challenge getting Caddy to provide it with a certificate that Caddy trusts

In which case, you might still have to manually administer certificates on your devices and so it won’t really save you time

Sharing is caring!