Category: DevOps

TLS Communication Error In a Docker Container

Problem

This weekend, I came across with an awkward error that I was not able to access https urls or ssl/tls based services while I was able to access non-tls based services over tcp without any problem inside a container.

First, I taught it was a TLS communication error and start digging around container image base, configurations, ca certificates and etc… But nothing worked. I was right it was a TLS communication error but not at the process level. Before I got into this error, I set up Wireguard Vpn which I added recently into my setup. So I stop working around wiring new container files and changed my way to other part of the setup.


Solution

The setup that my docker daemon was running on a host which was configured with Wireguard VPN. So I started troubleshooting on the network stack and boom…

~# ip link
X: wg0: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1420 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/none
X: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN mode DEFAULT group default
    link/ether XX:XX:XX:XX:XX:XX brd ff:ff:ff:ff:ff:ff

The problem was because of different MTU sizes on the docker and wg* interfaces.

The Docker Daemon does not check the MTU size of the any available network interfaces. Therefore, the value of the Docker MTU was set to 1500. When I change MTU size of docker0 interface to 1420 that matches the wireguard default MTU size, everything started working again.

Wireguard Docker Repository

It is also mentioned under others section on the repository.

For some clients (a GL.inet router in my case) you may have trouble with HTTPS (SSL/TLS) due to the MTU on the VPN. Ping and HTTP work fine but HTTPS does not for some sites.

This can be fixed with MSS Clamping. This is simply a checkbox in the OpenWRT Firewall settings interface.

https://github.com/cmulk/wireguard-docker

You can find more information on wireguard-docker repository.

Docker Daemon Configuration

/etc/docker/daemon.json
{
  "mtu": 1420
}

Restart the docker daemon and we are all set. If the output of MTU size doesn’t update suddenly, make sure you have run a container after the configuration.

Continuous Integration & Delivery vs Deployment

 

Continuous Integration

 

It is the practice that requires developers to integrate the code into a shared repository at least once a day. The integration is followed up by a build and test steps to solve problems quickly. Then you can fix the problems easily before they get serious. It would be hard to find and fix them on the repository where the developers don’t integrate their changes so often. When they do, they will be able to add new features instead of spending more time on debugging. Everyone can see what is happening on the repository. Unit and Integration tests have important roles on that pipeline. Overall, reduced risks, release more often and provide higher quality products.

“Continuous Integration doesn’t get rid of bugs, but it does make them dramatically easier to find and remove.”

— Martin Fowler, Chief Scientist, ThoughtWorks

Continue reading