Scaling Out with Docker Overlay Networking
In our previous article, we explored setting up a 3-node cluster using Docker Swarm. Building on that setup, let’s enhance our network capabilities by running a Docker overlay network on top of our existing cluster. This will allow us to achieve even greater efficiency and connectivity between containers distributed across multiple Docker hosts.
Docker overlay networks provide a powerful method for connecting containerized services and applications across multiple Docker daemons. This feature is especially useful in a multi-host Docker Swarm environment, where you need to ensure containers can communicate securely and efficiently. Here’s how to set up and scale services using Docker’s overlay networking.
Introduction to Docker Overlay Networking
Overlay networking in Docker allows containers connected to the same network to communicate, regardless of which host they are running on. This network model provides encapsulation and isolation for containers, making it ideal for scaling out applications and services across a cluster.
Setting Up Your Docker Overlay Network
Step 1: Create an Overlay Network
Start by creating an overlay network named myoverlaynet:
docker network create --driver overlay myoverlaynet
This command sets up the overlay network, enabling containers on this network to communicate across multiple Docker hosts.
Step 2: Deploy Services to the Network
Deploy a PostgreSQL Database
Launch a PostgreSQL service connected to your overlay network:
docker service create --name psql --network myoverlaynet -e POSTGRES_PASSWORD=mypass postgres
Verify the Service
Check that the PostgreSQL service is running:
docker service ps psql
You should see it active on one of the nodes, typically node1.
Step 3: Check Logs
Confirm that the database is ready by checking the logs:
docker container logs <container_name>
Look for a message indicating that the database system is ready to accept connections.
Step 4: Deploy a Drupal Web Server
Deploy a Drupal service on the same network:
docker service create --name drupal -p 80:80 --network myoverlaynet drupal
Check that the Drupal container is running, possibly on node2:
docker service ps drupal
docker service ls
Step 5: Inspect Network Configuration
Ensure both services are on the myoverlaynet network:
docker network inspect myoverlaynet
This should show a JSON output with both containers listed under the same network.
Understanding Routing Mesh and Load Balancing
Docker utilizes a routing mesh to route ingress (incoming) packets for a service to the proper task. This system spans all nodes in the swarm and uses IPVS from the Linux kernel.
Key Features of Docker’s Routing Mesh:
- Load Balancing: Automatically distributes service requests across all tasks in the swarm.
- Virtual IP (VIP): Acts as a stable front-end for clients to the swarm service. Containers talk to the VIP, not directly to each other, enhancing security and abstraction.
Testing Load Balancing
Deploy an Elasticsearch service with three replicas to see the routing mesh in action:
docker service create --name search --replicas 3 --network myoverlaynet -p 9200:9200 elasticsearch:2
Execute curl commands against the service:
curl localhost:9200
You will observe different responses, demonstrating load balancing across nodes.
Conclusion
Overlay networking and the routing mesh are critical components of Docker’s networking capabilities, allowing for scalable and efficient application deployment across multiple hosts. By using these features, you can ensure seamless communication and load balancing for your containerized applications.
