If you use docker and docker-compose then you might have come across cases where you want services running across different networks to interact. Typically this need arises when we use a single database container like postgres and we want other containers created using different docker-compose files to use the database container. It can become really confusing and difficult to setup. I got stuck recently with a similar issue and resolved it. Hope it might be useful for you as well.
postgres database container running in the default bridge network. Initialised using command:
docker run -d --name postgres postgres
Running a service with 2 containers –
redis. Web runs the web server. Web server uses redis for queing jobs. That’s why it’s linked in links option of the docker-compose file. We have a postgres database running separately on the default network.
*Problem: Docker-compose creates a separate network for all the containers defined in the compose file. Which means if you want to connect any external container with these services its a problem. *
Option1 : Use the default docker bridge
default docker bridge is a network present on all Docker hosts. If you do not specify a different network, new containers are automatically connected to the default bridge network. We did not specify any network in the
postgres container that we created and hence it would use the
default docker bridge. If we run containers mentioned in our docker-compose file in this default network they would be able to communicate with each other. Below is a docker-compose file for that.
version: '3' services: web: build: . image: some-image ports: - "3001:3000" dns: "18.104.22.168" volumes: - ".:/app" env_file: .env links: - redis:redis external_links: - postgres1 # In production instead add external_links for db command: bash -c "rake assets:precompile & rake db:create && rake db:migrate && rails s" network_mode: bridge redis: image: redis:latest network_mode: bridge
Assuming you have initialised your postgres database using the command
docker run -d --name postgres postgres then it would be created in the default docker bridge network. Using the
network_mode: bridge option both web and redis containers are attached to the default docker network and hence web, redis and postgres are all part of the same network and thats why the external link would work without any issues.
Drawbacks: This works fine but this has a potential problem. Assuming you are going to run 30-40 services like this each running multiple containers then all those services have containers running in same network. So technically you are not getting network level isolation.
Option 2 : Use an external network
The idea here is to create an external network and connect containers wanting to interact with each other to that network. For example we have a
web container and
postgres container both running on different networks but they want to talk to each other. We can attach multiple networks with a paritcular container and hence if we create a new network
testnetwork and attach it with both
postgres containers then they would be able to interact. This way
postgres would be able to talk but
redis would not be able to interact with
postgres which is what we need too.
Before running the below compose file, you must attach postgres container also to the
testnetwork network which we are going to create.
docker network create testnetwork
docker network connect testnetwork postgres
version: '3' services: web: build: . image: some-image ports: - "3001:3000" dns: "22.214.171.124" volumes: - ".:/app" env_file: .env links: - redis:redis external_links: - postgres1 command: bash -c "rake assets:precompile & rake db:create && rake db:migrate && rails s" networks: - testnetwork redis: image: redis:latest networks: testnetwork: external: true # We are not attaching redis to the testnetwork because it does not need access to the postgres database
Benefits of Option1 over Option2: in Option 2 our
redis container is running in an isolated network and hence can’t be reached from outside. Option2 allows us isolation. In Option1 all containers can connect to each other because they are all part of the same network.
NOTE: I am not an expert on this topic, I’ve posted the solution which worked for me when I was stuck. Each method has its own pros and cons and you are requested to look them up carefully before using them in production setup.