How to access MongoDB from PyMongo when using separate docker-compose yaml's?

Question

I have two separate Docker containers, and separate docker-compose YAML's, too. One ('mongodb') for running the MongoDB, the other ('logger') for data scraping in Python. The latter should write some results into MongoDB.

I used separate yaml's to be able to stop easily one container while not stopping the other one.

To resolve this task I used docker-compose' bridge network capability. So I used the following two yaml's:

networks:
  wnet:
    driver: bridge

services:
  mongodb:
    image: mongo:4.0.9
    container_name: mongodb
    ports:
      - "27018:27017"
    volumes:
      - mongodb-data:/data/db
    logging: *default-logging
    restart: unless-stopped
    networks:
      - wnet

volumes:
  mongodb-data:
    name: mongodb-data

and

networks:
  wnet:
    driver: bridge

services:
  logger:
    build:
      context: .
    image:logger:$VERSION
    container_name:logger
    environment:
      - TARGET=$TARGET
    volumes:
      - ./data:/data
    restart: unless-stopped
    networks:
      - wnet

The Python container should now persist the scraped data within the MongoDB database. So I tried the following variants:

from pymongo import MongoClient

client = MongoClient(port=27018, host='mongodb')  # V1
client = MongoClient(port=27018)  # V2
db = client['dbname']

Then, executing one of the following commands throws the error:

db.list_collection_names()
db.get_collection('aaa').insert_one({ 'a':1 })

The response I get is

pymongo.errors.ServerSelectionTimeoutError: mongodb:27018: [Errno -2] Name or service not known

Any idea?

Thanks.

Your hostname does not resolve, maybe you still need to configure something for that. — D. SM
– D. SM, Commented Jun 22, 2020 at 15:22
@D.SM - okay - but what is the 'something' to configure, and how? — lambruscoAcido
– lambruscoAcido, Commented Jun 22, 2020 at 15:23
Now I went into the container and tested it manually. The error is thrown during the insert(), and not before. I will add the information to the post above. — lambruscoAcido
– lambruscoAcido, Commented Jun 22, 2020 at 15:27
kerneltalks.com/networking/how-docker-container-dns-works explains how it works, I don't use docker so don't know what exact steps you need to do. — D. SM
– D. SM, Commented Jun 22, 2020 at 15:35
If you are using docker compose then you should really have one compose file with both services; no need to create a network it does that for you; just reference by service name — Belly Buster
– Belly Buster, Commented Jun 22, 2020 at 17:16

lambruscoAcido · Accepted Answer · 2020-06-23 12:43:52Z

What finally worked is to refer to the network (defined in container mongodb) by its composed name (mongodb + wnet = mongodb_wnet), and to add the external option. This makes the YAML file of the logger container look like:

services:
  logger:
    build:
      context: .
    image: logger:$VERSION
    container_name: logger
    environment:
      - TARGET=$TARGET
    volumes:
      - ./data:/data
    restart: unless-stopped
    networks:
      - mongodb_wnet

networks:
  mongodb_wnet:
    external: true

However, as mentioned by @BellyBuster, it might be a good idea to use one single docker-compose file. I was not aware that it is quite easy to start, stop, and build single containers belonging to the same YAML.

SO has also enough posts on that, e.g. How to restart a single container with docker-compose and/or docker compose build single container.

Collectives™ on Stack Overflow

How to access MongoDB from PyMongo when using separate docker-compose yaml's?

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related