Empty files in container, using docker-compose with volumes to init image with existing files

Question

I am trying to init container with existing configurations files I have. The files that I have are nginx.conf and ssl.conf. The volumes configuration is set like this:

volumes:
      - ./nginxletsencrypt_config:/config
      - ./nginxConfig/nginx.conf:/config/nginx/site-confs/default
      - ./nginxConfig/ssl.conf:/config/nginx/ssl.conf

This result is "default" and ssl.conf files, that have root:root owner and are even marked as executable. If I put only the first row , and I comment out the two other rows for nginx.conf and ssl.conf, then the image has those two files with their default content inside. The full docker compose file is:

version: '2'
services:
    nginx:
        restart: always
        image: linuxserver/letsencrypt
        container_name: nginx
        ports:
          - 443:443
          - 80:80
        cap_add:
          - NET_ADMIN
        environment:
          - HOST_HOSTNAME=prod
          - PUID=1001 # get on dockerhost through command "id <user>"
          - PGID=1002
          - TZ=Europe/London
          - URL=domainname.com
          - VALIDATION=http
          - [email protected]
          - STAGING=false
        volumes:
          - ./nginxletsencrypt_config:/config
          - ./nginxConfig/nginx.conf:/config/nginx/site-confs/default
          - ./nginxConfig/ssl.conf:/config/nginx/ssl.conf

I use linuxserver/letsencrypt in order to have ssl certificate. I managed to run everything and the certificate and nginx work, after I put manually the nginx.conf content inside the "default" file. Docker compose version

docker-compose version
docker-compose version 1.18.0, build 8dd22a9

docker version
Client:
 Version:      17.05.0-ce
 API version:  1.29
 Go version:   go1.7.5
 Git commit:   89658be
 Built:        Thu May  4 22:10:54 2017
 OS/Arch:      linux/amd64

Server:
 Version:      17.05.0-ce
 API version:  1.29 (minimum version 1.12)
 Go version:   go1.7.5
 Git commit:   89658be
 Built:        Thu May  4 22:10:54 2017
 OS/Arch:      linux/amd64
 Experimental: false

=============================================

After updating docker to last version and testing on clear enviroment(no older docker previously installed):

krasen@test:~$ docker version
Client:
 Version:           18.09.7
 API version:       1.39
 Go version:        go1.10.8
 Git commit:        2d0083d
 Built:             Thu Jun 27 17:57:09 2019
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          18.09.7
  API version:      1.39 (minimum version 1.12)
  Go version:       go1.10.8
  Git commit:       2d0083d
  Built:            Thu Jun 27 17:23:02 2019
  OS/Arch:          linux/amd64
  Experimental:     false

krasen@test:~/test$ pwd
/home/krasen/test
krasen@test:~/test$ cat docker-compose.yml 
version: '2'
services:
  nginx:
    restart: always
    image: linuxserver/letsencrypt
    container_name: nginx
    ports:
      - 443:443
      - 80:80
    cap_add:
      - NET_ADMIN
    environment:
      - HOST_HOSTNAME=prod
      - PUID=1001 # get on dockerhost through command "id <user>"
      - PGID=1002
      - TZ=Europe/Paris
      - URL=test.net
      - VALIDATION=http
      - [email protected]
    user: ${CURRENT_UID}
    volumes:
      - ./nginxletsencrypt_config:/config
      - ./nginxConfig/nginx.conf:/config/nginx/site-confs/default
      - ./nginxConfig/ssl.conf:/config/nginx/ssl.conf

krasen@test:~/test$ ls 
docker-compose.yml  Dockerfile  nginxConfig  nginxletsencrypt_config
krasen@test:~/test$ ls nginxConfig/
nginx.conf  ssl.conf
krasen@test:~/test$ ls -la nginxConfig/
total 16
drwxrwxr-x 2 krasen krasen 4096 Jul  8 09:06 .
drwxrwxr-x 4 krasen krasen 4096 Jul  8 09:09 ..
-rw-rw-r-- 1 krasen krasen 1500 Jul  8 09:06 nginx.conf
-rw-rw-r-- 1 krasen krasen 1405 Jul  8 09:06 ssl.conf
krasen@test:~/test$ ls -la nginxletsencrypt_config/
total 48
drwxrwxr-x 11 krasen krasen 4096 Jul  8 09:07 .
drwxrwxr-x  4 krasen krasen 4096 Jul  8 09:09 ..
drwxr-xr-x  2 krasen krasen 4096 Jul  8 09:07 crontabs
-rw-rw-r--  1 krasen krasen    0 Jul  8 09:07 .directory
drwxr-xr-x  2 krasen krasen 4096 Jul  8 09:07 dns-conf
-rw-r--r--  1 krasen krasen  204 Jul  8 09:07 donoteditthisfile.conf
drwxr-xr-x  3 krasen krasen 4096 Jul  8 09:07 etc
drwxr-xr-x  4 krasen krasen 4096 Jul  8 09:07 fail2ban
drwxr-xr-x  2 krasen krasen 4096 Jul  8 09:07 keys
drwxr-xr-x  6 krasen krasen 4096 Jul  8 09:07 log
drwxrwxr-x  4 krasen krasen 4096 Jul  8 09:07 nginx
drwxr-xr-x  2 krasen krasen 4096 Jul  8 09:07 php
drwxrwxr-x  2 krasen krasen 4096 Jul  8 09:07 www
krasen@test:~/test$ ls -la nginxletsencrypt_config/nginx/
total 44
drwxrwxr-x  4 krasen krasen  4096 Jul  8 09:07 .
drwxrwxr-x 11 krasen krasen  4096 Jul  8 09:07 ..
-rw-rw-r--  1 krasen krasen   424 Jul  8 09:07 dhparams.pem
-rw-rw-r--  1 krasen krasen  4140 Jul  8 09:07 ldap.conf
-rw-rw-r--  1 krasen krasen  1832 Jul  8 09:07 nginx.conf
-rw-rw-r--  1 krasen krasen  1019 Jul  8 09:07 proxy.conf
drwxrwxr-x  2 krasen krasen 12288 Jul  8 09:07 proxy-confs
drwxrwxr-x  2 krasen krasen  4096 Jul  8 09:07 site-confs
krasen@test:~/test$ ls -la nginxletsencrypt_config/nginx/site-confs/
total 8
drwxrwxr-x 2 krasen krasen 4096 Jul  8 09:07 .
drwxrwxr-x 4 krasen krasen 4096 Jul  8 09:07 ..
krasen@test:~/test$ docker-compose build 
nginx uses an image, skipping
krasen@test:~/test$ docker-compose -f docker-compose.yml up -d
Creating network "test_default" with the default driver
Pulling nginx (linuxserver/letsencrypt:latest)...
latest: Pulling from linuxserver/letsencrypt
2430498be9ad: Pull complete
ab781ac10f2e: Pull complete
365f82032c47: Pull complete
c3805ea8a362: Pull complete
6e0559b48e8a: Pull complete
5ecfcf110062: Pull complete
7aee6d88249e: Pull complete
Digest: sha256:9ad0d0d48a9f7e784569cbecef95b753a87133574140f9e3ee790dc9ee7f3d8a
Status: Downloaded newer image for linuxserver/letsencrypt:latest
Creating nginx ... done
krasen@test:~/test$ ls ./nginxletsencrypt_config/
crontabs/               dns-conf/               etc/                    keys/                   nginx/                  www/                    
.directory              donoteditthisfile.conf  fail2ban/               log/                    php/                    
krasen@test:~/test$ ls ./nginxletsencrypt_config/nginx/
dhparams.pem  ldap.conf  nginx.conf  proxy.conf  proxy-confs  site-confs  ssl.conf
krasen@test:~/test$ ls -la ./nginxletsencrypt_config/nginx/
total 44
drwxrwxr-x  4 krasen krasen  4096 Jul  8 09:12 .
drwxrwxr-x 11 krasen krasen  4096 Jul  8 09:07 ..
-rw-rw-r--  1 krasen krasen   424 Jul  8 09:07 dhparams.pem
-rw-rw-r--  1 krasen krasen  4140 Jul  8 09:07 ldap.conf
-rw-rw-r--  1 krasen krasen  1832 Jul  8 09:07 nginx.conf
-rw-rw-r--  1 krasen krasen  1019 Jul  8 09:07 proxy.conf
drwxrwxr-x  2 krasen krasen 12288 Jul  8 09:07 proxy-confs
drwxrwxr-x  2 krasen krasen  4096 Jul  8 09:12 site-confs
-rwxr-xr-x  1 root   root       0 Jul  8 09:12 ssl.conf
krasen@test:~/test$ cat  ./nginxletsencrypt_config/nginx/ssl.conf
krasen@test:~/test$ cat ./nginxConfig/ssl.conf
## Version 2018/05/31 - Changelog: https://github.com/linuxserver/docker-letsencrypt/commits/master/root/defaults/ssl.conf

# session settings
ssl_session_timeout 1d;
ssl_session_cache shared:SSL:50m;
ssl_session_tickets off;

# Diffie-Hellman parameter for DHE cipher suites
ssl_dhparam /config/nginx/dhparams.pem;

# ssl certs
ssl_certificate /config/keys/letsencrypt/fullchain.pem;
ssl_certificate_key /config/keys/letsencrypt/privkey.pem;

# protocols
ssl_protocols TLSv1.2 TLSv1.3;
ssl_prefer_server_ciphers on;
ssl_ciphers '............................................................................................................................................................';

# HSTS, remove # from the line below to enable HSTS
add_header Strict-Transport-Security "max-age=63072000; includeSubDomains; preload" always;

# OCSP Stapling
ssl_stapling on;
ssl_stapling_verify on;

krasen@test:~/test$

Here user: ${CURRENT_UID} is 1001:1002, but with this and without this still the ssl.conf file is empty file after running "sudo cat ssl.conf" inside the ~/test/nginxletsencrypt_config/nginx directory

=================New edit

I saw the log of nginx and there was this

nginx    | nginx: [emerg] host not found in upstream "web:8000" in /config/nginx/site-confs/default:2

means the the "default" file is not empty. Then the file was copied. But why I cannot list it with

cat ./nginxletsencrypt_config/nginx/site-confs/default

or

sudo cat ./nginxletsencrypt_config/nginx/site-confs/default

Cant' reproduce with docker-compose version 1.23.2. Volumes work as intended and files are there when container is started. Is there any particular reason why you have omitted version from top of docker-compose file? Also ensure that host's paths in volume section are valid(ex typos). — leopal
– leopal, Commented Jul 8, 2019 at 6:54
@TarunLalwani root:root is not a problem if the files are copied correctly with their content inside. Currently they are empty. I guess that the root user is the problem. Probably those files are created first from the container with root owner, and later writing to those files is run as another user. Although I don't understand why files are created first empty, and not copied directly with their content. — makkasi
– makkasi, Commented Jul 8, 2019 at 7:27
@makkasi, I have tried on Ubuntu 16.04 and not able to reproduce your issue, see the screenshot i.sstatic.net/ewRtu.png — Tarun Lalwani
– Tarun Lalwani, Commented Jul 9, 2019 at 5:52

Tarun Lalwani · Accepted Answer · 2019-07-09 07:17:06Z

So I was able to reproduce your issue

$ cat nginxletsencrypt_config/nginx/site-confs/default

$ ls -ialh nginxletsencrypt_config/nginx/site-confs/default
23518410 -rwxr-xr-x  1 tarunlalwani  staff     0B Jul  9 12:37 nginxletsencrypt_config/nginx/site-confs/default

$ ls -ialh nginxConfig/nginx.conf
23492731 -rw-rw-r--@ 1 tarunlalwani  staff    22B Jul  8 13:14 nginxConfig/nginx.conf

$ docker-compose exec nginx sh

root@9d6d58328d33:/# ls -alhi config/nginx/site-confs/default
23492731 -rw-rw-r-- 1 abc abc 22 Jul  8 08:44 config/nginx/site-confs/default
root@9d6d58328d33:/#

As you can see the inodes for the one inside the nginxletsencrypt_config is different and hence it is a blank file. What my hunch is that this because of the layers of the filesystem within the container.

Is it a bug in docker? Possibly yes, or it could be a special case which requires an even deeper understanding as to why it happens

Collectives™ on Stack Overflow

Empty files in container, using docker-compose with volumes to init image with existing files

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related