Python script can't access some environment variables

Question

I am building Docker Image FROM node:8.9.3-alpine (which is Debian) and then running it as usual and passing parameters like this:

docker run -dt \
-e lsRegion=${bamboo_lsRegion} \
-e lsCluster=${bamboo_lsCluster} \

Then inside that container I am exporting some variables and when I echo them, I can see proper value

export lsEnv=${lsEnv:-'dev'}

Later in scripts I run python script and when I run the print(os.environ) I can see all the variables from docker run like lsRegion but I do not see the newly exported one like lsEnv.

I already found and tried to solve with this: Python: can't access newly defined environment variables by calling the source ~/.bashrc but I cannot find that file.

I have tried

~/.bashrc
/etc/bash.bashrc
/root/.bashrc

But neither of those exist (also does not know if this solve my problem) and it ends with this error message /app/deploy.sh: source: line 16: can't open '/root/.bashrc'

More reproducible example:

Dockerfile

FROM node:8.9.3-alpine

RUN apk add --no-cache \
    python \
    py-pip \
    ca-certificates \
    openssl \
    groff \
    less \
    bash \
    curl \
    jq \
    git \
    zip \
    build-base \
  && pip install --no-cache-dir --upgrade pip awscli \
  && aws configure set preview.cloudfront true

ENV TERRAFORM_VERSION 0.11.10

RUN wget -O terraform.zip https://releases.hashicorp.com/terraform/${TERRAFORM_VERSION}/terraform_${TERRAFORM_VERSION}_linux_amd64.zip && \
    unzip terraform.zip -d /usr/local/bin && \
    rm -f terraform.zip

RUN apk -v --update add python py-pip
RUN pip install --upgrade awscli
RUN pip install --upgrade boto3

COPY ./build.variables /app/build.variables
COPY ./aws/taskdef/template.json /app/template.json
COPY ./deploy.sh /app/deploy.sh
COPY ./deploy.py /app/deploy.py
COPY ./terraform /app/terraform

CMD ["sh", "/app/deploy.sh"]

deploy.sh

#!/bin/bash -x

cd /app/terraform
./run-terraform.sh
cd ..
python /app/deploy.py

terraform/run-terraform.sh

   ...
   export lsEnv="NotThere"
   ...

python script

#!/usr/bin/env python
import os
print(os.environ)

The print will show lsRegion or lsCluster but it will not show the lsEnv

Create a ~/.bashrc file yourself and add your exports to it. Then source this file. — max
– max, Commented Nov 13, 2018 at 14:35
@max And if I dont have it, what happens with exported variables? They are just living in a memory? — libik
– libik, Commented Nov 13, 2018 at 14:36
If .bashrc contains calls to export, then sourcing it does the same thing as running the same export commands manually. — chepner
– chepner, Commented Nov 13, 2018 at 14:39
How exactly do you start the container, run the export command, and run the Python script. It sounds like you aren't starting the Python script from the environment you think you are. — chepner
– chepner, Commented Nov 13, 2018 at 14:41
You probably call the Python script from another context than where you exported the variables. So adding them to ~/.bashrc should fix that. — max
– max, Commented Nov 13, 2018 at 14:41

chepner · Accepted Answer · 2018-11-14 13:30:37Z

1

Inside deploy.sh, you need to source run-terraform.sh if you want to affect the environment of the process that runs deploy.py, rather than the environment created for the process that runs run-terraform.sh.

#!/bin/bash -x

cd /app/terraform
source ./run-terraform.sh
cd ..
python /app/deploy.py

(You could also use . ./run-terraform.sh; source is a more readable bash synonym for the POSIX . command, but . is necessary if you are using some other POSIX-compliant shell that doesn't support source.)

answered Nov 14, 2018 at 13:30

chepner

538k77 gold badges594 silver badges746 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

libik Over a year ago

I have encounter another problem. When I do this, it fails with ./run-terraform.sh: line 41: syntax error: bad substitution which is

terraform apply -var "environment=$lsSpecificEnv" -var "environment_vpc=${lsEnv^}" -var "region=$lsRegion" -var "service=$SERVICE_NAME" -var "path_pattern=/v*/$SERVICE_NAME" -auto-approve

. I dont understand whats going on as both files (deploy and run-terraform) starts with #!/bin/bash -x anyway. Does source change the execution?

libik Over a year ago

I can confirm that this ${lsEnv^} does work without source when running the script and when I run it with source it stops working with that error

chepner Over a year ago

You are either using an older version (pre-4.0) version of bash, or more likely sh does not refer to bash in Alpine Linux, but some other POSIX shell that doesn't support ${...^} syntax. (In that case, source itself probably won't work, but the shell doesn't get as far as trying to run the source command before the parameter expansion fails.)

chepner Over a year ago

With CMD ["sh", "/app/deploy.sh"], the shebang is irrelevant; sh is used to run the script, ignoring the shebang.

libik Over a year ago

actually I used source and it works (look to my answer), when I dont use source the line is executed properly, but if its behind source, it does not. Really weird. However I solve it with CMD ["bash", "/app/deploy.sh"] thanks for noticing that.

libik · Accepted Answer · 2018-11-14 11:15:29Z

0

I solve it by calling this command in terraform/run-terraform.sh for each environment variable I will need in python script:

echo "export lsTargetGroup=$lsTargetGroup" >> ~/.bashrc

And then in deploy.sh I just add source ~/.bashrc before calling python script

answered Nov 14, 2018 at 11:15

libik

23.1k10 gold badges52 silver badges94 bronze badges

Collectives™ on Stack Overflow

Python script can't access some environment variables

2 Answers 2

5 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

5 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related