How to define and use environment variables within strings in dockerfile?

Question

Our directory contains dags and plugins subdirectories. We have been trying for an hour to make this Dockerfile work :

...
# Set AIRFLOW_HOME
ENV AIRFLOW_HOME="/root/airflow"

# Create airflow directory to populate it with dags
RUN mkdir -vp $AIRFLOW_HOME
## Import dags
ADD dags '${AIRFLOW_HOME}/dags'
ADD plugins '${AIRFLOW_HOME}/plugins'
...

No matter what we tried, the dags and plugins subdirectories would just not be created under the specified path. Until we did this :

ADD dags ${AIRFLOW_HOME}'/dags'
ADD plugins ${AIRFLOW_HOME}'/plugins'

I.E. Removed the ${AIRFLOW_HOME} from under the string brackets... Why is this behaving this way? Could you provide an explanation of what we're doing wrong?

Antonio Petricca · Accepted Answer · 2021-06-15 14:28:09Z

1

The main error is that the ADD source folder must be relative to docker file context path, so you cannot use absolute paths:

The second error is that you cannot pass environment variables to some docker commands, so you have to use ARG.

Here an example:

ARG AIRFLOW_HOME /root/airflow
WORKDIR ${AIRFLOW_HOME}

## Import dags

ADD dags {{dags target folder under WORKDIR}}
ADD plugins {{plugins target folder under WORKDIR}}

ANd remember to place dags and plugins source folder under the docker context path.

edited Jun 15, 2021 at 14:28

answered Jun 15, 2021 at 14:22

Antonio Petricca

11.3k6 gold badges47 silver badges91 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Imad Over a year ago

Oh so I have to define a WORKDIR and use it. Thnx !

David Maze Over a year ago

Since the container's filesystem layout is fixed at build time, I wouldn't even use an ARG here. Set WORKDIR /root/airflow, and then use relative paths on the right-hand side of ADD (or, better, COPY): COPY dags ./dags.

Collectives™ on Stack Overflow

How to define and use environment variables within strings in dockerfile?

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related