19

I'm struggling with gitlab ci/cd variables. I see so many conflicting examples. Anyhow, what I would like to know is how to use variables outside and within scripts.

For example, in a job config, can I assign a variable in the script section with a bash command?

some-job:
   variables:
      SOME_VAR: ''
   script:
      - SOME_VAR = $(<file_with_a_line_of_text.txt)

In the above case, I'm not sure if I can do this. But I need to populate a variable with the file contents (i.e. artifact). Also, when do I use '$' in front of the variable? Some examples I see using these formats:

"SOME_VAR" #in quotes, no dollar sign
"${SOME_VAR}" #in quotes, with dollar sign and wrapped with curly braces
${SOME_VAR} #no quotes, with dollar sign and wrapped with curly braces
$SOME_VAR #i.e. without the double quotes or curly braces
SOME_VAR #i.e. without the double quotes, dollar sign, and curly braces

So many variations of usage that I can see in examples but don't really know when to use each style. And I can't find one example online of a custom variable being set in a script using a bash command.

1
  • 1
    Maybe you should start with a shell introduction? Commented Mar 30, 2021 at 10:08

2 Answers 2

12

When I'm setting variables in bash, I always do it without the spaces around the =:

VAR1="some string"
VAR2=23
VAR3=true
VAR4=$(cat /path/to/file.txt)

Let's go through these examples one at a time:

  1. You can set a variable as a string by using quotes around the string.
  2. You can set it to an int (probably a float too, but haven't personally used it)
  3. You can set it to a bool
  4. You can set it to the output of a command. The command is inside the command: $(#command).

Now let's use them:

echo $VAR1
# some string
echo "This is my variable $VAR1"
# This is my variable some string
echo "This is my variable ${VAR1}"
# This is my variable some string
echo ${VAR1}
# some string
echo "Error code ${VAR2}A"
# Error code 23A
echo "Error code $VAR2A"
# Error code --- Note: the variable $VAR2A dosn't exist
echo "Error code ${VAR2}${VAR1}"
# Error code 23some string
echo VAR1
# VAR1
echo "VAR1"
# VAR1

This illustrates the difference between the different forms, but in general, you reference a variable's value with $+variable-name. Doing "SOME_VAR" or SOME_VAR just prints out the string "SOME_VAR" (ie, not referencing a variable at all).

The difference between $SOME_VAR and ${SOME_VAR} is that the latter lets you use it when there is other content directly before or after the variable without erroring.

Sign up to request clarification or add additional context in comments.

Comments

8

How to use custom variables in gitlab ci/cd?

Normally like in any other shell.

But note that gitlab-ci.yml is a yaml file and yaml has special parsings. Because of that in script: ex. - echo bla is the same as - 'echo bla', because in yaml the content of script: is an array of strings that are later spitted by shell.

how to use variables outside and within scripts.

Normally like in any other shell script.

when to use each style

"SOME_VAR" #in quotes, no dollar sign
SOME_VAR #i.e. without the double quotes, dollar sign, and curly braces

when you want to have a string SOME_VAR literally

"${SOME_VAR}"

is the same as "$SOME_VAR". When you want to have the content of SOME_VAR variable literally.

${SOME_VAR} #no quotes, with dollar sign and wrapped with curly braces
$SOME_VAR #i.e. without the double quotes or curly braces

When you want the content of SOME_VAR variable after word splitting and filename expansion. That means that SOME_VAR='*' and then echo "$SOME_VAR" will print *, but echo $SOME_VAR will print all files in current directory. You usually always want to quote expansions.

The form ${SOME_VAR} is used if concatenated with some other string, ex. $SOME_VARbla is not ${SOME_VAR}bla.

Do not use upper case variables in your scripts - prefer lower case. Prefer using upper case variables for exported variables. Be aware of clashes - COLUMN PATH USER UID are examples of already used variables.

can I assign a variable in the script section with a bash command?

Shell is space aware. var = val will execute a command named var with two arguments = and val. var=val will assign the string val to variable named var. Do:

- SOME_VAR=$(<file_with_a_line_of_text.txt)

In gitlab-ci I would prefer to use cat in case I will want to move to alpine. $(< is a bash extension.

- SOME_VAR=$(cat file_with_a_line_of_text.txt)

There doesn't seem to be any point in setting providing SOME_VAR in environment with variables: SOME_VAR.

When do I use '$' in front of the variable?

When you want to trigger variable expansion. Variable expansion substitutes variable name for the variable value.

Check your scripts with http://shellcheck.net . Read https://mywiki.wooledge.org/BashGuide a good shell introduction and https://wiki.bash-hackers.org/scripting/obsolete .

6 Comments

So regarding gitlab ci/cd, if I declare a variable in .gitlab-ci.yml under variables, that variable should be accessible to all jobs in each stage, correct? Is there a way to change the contents of that variable later in a job, so that another job gets the change? What I'm struggling with now is that I declare the var, change it in a job (I also echo it to see that it has changed), then when I use it in another job in the same stage, it has the original value and not the new one.
that variable should be accessible to all jobs in each stage, correct Depends. Doing variables: at "root scope" will export the variable to all jobs. Doing variables: at job scope will affect only that one job. See gitlab documentation. Is there a way to change the contents of that variable later in a job sure, just var=value. so that another job gets the change? No, each job is it's own process. Even each job is run in it's own environment. Tbh, I fear such question is a sign of misunderstanding how docker works and processes work - be sure to research these topics.
What I'm struggling with now is that I declare the var, change it in a job (I also echo it to see that it has changed), then when I use it in another job in the same stage, it has the original value and not the new one Use artifacts to transfer context between stages. In bash, you can use declare -p to serialize variable value, and then just source the file to deserialize it's value.
The specific problem I'm dealing with is DAST, and the api scanner is expecting the variable DAST_API_SPECIFICATION to point to the api spec. The problem is that I'm trying to use an artifact to build that api url, but only a script will be able to assign to that variable. Example: DAST_API_SPECIFICATION=$(cat api_url.txt)/swagger.json, but this doesn't work because the var needs to be declared first.
If I don't use DAST_API_SPECIFICATION, it tries to look for a /zap/wrk directory and fails because it doesn't exist. I don't want to use custom zap endpoints for this one
|

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.