SLURM sbatch job array for the same script but with different input string arguments run in parallel

Question

My question is similar with this one, and the difference is that my different arguments are not numbers but strings.

If I have a script (myscript.R) that takes two strings as arguments: "text-a", "text-A". My shell script for sbatch would be:

#!/bin/bash

#SBATCH -n 1
#SBATCH -c 12
#SBATCH -t 120:00:00
#SBATCH --partition=main
#SBATCH --export=ALL

srun ./myscript.R "text-a" "text-A"

Now I have a few different input strings that I'd like to run with:

first <- c("text-a","text-b","text-c","text-d")
second <- c("text-A","text-B","text-C","text-D")

and I want to run myscript.R with combinations of the texts, for example:

srun ./myscript.R "text-a" "text-A"
srun ./myscript.R "text-b" "text-B"
srun ./myscript.R "text-c" "text-C"
srun ./myscript.R "text-d" "text-D"

But if I put them in the same shell script, they'll run sequentially. I only know that I can use #SBATCH -a 0-10 when the arguments are index. If I want to submit the four scripts at the same time and each of them with the exact same settings (especially each one need to be assigned -c 12), how can I do that?

Thanks!

damienfrancois · Accepted Answer · 2017-04-28 13:47:14Z

4

You can store de list of argument values in an array and use the SLURM_ARRAY_TASK_ID env variable to index that array.

#!/bin/bash

#SBATCH -n 1
#SBATCH -c 12
#SBATCH -t 120:00:00
#SBATCH --partition=main
#SBATCH --export=ALL
#SBATCH --array=0-3

A=(text-{a..d}) # This is equivalent to A=(text-a text-b ... text-d)
B=(text-{A..D})

srun ./myscript.R "${A[$SLURM_ARRAY_TASK_ID]}" "${B[$SLURM_ARRAY_TASK_ID]}"

and simply submit it with sbatch.

answered Apr 28, 2017 at 13:47

damienfrancois

60.4k9 gold badges116 silver badges128 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

SLURM sbatch job array for the same script but with different input string arguments run in parallel

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related