Using bash, I have an indexed array that is a list of files:

a=("1.json" "2.json" "3.json" ... "5309.json")

I am parsing two data fields from those JSON files into two associative arrays:

declare -A idArr
declare -A valueArr

for i in "${a[@]}"; do
    jqId="$(jq -M ".fileId" <"${i}")"
    jqValue="$(jq -M ".value" <"${i}")"
    # If there are already items in the associative array, add the new items separated by a newline
    idArr[${i}]="${idArr[${i}]}${idArr[${i}]:+$'\n'}${jqId}"
    valueArr[${jqId}]="${valueArr[${jqId}]}${valueArr[${jqId}]:+$'\n'}${jqValue}"
done

Because I'm iterating through one file at a time, it takes some time to get through all the files. But I need the associative arrays created in the loop to exist outside of the loop/after the loop is completed. Is there a way, using parallel or some other method, that I could concurrently run a bunch of these array items and still have them add data to the associative arrays?

Using Bash, I have an indexed array that represents a list of files:

a=("1.json" "2.json" "3.json" ... "5309.json")

I am parsing two data fields from those JSON files into two associative arrays:

declare -A idArr
declare -A valueArr

for i in "${a[@]}"; do
    jqId="$(jq -M ".fileId" <"${i}")"
    jqValue="$(jq -M ".value" <"${i}")"
    # If there are already items in the associative array, add the new items separated by a newline
    idArr[${i}]="${idArr[${i}]}${idArr[${i}]:+$'\n'}${jqId}"
    valueArr[${jqId}]="${valueArr[${jqId}]}${valueArr[${jqId}]:+$'\n'}${jqValue}"
done

Since I'm iterating through one file at a time, it takes a considerable amount of time to process all the files. I require the associative arrays created within the loop to persist beyond its scope, even after the loop has finished. 

Is there a method, such as using parallel processing or any other approach, that would allow me to concurrently process multiple array items and still enable them to contribute data to the associative arrays?

Using bash, I have an indexed array that is a list of files:

a=("1.json" "2.json" "3.json" ... "5309.json")

I am parsing two data fields from those JSON files into two associative arrays:

declare -A idArr
declare -A valueArr

for i in "${a[@]}"; do
    jqId="$(jq -M ".fileId" <"${i}")"
    jqValue="$(jq -M ".value" <"${i}")"
    # If there are already items in the associative array, add the new items separated by a newline
    idArr[${i}]="${idArr[${i}]}${idArr[${i}]:+$'\n'}${jqId}"
    valueArr[${idArr[${i}]}]="${valueArr[${idArr[${i}]}]}${valueArr[${idArr[${i}]}]:+$'\n'}${jqValue}"
done

Because I'm iterating through one file at a time, it takes some time to get through all the files. But I need the associative arrays created in the loop to exist outside of the loop/after the loop is completed. Is there a way, using parallel or some other method, that I could concurrently run a bunch of these array items and still have them add data to the associative arrays?

Using bash, I have an indexed array that is a list of files:

a=("1.json" "2.json" "3.json" ... "5309.json")

I am parsing two data fields from those JSON files into two associative arrays:

declare -A idArr
declare -A valueArr

for i in "${a[@]}"; do
    jqId="$(jq -M ".fileId" <"${i}")"
    jqValue="$(jq -M ".value" <"${i}")"
    # If there are already items in the associative array, add the new items separated by a newline
    idArr[${i}]="${idArr[${i}]}${idArr[${i}]:+$'\n'}${jqId}"
    valueArr[${jqId}]="${valueArr[${jqId}]}${valueArr[${jqId}]:+$'\n'}${jqValue}"
done

Because I'm iterating through one file at a time, it takes some time to get through all the files. But I need the associative arrays created in the loop to exist outside of the loop/after the loop is completed. Is there a way, using parallel or some other method, that I could concurrently run a bunch of these array items and still have them add data to the associative arrays?