Nested Parallel loop gives index was outside the bound while adding value to a list

Question

I have a List-A to populate List-B with more value that comes from database and populates another List-C. I'm using parallel loop and normal foreach loop. If I use just foreach loop, it takes about 19 minutes to populate List-C, but using parallel loop it only takes 3 minutes to populate same list.

But for some reason, when adding List-B to List C, it gives out an error "Index was outside the bounds of the array", with no informations

List<Variance> Variances = new List<Variance>();
Parallel.ForEach(words, (word) =>
{
     List<wordConfig> configlist = new List<wordConfig>();
     foreach (VariancePopulationConfig value in values)
     {
         VariancePopulationConfig config = new VariancePopulationConfig(Named, Category, Schedule);

         using (SqlConnection con = new SqlConnection(ConnectionString))
         {
             con.Open();
             SqlCommand cmd = new SqlCommand();
             cmd.Connection = con;
             cmd.Parameters.AddWithValue("@word", word);
             cmd.Connection = con;
             cmd.CommandType = CommandType.Text;
             cmd.CommandText = value.SelectQuery;
             returnedvalue = cmd.ExecuteScalar().ToString();
             config.DestinationColOrdinal = value.DestinationColOrdinal;
             config.CurrentOrHistory = value.CurrentOrHistory;
             config.WordValue = returnedvalue.Equals("0") ? string.Empty : returnedvalue;
             config.Word = word;
         }
         configlist.Add(config);
     }
     Variances.Add(new Variance { Word = word, VarianceConfigs = configlist });
});

Variances.Add is not thread safe inside parallel foreach. consider using lock statement or concurrent collections like concurrent bag — M.kazem Akhgary
– M.kazem Akhgary, Commented Nov 19, 2015 at 2:08
No more error, but some values are populating at random places for some of the items. — messed-up
– messed-up, Commented Nov 19, 2015 at 5:22

Sergey Kalinichenko · Accepted Answer · 2015-11-19 16:42:24Z

You get an out-of-bound error because you are adding items to a growing list from multiple threads. One thread tries to expand the array, while the other is accessing the old one.

You can fix this problem by replacing Parallel.ForEach with AsParallel and Select, like this:

var Variances = words.AsParallel().Select((word) => {
     List<wordConfig> configlist = new List<wordConfig>();
     foreach (VariancePopulationConfig value in values) {
         VariancePopulationConfig config = new VariancePopulationConfig(Named, Category, Schedule);
         using (SqlConnection con = new SqlConnection(ConnectionString)) {
             con.Open();
             SqlCommand cmd = new SqlCommand();
             cmd.Connection = con;
             cmd.Parameters.AddWithValue("@word", word);
             cmd.Connection = con;
             cmd.CommandType = CommandType.Text;
             cmd.CommandText = value.SelectQuery;
             returnedvalue = cmd.ExecuteScalar().ToString();
             config.DestinationColOrdinal = value.DestinationColOrdinal;
             config.CurrentOrHistory = value.CurrentOrHistory;
             config.WordValue = returnedvalue.Equals("0") ? string.Empty : returnedvalue;
             config.Word = word;
         }
         configlist.Add(config);
     }
     return new Variance { Word = word, VarianceConfigs = configlist };
}).ToList();

No more error, but some values are populating at random places for some of the items.

By default parallel select does not preserve original ordering. You can fix this by adding .AsOrdered() after AsParallel(), but there is a chance that the performance would degrade.

Collectives™ on Stack Overflow

Nested Parallel loop gives index was outside the bound while adding value to a list

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related