C# InvokeAsync in foreach only invoking Lambda one time

Question

I have a lambda in C# with a simple foreach loop. The foreach loop invokes a python lambda for each item in the List..

foreach (NOAAQInfo noaaqInfo in batchItemsForMyPython)
{
    Console.WriteLine($"invoking MyPython api for: {noaaqInfo.Email}");

    Task<NOAAQInfo> t = SendToMyPython.executeLambda(dbLogic, noaaqInfo, context);
    Task tCleanup = t.ContinueWith((antecedent) =>
    {
        Console.WriteLine($"MyPython success: {antecedent.Result.Email}");
    });
}

Here is the function that actually invokes the Python Lambda.

internal static async Task<NOAAQInfo> executeLambda(DBLogic dbLogic, NOAAQInfo noaaqInfo, ILambdaContext context)
{
    AmazonLambdaClient amazonLambdaClient = new AmazonLambdaClient();

    NOAAQInfo noaaqInfoResult = null;

    InvokeRequest request = new InvokeRequest()
    {
        FunctionName = MyPython,
        Payload = JsonConvert.SerializeObject(payload)
    };


    CancellationToken cancellationToken = new CancellationToken();
    string messageStr = null;
    InvokeResponse invokeResponse = null;

    invokeResponse = await amazonLambdaClient.InvokeAsync(request, cancellationToken);

    if (invokeResponse.StatusCode == (int)HttpStatusCode.OK)
    {
        if (resp.body != VerifiedStrResponse)
        {
            noaaqInfoResult = noaaqInfo;
        }
    }

    return noaaqInfoResult;
}

If i look at the logs for the C# lambda, i can see the 2 items being sent to the python lambda:

2021-02-19T14:04:39.837-08:00   invoking MyPython api for: [email protected]
2021-02-19T14:04:40.057-08:00   invoking MyPython api for: [email protected]

However, when I look at the Python lambda logs, i only see one item being received:

[INFO]  2021-02-19T14:04:40.837-08:00Z  adafecd6-8f32-4b7f-8d46-94465ac03a63    received params:{'Email': '[email protected]'}

Why don't I see testBBB in the python logs? ie there should be 2 invocations of the python lambda.

I also double checked that there wasn't an additional log stream, and that the python lambda doesnt have any concurrency limit set.

Really scratching my head on this one, as I'm not getting any errors. It's just that only one item is getting sent to the python lambda.

Additionally, The logging line inside the ContinueWith() isn't showing either.

Recommend reading stackoverflow.com/help/minimal-reproducible-example for guidance on creating minimal code snippets for questions. — Philip Pittle
– Philip Pittle, Commented Feb 24, 2021 at 20:37

Robert McKee · Accepted Answer · 2021-02-20 01:53:31Z

1

You are missing some awaits.

I believe this should work for you:

var noaaqInfos = batchItemsForMyPython.Select(async noaaqInfo => {
  Console.WriteLine($"invoking MyPython api for: {noaaqInfo.Email}");
  var t = await SendToMyPython.executeLambda(dbLogic, noaaqInfo, context);
  Console.WriteLine($"MyPython success: {t.Result.Email}");
  return t;
});
Task.WhenAll(noaaqInfos);

Also much easier if you move the Console stuff to your async method, remove DbLogic and ILambdaContext that you don't seem to be using, then the call just becomes:

var tasks = batchItemsForPython.Select(SendToMyPython.executeLambda);
Task.WhenAll(tasks);

edited Feb 20, 2021 at 1:53

answered Feb 20, 2021 at 1:28

Robert McKee

21.5k1 gold badge48 silver badges64 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

mcs Over a year ago

thanks @Robert McKee can you explain how the awaits help?

Robert McKee Over a year ago

You have to await methods that are async. Your problem was that you were firing off the events, but never await-ing the result, so the program would end. Sort of like if I told you "Count to 100", then stopped you immediately. You might have been able to say "1" before I interrupted you and had you stop, or maybe not. It's a little bit more complicated than that, especially since async isn't multithreading, but that's it on a very simple level.

Robert McKee Over a year ago

I made the assumption that you actually wanted the python methods to be running concurrently rather than one at a time. If you want them to run one at a time, just add the await here: NOAAQInfo t =await SendToMyPython.executeLambda(dbLogic, noaaqInfo, context); and get rid of the continue stuff after and just console.write afterwards.

mcs Over a year ago

great analogy! going to try your solution

Collectives™ on Stack Overflow

C# InvokeAsync in foreach only invoking Lambda one time

1 Answer 1

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related