0

I have a very complex solution, written in C#/.NET 6, composed by Azure Service Triggers that use an API running as an Azure HTTP Trigger. There are 4 trigger function, each running up to 30 parallel sessions, and each session spans up to 10 processing tasks. Each task calls the API for accessing the resources concurrently. I am using Flurl serverless default behavior, configured for HTTP/2. All APIs are in the same endpoint, but with different paths, configured with Polly for retry strategy

For the loading tasks, I have a console app built using the same stack, calling the API that starts the processing up to 1000 times, while keeping the system running for processing the requests concurrently. Usually, the first 800 call runs properly, but after that, I start to have time-outs in both parts - the console app and the Azure functions. All the time, the CPU and memory allocation in the API is low <30%, but I can observe an increase in the number of sockets used for processing, but under the allowed limits.

The error reported is: TaskCanceledException - "The operation was canceled."

With the inner stack trace:

  at System.Net.Http.Http2Connection.Http2Stream.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token)
   at System.Net.Http.Http2Connection.Http2Stream.<ReadResponseHeadersAsync>d__74.MoveNext()
   at System.Net.Http.Http2Connection.<SendAsync>d__111.MoveNext()
   at System.Net.Http.HttpConnectionPool.<SendWithVersionDetectionAndRetryAsync>d__89.MoveNext()
   at System.Net.Http.DecompressionHandler.<SendAsync>d__16.MoveNext()
   at Microsoft.Extensions.Http.PolicyHttpMessageHandler.<SendCoreAsync>d__6.MoveNext()
   at Polly.Retry.AsyncRetryEngine.<ImplementationAsync>d__0`1.MoveNext()
   at Polly.AsyncPolicy`1.<ExecuteInternalAsync>d__23.MoveNext()
   at Microsoft.Extensions.Http.PolicyHttpMessageHandler.<SendAsync>d__5.MoveNext()
   at System.Net.Http.HttpClient.<<SendAsync>g__Core|83_0>d.MoveNext()

I noticed that my retry strategy is not triggered when the errors occurs. This hint me the call is not reaching the server, so the problem is not an overloading of resources there. I am also opening a ticket at Microsoft, but I would like to know if this there is something to be done on the client.

Improve this question
8
  • 1
    Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Commented Feb 20 at 11:08
  • Sounds very similar to this: stackoverflow.com/a/60116243/68432 Commented Feb 20 at 11:20
  • I noticed that my retry strategy is not triggered when the errors occurs. << how did you define your retry policy? Commented Feb 20 at 15:15
  • Yes, the retry logic is working at HTTP level. Commented Feb 21 at 10:53
  • How are you creating the HttpClient instances? Could you share a code sample to illustrate? Commented Feb 24 at 21:45
Related questions
2
Cannot access a disposed object. Object name: 'SslStream'
0
Could not load file or assembly 'netstandard, Version=2.0.0.0
1
System.ComponentModel.Win32Exception: The wait operation timed out from Azure VPN
Load 4 more related questions Show fewer related questions

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.