Selenium tests failing on Azure DevOps but work locally and in Jenkins

Question

We have a suite of selenium/c# tests which have been tested in the following locations.

Local in Visual Studio – All good, everything passed 1st run

Jenkins Server – All good, 99% pass 1st run, 100% on first re-run

Azure release pipeline – Mostly good. Total tests 97, 94 pass every time, the remaining 3 fail every time.

Looking at the logs it seems the tests are timing out because an element was not found within 60 seconds, this however never fails local or Jenkins, so why on Azure pipelines? All 3 methods are hitting the same test server for the App under test and they are all using the same test code and it is the same element in each test that is not found.

We are using VS 2022 Prof, Selenium 4.4, MSTest 2.2, Specflow 3.9

I have checked the chromedriver version, it is correct

This is a typical error on the azure server

-> error: The HTTP request to the remote WebDriver server for URL http://localhost:59062/session/899fc92fbc887b4d8431ecfb7e51256f/element timed out after 60 seconds. (125.3s)
 <pre>   at OpenQA.Selenium.Remote.HttpCommandExecutor.Execute(Command commandToExecute)
    at OpenQA.Selenium.Remote.DriverServiceCommandExecutor.Execute(Command commandToExecute)
    at OpenQA.Selenium.WebDriver.Execute(String driverCommandToExecute, Dictionary`2 parameters)
    at OpenQA.Selenium.WebDriver.FindElement(String mechanism, String value)
    at OpenQA.Selenium.By.<.ctor>b__11_0(ISearchContext context)
    at OpenQA.Selenium.By.FindElement(ISearchContext context)
    at OpenQA.Selenium.WebDriver.FindElement(By by)
    at TestApp.Pages.TestPage.MatchTheGrandTotal() in D:\a\r1\a\_Regression\PelTest\PageObjects\IntelPage.cs:line 325
    at TechTalk.SpecFlow.Bindings.BindingInvoker.InvokeBinding(IBinding binding, IContextManager contextManager, Object[] arguments, ITestTracer testTracer, TimeSpan& duration)
    at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.ExecuteStepMatch(BindingMatch match, Object[] arguments, TimeSpan& duration)
    at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.ExecuteStep(IContextManager contextManager, StepInstance stepInstance)
    at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.OnAfterLastStep()
    at TechTalk.SpecFlow.TestRunner.CollectScenarioErrors()

The timeout parameter in the the command.

jdweng
– jdweng

2022-10-18 11:19:19 +00:00
Commented Oct 18, 2022 at 11:19 — jdweng
– jdweng, Commented Oct 18, 2022 at 11:19

Nischitha K N · Accepted Answer · 2022-10-18 10:52:26Z

1

I have faced similar issue. It is not about the synchronization in the UI, but it is the response time from the server which is taking more than the default timeout 60 seconds. Increase the response timeout while launching the browser.

Syntax to increase the response timeout in chrome browser in the below snippet. Check increasing the timeout to 2mins/3mins by executing and figure out the ideal response time which works well in all the execution environment.

driver = new ChromeDriver(serviceObj, chromeOptions, TimeSpan.FromMinutes(2));

answered Oct 18, 2022 at 10:52

Nischitha K N

4072 silver badges8 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

Simmetric Over a year ago

Even then I suggest looking at what happens serverside to cause these requests to take so long. Maybe for testing purposes you can replace any expensive serverside code or data access with a simple stub. Extending the timeout only goes so far as Azure Devops will have a hard maximum for how long a job can run.

Nischitha K N Over a year ago

yes @Simmetric agreed if the timeout crosses the threshold of the standards

Kev Over a year ago

Thanks, I will try increasing the timeout and see what happens

Kev Over a year ago

Increasing the timeouts has helped on most tests, just one step failing now. This involves trying to click a span element with a special button class. Again this works fine on local and Jenkins but always fails in the Azure pipeline, any ideas anyone?

Nischitha K N Over a year ago

Please raise a different question for this with more specific details to button. The context of this button failure and context of your question is different. Please accept the answer if it is working, so that it will be helpful to the community as well.

Collectives™ on Stack Overflow

Selenium tests failing on Azure DevOps but work locally and in Jenkins

1 Answer 1

5 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

5 Comments

Your Answer

Sign up or log in

Post as a guest

Related