First of all you should write better locators. What you are doing is that you are querying all the <span> elements into list of webelements and then in your foreach loop you are again triggering the findElement method on each and every element until you break from the loop and inside the loop you are triggering the getText method. This is definitely an overkill. Better write a locator that will query you that specific element, something like //span[text() == 'valueOfelementNameVariable'] and then use WebDriverWait class and wait for element to be available for interaction.
For instance you could use the visibilityOfElementLocated expected condition like:
WebElement yourSpanElement = (new WebDriverWait(driver, 60))
.until(ExpectedConditions.visibilityOfElementLocated(By.xpath("//span[text() == \"valueOfelementNameVariable\"]")));
The until function of WebDriverWait will poll the DOM until the ExpectedCondition returns true or timeout occurs. To be more precise:
public V until(com.google.common.base.Function
isTrue)
Repeatedly applies this instance's input value to the given function
until one of the following occurs:
the function returns neither null nor false,
the function throws an unignored exception,
the timeout expires,
the current thread is interrupted
If you look at the implementation of visibilityOfElementLocated
/**
* An expectation for checking that an element is present on the DOM of a page
* and visible. Visibility means that the element is not only displayed but
* also has a height and width that is greater than 0.
*
* @param locator used to find the element
* @return the WebElement once it is located and visible
*/
public static ExpectedCondition<WebElement> visibilityOfElementLocated(
final By locator) {
return new ExpectedCondition<WebElement>() {
@Override
public WebElement apply(WebDriver driver) {
try {
return elementIfVisible(findElement(locator, driver));
} catch (StaleElementReferenceException e) {
return null;
}
}
@Override
public String toString() {
return "visibility of element located by " + locator;
}
};
}
It is already ignoring the staleness of element (means it will return null if the element is stale and the polling continues) and the same function will return you the element once it is found or throw an exception if something that is described above happens.
On a larger question, is there a good fix for stale element error?
You should use the Page Object design model. If you initiate all you element using PageFactory, among with a lot of other benefits it brings (mostly in terms of maintenance), you will also lose most of the staleness problems because in this case WebElements are evaluated lazily, that means that lookup of an element is performed every time you call that element.
webdriverin almost every aspect when speaking of web scraping, starting withcurlorwgetin combination with something likesedoregrep. But all in all, point taken.