I have a java list of URLs. I wish to call a function for each URL in the list. And this function adds URL to the list. Is it possible to loop over all the URLs including newly added URLs in the list?
for(String links: urls) {
ar = getNews(links);
}
inside getNews() there is:
urls.add(json.optString("next"));
I did this successfully using recursion. By calling:
getNews(json.optString("next"));
inside getNews()
You can use regular for loop and change the condition each iteration
int size = 1;
for (int i = 0 ; i < size ; ++i) {
ar = getNews(urls.get(i));
size = urls.size();
}
getNews()
Use a Queue instead of a list:
Queue<String> q = new LinkedList<>();
q.add(initial urls);
while (!q.isEmpty()) {
String url = q.pop();
q.addAll(readNews(url));
}
I am assuming that this code lives outside the readNews method, i.e. there is no recursion here.
Also, since I assume this is some sort of crawler, you might want to keep track of URLs you visited previously, to avoid visiting them again:
Queue<String> q = new LinkedList<>();
Set<String> visited = new LinkedHashSet<>();
q.add(initial urls);
while (!q.isEmpty()) {
String url = q.pop();
if (visited.add(url)) {
q.addAll(readNews(url));
}
}
for i = 0; ...), since it allows concurrent modification, and checks for the exit condition at every iteration