Error in curl::curl_fetch_memory(url, handle = handle) : Timeout was reached: Send failure: Connection was reset

Question

I am trying to extract comments from 5000 videos from Youtube's API. My code works perfectly when I download 5 videos or more but when I plug the entire list of videos that I want, it throws the below error after running for hours. I am not sure what it means or if it is a memory problem. Thank you!

Line of code in R:

 comments_1 <- CollectDataYoutube(video1,key,writeToFile = FALSE)

video1: is a list of 5 thousand videos codes that I have so that the API extracts comments from it directly.

key: is my API key stored in an object

Error message:

Error in curl::curl_fetch_memory(url, handle = handle) : Timeout was reached: Send failure: Connection was reset

peterhurford · Accepted Answer · 2018-12-05 20:17:10Z

5

RCurl can only keep open an internet connection for so long to stream data in. It's an internet latency problem, not a memory problem. You'd be better off batching your requests in smaller batches and then combining them.

answered Dec 5, 2018 at 20:17

peterhurford

1,1021 gold badge13 silver badges22 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Error in curl::curl_fetch_memory(url, handle = handle) : Timeout was reached: Send failure: Connection was reset

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related