This site has an "export as csv" button next to the "Power and Energy" text. This is the link of that button which can be found on browser's developer tools (Note that the link changes every 15 minutes)
My question is, is there any way to get that link using python without opening the browser? (I.e. without using selenium because it would take time if I did this for multiple sites.)
My goal is to automate the process of downloading that CSV file, every 1 hour.
That url works in curl. Therefore the data doesn't depend on javascript and you do not need selenuim. The st and et are unix time:
datetime.datetime.utcfromtimestamp(1567296000000/1000)
The timeUnit=4 is daily, so making that request hourly is wasting the site's resource and yours. If you really need hourly, change time timeUnit=3 and pn0=Power. Energy=Power x Hr
Lastly, I assume that since you are planning to run hourly, you are planning to save the data somewhere. If so, consider adjusting your st and et to only that range you need to further reduce your request size.
If you open your browsers debugging tools and go to the "Network" tab, you can see web requests as they are being made.
For example, the request for download from the site you linked is:
The various URL parameters mean that that link will probably always point to that specific file (rather than one that is updated over time).
You can use this method to get the links for the other sites as well, but like this one, the URLs will probably point to a link for that specific time.
