i am totally new to web scraping, can anyone please tell me how can I scrape trailing returns(%) table from this website- https://www.valueresearchonline.com/funds/newsnapshot.asp?schemecode=16854 open it in incognito mode, and save it to excel in the same way as it has been shown.
Thank you.
import requests
from bs4 import BeautifulSoup
import pandas as pd
res = requests.get("https://www.valueresearchonline.com/funds/16854/franklin-india-ultra-short-bond-fund-super-institutional-plan-direct-plan/")
soup = BeautifulSoup(res.text, "html.parser")
table = soup.find("table",{"id":"trailing-returns-table"})
columns = [i.get_text(strip=True) for i in table.find_all("th")]
data = []
for tr in table.find("tbody").find_all("tr"):
data.append([td.get_text(strip=True) for td in tr.find_all("td")])
df = pd.DataFrame(data, columns=columns)
df.to_excel("data.xlsx", index=False)
The output will be saved to an excel file.
Output:
YTD 1-Day 1-W 1-M 3-M 6-M 1-Y 3-Y 5-Y 7-Y 10-Y
0 Fund 0.76 0.07 0.20 0.90 2.48 4.59 4.60 7.36 8.24 8.85 --
1 CCIL T Bill Liquidity Weight 2.12 -0.01 0.03 0.14 0.68 1.86 3.74 4.09 4.29 4.79 --
2 Debt: Ultra Short Duration 3.85 0.02 0.06 0.53 1.88 3.36 6.98 6.55 7.19 8.18 --
Pandas is easier to save to excel file or csv file and also to analyse data. Else it will take a bit of work to write to excel using openpyxl or xlsxwriter which are internally handled by pandas
pd.concat
