I am trying to extract the tables in the fedex website () using selenium and beautiful soup into dataframes using id but nothing comes up into dataframe
url = 'https://www.fedex.com/en-us/shipping/fuel-surcharge.html#'
response = requests.get(url)
print(response.status_code)
soup = BeautifulSoup(response.text,"html.parser")
tables = soup.findAll('table',{"id":"international-table"})
how do i read tables into dataframes
The tag should be <div>, not <table>:
import requests
import pandas as pd
from bs4 import BeautifulSoup
url = 'https://www.fedex.com/en-us/shipping/fuel-surcharge.html#'
response = requests.get(url)
soup = BeautifulSoup(response.text,"html.parser")
table = soup.find('div',{"id":"international-table"}) # <-- should be <div>, not <table>
df = pd.read_html(str(table))[0]
print(df)
Prints:
0 1 2 3
0 At Least But Less Than Export Import
1 $0.71 $0.75 1.75% 4.50%
2 $0.75 $0.79 2.00% 4.75%
3 $0.79 $0.83 2.25% 5.00%
4 $0.83 $0.87 2.50% 5.25%
5 $0.87 $0.91 2.75% 5.50%
6 $0.91 $0.95 3.00% 5.75%
7 $0.95 $0.99 3.25% 6.00%
8 $0.99 $1.03 3.50% 6.25%
9 $1.03 $1.07 3.75% 6.50%
10 $1.07 $1.11 4.00% 6.75%
11 $1.11 $1.15 4.25% 7.00%
12 $1.15 $1.19 4.50% 7.25%
13 $1.19 $1.23 4.75% 7.50%
now fedex does not allow to use selenium, I guess the only way is to get fedex api
