Web-Scraping using R (I want to extract some table like data from a website)

Question

I'm having some problems scraping data from a website. I do have not a lot of experience with web-scraping. My intended plan is to scrape some data using R from the following website: https://www.myfxbook.com/forex-broker-swaps

More precisely, I want to extract the Forex Brokers Swap Comparison for all the available pairs.

My idea so far:

  library(XML)
  url <- paste0("https://www.myfxbook.com/forex-broker-swaps")
  source <- readLines(url, encoding = "UTF-8")
  parsed_doc <- htmlParse(source, encoding = "UTF-8")
  test<-xpathSApply(parsed_doc, path = '/html/body/div[3]/div[6]/div/div/div/div/div/div/div/div/div[3]/div[4]/div/div[2]/div', xmlValue)

But this doesn't bring up the intended information. Some help would be really appreciated here! Thanks!

DaveArmstrong · Accepted Answer · 2022-05-25 10:28:08Z

1

How about this:

library(dplyr)
library(rvest)
h <- read_html("https://www.myfxbook.com/forex-broker-swaps")
h %>% html_table() %>% 
  purrr::pluck(3) %>% 
  setNames(paste(names(.), .[1,], sep="_")) %>% 
  rename("Broker" = "_Broker") %>% 
  filter(Broker != "Broker") %>%
  mutate(across(-Broker, as.numeric))
#> # A tibble: 91 × 13
#>    Broker          `EUR/USD_Short` `EUR/USD_Long` `EUR/USD_Type` `GBP/USD_Short`
#>    <chr>                     <dbl>          <dbl>          <dbl>           <dbl>
#>  1 Axi                        0.17          -0.56              0           -0.18
#>  2 Tickmill                   0.24          -0.55              0           -0.22
#>  3 Blueberry Mark…            0.31          -0.55              0           -0.17
#>  4 Eightcap                   0.31          -0.55              0           -0.17
#>  5 Rakuten Securi…            0.19          -0.5               0            0   
#>  6 ACY Securities            -0.34          -3.75              3           -1.28
#>  7 AAAFx                      1.98          -6.42              1           -2.07
#>  8 MultiBank Group            0.3           -0.66              0           -0.12
#>  9 Just2Trade                 0.12          -0.9               0           -0.26
#> 10 Fusion Markets             0.31          -0.55              0           -0.15
#> # … with 81 more rows, and 8 more variables: `GBP/USD_Long` <dbl>,
#> #   `GBP/USD_Type` <dbl>, `USD/CAD_Short` <dbl>, `USD/CAD_Long` <dbl>,
#> #   `USD/CAD_Type` <dbl>, `USD/JPY_Short` <dbl>, `USD/JPY_Long` <dbl>,
#> #   `USD/JPY_Type` <dbl>

^{Created on 2022-05-25 by the reprex package (v2.0.1)}

answered May 25, 2022 at 10:28

DaveArmstrong

22.5k2 gold badges16 silver badges30 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Kaspro Over a year ago

Thanks a lot @DaveArmstrong for this. Is it possible to export the rest FX pairs? the web bage allows only 5 per view. The full xpath for selecting other pairs is: "/html/body/div[3]/div[6]/div/div/div/div/div/div/div/div/div[3]/div[4]/div/div[1]/div[2]/div/a/i"

Collectives™ on Stack Overflow

Web-Scraping using R (I want to extract some table like data from a website)

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related