how to retrieve data from a web server using an oauth2 token in R?

Question

I've successfully received an access token from an oauth2.0 request so that I can start obtaining some data from the server. However, I keep getting error 403 on each attempt. APIs are very new to me and I only am entry level in using R so I can't figure out whats wrong with my request. I'm using the crul package currently, but I've tried to make the request with the httr package as well, but I can't get anything through without encountering the 403 error. I have a shiny app which in the end I'd like to be able to refresh with data imported from this other application which actually stores data, but I want to try to pull data to my console locally first so I can understand the basic process of doing so. I will post some of my current attempts.

(x <- HttpClient$new(
url = 'https://us.castoredc.com',
opts = list( exceptions = FALSE),
headers = list())
)
res.token <- x$post('oauth/token',
body = list(client_id = "{id}",
client_secret = "{secret}",
grant_type = 'client_credentials'))

importantStuff <- jsonlite::fromJSON(res$parse("UTF-8"))

token <- paste("Bearer", importantStuff$access_token)

I obtain my token, but the following doesn't seem to work.### I'm attempting to get the list of study codes so that I can call on them in further requests to actually get data from a study.

res.studies <- x$get('/api/study',headers = list(Authorization = 
token,client_id = "{id}",
client_secret = "{secret}",
grant_type = 'client_credentials'),
body = list(
content_type = 'application/json'))

Their support team gave me the above endpoint to access the content, but I get 403 so I think i'm not using my token correctly?

status: 403
access-control-allow-headers: Authorization
access-control-allow-methods: Get,Post,Options,Patch

Is there some documentation for this API somewhere? Every API is free to handle authentication in a different way. It's not going to be easy to help without some known working examples or documentation first. — MrFlick
– MrFlick, Commented Apr 8, 2019 at 19:06
They have API documentation that should be accessible by anyone. data.castoredc.com/api — Jdev
– Jdev, Commented Apr 8, 2019 at 19:07
looks like where you call the route /api/study maybe you should only be including your bearer token in the headers — sckott
– sckott, Commented Apr 8, 2019 at 19:15
On that documentation website they use "data.castoredc.com" for the sever but you seem to be using "us.castoredc.com". Are you sure those are the same? — MrFlick
– MrFlick, Commented Apr 8, 2019 at 19:28
The support team told me that your base url depends on your home server. So they asked me to change it to us. — Jdev
– Jdev, Commented Apr 8, 2019 at 19:32

Derk Arts · Accepted Answer · 2019-04-25 20:34:49Z

4

I'm the CEO at Castor EDC and although its pretty cool to see a Castor EDC question on here, I apologize for the time you lost over trying to figure this out. Was our support team not able to provide more assistance?

Regardless, I have actually used our API quite a bit in R and we also have an amazing R Engineer in house if you need more help.

Reflecting on your answer, yes, you always need a Study ID to be able to do anything interesting with the API. One thing that could make your life A LOT easier is our R API wrapper, you can find that here: https://github.com/castoredc/castoRedc

With that you would:

remotes::install_github("castoredc/castoRedc")
library(castoRedc)
castor_api <- CastorData$new(key = Sys.getenv("CASTOR_KEY"), 
                             secret = Sys.getenv("CASTOR_SECRET"), 
                             base_url = "https://data.castoredc.com")
example_study_id <- studies[["study_id"]][1]
fields <- castor_api$getFields(example_study_id)

etc.

Hope that makes you life a lot easier in the future.

answered Apr 25, 2019 at 20:34

Derk Arts

3,4702 gold badges20 silver badges36 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Jdev Over a year ago

Your support team was actually very helpful. I am just very new to API's. I'm going to download that and give it a try. Thank you!

Jdev · Accepted Answer · 2019-04-08 19:40:07Z

1

So, After some investigation, It turns out that you first have to make a request to obtain another id for each Castor study under your username. I will post some example code that worked finally.

req.studyinfo <- httr::GET(url = "us.castoredc.com/api/study"
,httr::add_headers(Authorization = token))
json <- httr::content(req.studyinfo,as = "text")
studies <- fromJSON(json)

Then, this will give you a list of your studies in Castor for which you can obtain the ID that you care about for your endpoints. It will be a list that contains a data frame containing this information. you use the same format with whatever endpoint you like that is posted in their documentation to retrieve data. Thank you for your observations! I will leave this here in case anyone is employed to develop anything from data used in the Castor EDC. Their documentation was vague to me, so maybe it will help someone in the future. Example for next step:

req.studydata <- httr::GET("us.castoredc.com/api/study/{study id obtained 
from previous step}/data-point- 
collection/study",,httr::add_headers(Authorization = 
token))
json.data <- httr::content(req.studydata,as = "text")
data <- fromJSON(json.data)

answered Apr 8, 2019 at 19:40

Jdev

1107 bronze badges

2 Comments

MrFlick Over a year ago

But above you said you got a 403 at the "/api/study" endpoint. Is that not the case? Where you getting the 403 for a different endpoint while the "/api/study" endpoint was working all along? I'm not sure I see how your first request is any different here.

Jdev Over a year ago

My interpretation is that the additional body contents and headers were some how blocking my access. When I only used the token with the authorization header, I was able to get a code 200. I'm not sure however if the httr package has something subtle that also assisted in this. I'm attaching the link for which I found a twitter example that I basically copied and adjusted. cran.r-project.org/web/packages/jsonlite/vignettes/…

Lakhwinder Singh · Accepted Answer · 2019-11-14 12:59:36Z

1

This worked for me, I removed the Sys.getenv() part

library(castoRedc)
castor_api <- CastorData$new(key = "CASTOR_KEY", 
                             secret = "CASTOR_SECRET", 
                             base_url = "https://data.castoredc.com")
example_study_id <- studies[["study_id"]][1]
fields <- castor_api$getFields(example_study_id)

edited Nov 14, 2019 at 12:59

Lakhwinder Singh

5,5825 gold badges30 silver badges52 bronze badges

answered Nov 14, 2019 at 12:55

Danny Cao

477 bronze badges

1 Comment

Jdev Over a year ago

It's true that the castoRedc package is very quick and easy, but I was trying to write my own requests code for learning purposes and the fact that calculation fields and checkboxes have all only exported missing values for me when using this package. I let their support team know and they said it was a bug that they may have fixed in the future.

Collectives™ on Stack Overflow

how to retrieve data from a web server using an oauth2 token in R?

3 Answers 3

1 Comment

2 Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

2 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related