Google Apps Script script.run -- getting unexpected 401, even while sending Oauth token

I am trying to run a Google Apps script using the Google Apps Script API method scripts.run. However, I get a 403 error with the following error message:

{
  "error": {
    "code": 401,
    "message": "Request is missing required authentication credential. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.",
    "status": "UNAUTHENTICATED"
  }
}

Here are the headers for the POST request I am sending:

authorization: Bearer my-oauth-token
Content-Type: application/json
Origin: http://localhost:5000
Referer: http://localhost:5000/static/test.html
User-Agent: Mozilla/5.0...

I am sending the request to https://script.googleapis.com/v1/scripts/my_script_id:run?key=my_api_key

Any ideas? I have tried searching for examples, but all I get are ones that you the google apis client libraries, when I need to use the REST api. I know for sure that my oauth tokens are correct, as I make requests to other Google apis with the same token.

This is my current flow:

1. Redirect user to oauth url, and get exchange code. My redirect url is

"https://accounts.google.com/o/oauth2/v2/auth?"
        "scope=https://www.googleapis.com/auth/drive&"
        "state=%s&"
        "redirect_uri=redirect_uri&"
        "client_id=id&"
        "response_type=code&"
        "access_type=offline&"
        "prompt=consent"

2. Exchange code for refresh token
3. Use refresh token to get an oAuth access token. I send a POST request to https://www.googleapis.com/oauth2/v4/token to do this.
4. I use access token to get thumbnails from a user's google slides. This request succeeds.
5. I send request to execute Google Apps Script. Here is the summarized code for this request:

xhr.open("POST", "https://script.googleapis.com/v1/scripts/id:run", true);
xhr.setRequestHeader("authorization", "Bearer " + oauth_token);
xhr.onload = function() { // do stuff }
xhr.onerror = function() { // print error }
xhr.send(JSON.stringify({
   "function": "run",
    "parameters": [
       id1,
       id2
    ]
}));

This gives me 401 error. I also get message "Provisional Headers are shown". I looked into that and it does not seem related to my issue.

This is the script that I am trying to run:

function doGet(e) {
  if(!(e.parameter.source && e.parameter.destination)) {
    throw new Error("Not all parameters specified");
  }
  copySlides(e.parameter.source, e.parameter.destination);
}

function copySlides(sourceId, destinationId) {
  var src = SlidesApp.openById(sourceId);
  var dest =  SlidesApp.openById(destinationId);

  src.getSlides().forEach(function(slide, index) {
      dest.appendSlide(slide);
  });
  return ContentService.createTextOutput("Done Copying Slides");
}