The access token that your API received is intended only for your API. What you need is to obtain a new access token, on behalf of the current user, for the Azure AD Graph API.
Fortunately, this is exactly what the on-behalf-of flow is for. From Authentication Scenarios for Azure AD:
Delegated User Identity with OAuth 2.0 On-Behalf-Of Draft Specification
The flow discussed below assumes that a user has been authenticated on another application (such as a native application), and their user identity has been used to acquire an access token to the first-tier web API.
- The native application sends the access token to the first-tier web API.
- The first-tier web API sends a request to Azure AD’s token endpoint, providing its client ID and credentials, as well as the user’s access token. In addition, the request is sent with an on_behalf_of parameter that indicates the web API is requesting new tokens to call a downstream web API on behalf of the original user.
- Azure AD verifies that the first-tier web API has permissions to access the second-tier web API and validates the request, returning a JWT access token and a JWT refresh token to the first-tier web API.
- Over HTTPS, the first-tier web API then calls the second-tier web API by appending the token string in the Authorization header in the request. The first-tier web API can continue to call the second-tier web API as long as the access token and refresh tokens are valid.
Be sure to configure your API to request the right set of permissions for the Azure AD Graph API.
Edit: If you are constructing the token request yourself, the request that your API would make to Azure AD to get a new token to the Graph API on behalf of the current user would be a POST against:
https://login.microsoftonline.com/{tenant-id}/oauth2/token
With the following parameters in the body (un-encoded, for readability, in reality these would be application/x-www-form-urlencoded, of course):
grant_type=urn:ietf:params:oauth:grant-type:jwt-bearer&
requested_token_use=on_behalf_of&
assertion={access-token}&
client_id={api-client-id}&
client_secret={api-client-secret}&
resource=https://graph.windows.net&
scope=openid
Where {tenant-id} is the directory identifier (domain name or Guid value), {access-token} is the access token that your SPA provided to your API (the one you're exchanging for an access token to the Graph API), {api-client-id} is the client ID for your API, and {api-client-secret} is the API's secret password credential.
(Note, for simplicity, this example uses password credentials (client_secret) to authenticate the API, though it could very well be using instead an assertion signed by a client certificate.)
