Why am I getting 403 on GetAsync from api.stackexchange?

Question

I am writing a demo app for myself to learn and suddenly I started getting 403 when doing api.stackexchange.com requests and I fail at EnsureSuccessStatusCode(). I'm doing them to populate my database with some tags data. (injected client just sets AutomaticDecompression to GZip)

public async Task<List<TagsImport>> ImportStackOverflowTagsAsync()
{
    var tagImportsList = new List<TagsImport>();
    var httpClient = _httpClient.GetClient();

    httpClient.BaseAddress = new Uri("https://api.stackexchange.com/");
    httpClient.DefaultRequestHeaders.Accept.Clear();

    for (var i = 1; i <= 10; i++)
    {
        var response = await httpClient.GetAsync($"2.3/tags?page={i}&pagesize=100&order=desc&sort=popular&site=stackoverflow");
        response.EnsureSuccessStatusCode();

        var responseBody = await response.Content.ReadAsStringAsync();

        var tags = JsonSerializer.Deserialize<TagsImport>(responseBody);
        tagImportsList.Add(tags);
    }

    return tagImportsList;
}

Tried manually testing the request and then testing previous versions of the app, the test in the app just returns "System.Net.Http.HttpRequestException : Response status code does not indicate success: 403 (Forbidden)." while manually writing "https://api.stackexchange.com/2.3/tags?page=1&pagesize=100&order=desc&sort=popular&site=stackoverflow" in browser returns the data properly.

Am I missing something to make it work?

youre doing the api request in a loop here. does it fail on the first try or only in later ones? — Molbac
– Molbac, Commented Aug 20, 2024 at 14:52
Do you use an API key? Do you respect the backoff field (when it is not absent)? Try adding a small delay between the requests you make. — double-beep
– double-beep, Commented Aug 20, 2024 at 15:00
When your authentication fails according to the status code something is wrong with your authentication. And i don't see any in your code. — Ralf
– Ralf, Commented Aug 20, 2024 at 15:34
I want to use the API anonymously, I still have quota left and I'm doing max 10 requests per use. It also fails on the very first request. Don't really know how can I authenticate my app which runs only locally to use the stackexchange api then. — Lordzio
– Lordzio, Commented Aug 20, 2024 at 15:44
You definitely need to add the UserAgent Header, as one of the answers here points out. Otherwise a 403 is guaranteed. Then, you should check the has_more Property of the resulting JSON, to verify that you have more pages to get. Maybe, instead of a loop, increment a counter and bail out either if has_more = false or you have reached your goal. Add a Task.Delay(100) between calls to GetAsync() (or GetStringAsync() or GetFromJsonAsync<T>()) — Jimi
– Jimi, Commented Aug 20, 2024 at 17:05

0xn0b174 · Accepted Answer · 2024-08-26 00:31:55Z

1

This error is caused by User-Agent header. Basically what you need to do is that add user agent header into your httpClient and it would behave like a browser getting the data without issues.

copy & paste this:

 httpClient.DefaultRequestHeaders.UserAgent.TryParseAdd("Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36");

edited Aug 26, 2024 at 0:31

0xn0b174

1,0477 silver badges17 bronze badges

answered Aug 20, 2024 at 15:42

Guga Rukhadze

604 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Why am I getting 403 on GetAsync from api.stackexchange?

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related