Is it possible to generate headers automatically with HttpClient / RestSharp?

Question

At the moment I am using the following RestSharp request to get a website's content:

        var client = new RestClient(productLink);
        var request = new RestRequest(Method.GET);
        request.AddHeader("Cookie", "insert-cookie-content"); 
        IRestResponse response = client.Execute(request);
        Console.WriteLine(response.Content);

I have tried converting it into HttpClient as i will need to use the AllowRedirect property later:

        var client = new HttpClient();
        client.DefaultRequestHeaders.Accept.Clear();
        client.DefaultRequestHeaders.Add("Cookie", "insert-cookie-content");
        var response = await client.GetAsync(productUrl);
        Console.WriteLine(response);

The URL I am trying to get a response from is: https://www.nike.com/sg/launch/t/air-max-90-orange-duck-camo

My first problem is that the HttpClient request is giving me 403 Errors whereas the RestClient request was working fine. How can I fix this?

My second problem is that the cookie expires after a couple of uses, and I have to manually get a new one from postman and insert it. Is there anyway for the request to generate its own cookie?

Here is the two fiddler responses compared: https://i.sstatic.net/UK2Lo.jpg

Have you compared the two outgoing requests? Like capturing them with Fiddler (or some similar tool) and compare them side-by-side. — Peter Csala
– Peter Csala, Commented Jun 16, 2020 at 11:57
@PeterCsala I have, doesn't make much sense to me but I can see that the RestSharp request has some security headers whereas the HttpClient does not. — kalek
– kalek, Commented Jun 16, 2020 at 12:14
Could please include the result of your comparison into the question? So, which ones are missing from the HttpClient based solution. — Peter Csala
– Peter Csala, Commented Jun 16, 2020 at 12:17
A web server does not care what library is being used to call it. All the web server sees are the requests. So if you compare the requests, find out what's different about them, and make them the same, then the server will treat them the same. That's why it was suggested to use Fiddler. — mason
– mason, Commented Jun 16, 2020 at 14:36

Peter Csala · Accepted Answer · 2020-06-16 14:28:12Z

1

In case of HttpClient if you want to pass the Cookies manually through the DefaultRequestHeaders then you have to tell this to the HttpClient to do NOT use CookieContainer. You have to use HttpClientHandler's UseCookie flag to indicate it.

var client = new HttpClient(new HttpClientHandler { UseCookies = false });
client.DefaultRequestHeaders.Accept.Clear();
client.DefaultRequestHeaders.Add("Cookie", "insert-cookie-content");
var response = await client.GetAsync(productUrl);
Console.WriteLine(response);

answered Jun 16, 2020 at 14:28

Peter Csala

23.7k16 gold badges51 silver badges96 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Is it possible to generate headers automatically with HttpClient / RestSharp?

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related