How to use the HTTP API (wp_remote_get) instead of cURL?

Question

I'm running into a problemen while trying to change my cURL request into a request using the HTTP API.

Below code is my (old) cURL request. Here I'm getting some contents from an external URL. This is working perfectly.

The targeted URL in this example

$myURL = 'https://www.google.com/maps?cid=4493464801819550785';

My cURL request

$ch = curl_init($myURL);
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.116 Safari/537.36');
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$result = curl_exec($ch);
curl_close($ch);
$pattern = '/window\.APP_INITIALIZATION_STATE(.*);window\.APP_FLAGS=/ms';

if ( preg_match($pattern, $result, $match) ) {
    $match[1]       = trim($match[1], ' =;');
    $output         = json_decode($match[1]);
}

var_dump($output); // SHOWS CONTENTS

Now, what I'm trying to accomplish is to use the HTTP API to achieve the same result. I believe the main problem lies in the fact that $myURL has a 302 redirect while following. In my cURL request, the use of CURLOPT_FOLLOWLOCATION set to true is fixing this problem. The request is following the redirects as it should. Sadly, I cannot find a similar solution for using the HTTP API. Below code is what I have so far.

My wp_remote_get request

$result = wp_remote_get(
    $myURL,
    array(
        'method'        => 'GET',
        'timeout'       => 45,
        'redirection'   => 10,
        'httpversion'   => '1.0',
        'body'          => array(
            'returnFormat'  => 'json',
        ),
        'sslverify'   => false
    )
);

var_dump($result); // SHOWS CONTENTS FROM A REDIRECTED PAGE

This is the output of var_dump($result) (using wp_remote_get)

array(6) {
  ["headers"]=>
  object(Requests_Utility_CaseInsensitiveDictionary)#16709 (1) {
    ["data":protected]=>
    array(18) {
      ["content-type"]=>
      string(24) "text/html; charset=utf-8"
      ["vary"]=>
      string(46) "Sec-Fetch-Dest, Sec-Fetch-Mode, Sec-Fetch-Site"
      ["cache-control"]=>
      string(46) "no-cache, no-store, max-age=0, must-revalidate"
      ["pragma"]=>
      string(8) "no-cache"
      ["expires"]=>
      string(29) "Mon, 01 Jan 1990 00:00:00 GMT"
      ["date"]=>
      string(29) "Tue, 16 Aug 2022 08:13:40 GMT"
      ["content-security-policy"]=>
      string(161) "script-src 'report-sample' 'nonce-CbZa7ma1grUKc1m413SbUA' 'unsafe-inline';object-src 'none';base-uri 'self';report-uri /_/ConsentHttp/cspreport;worker-src 'self'"
      ["accept-ch"]=>
      string(151) "Sec-CH-UA-Arch, Sec-CH-UA-Bitness, Sec-CH-UA-Full-Version, Sec-CH-UA-Full-Version-List, Sec-CH-UA-Model, Sec-CH-UA-Platform, Sec-CH-UA-Platform-Version"
      ["cross-origin-opener-policy"]=>
      string(11) "unsafe-none"
      ["content-security-policy-report-only"]=>
      string(70) "require-trusted-types-for 'script';report-uri /_/ConsentHttp/cspreport"
      ["cross-origin-resource-policy"]=>
      string(9) "same-site"
      ["permissions-policy"]=>
      string(137) "ch-ua-arch=*, ch-ua-bitness=*, ch-ua-full-version=*, ch-ua-full-version-list=*, ch-ua-model=*, ch-ua-platform=*, ch-ua-platform-version=*"
      ["content-encoding"]=>
      string(4) "gzip"
      ["server"]=>
      string(3) "ESF"
      ["x-xss-protection"]=>
      string(1) "0"
      ["x-frame-options"]=>
      string(10) "SAMEORIGIN"
      ["x-content-type-options"]=>
      string(7) "nosniff"
      ["alt-svc"]=>
      string(162) "h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43""
    }
  }
  ["body"]=>
  string(13943) "
Inloggen
Google
Voordat je verdergaat naar Google
We gebruiken cookies en gegevens voor het volgende:
Google-services leveren en onderhouden
Uitval bijhouden en bescherming bieden tegen spam, fraude en misbruik
Doelgroepbetrokkenheid en sitestatistieken meten om inzicht te krijgen in hoe onze services worden gebruikt en de kwaliteit van die services te verbeteren
Als je Alles accepteren kiest, gebruiken we cookies en gegevens ook voor het volgende:
Nieuwe services ontwikkelen en verbeteren
Advertenties laten zien en de effectiviteit ervan meten
Gepersonaliseerde content laten zien (afhankelijk van je instellingen)
Gepersonaliseerde advertenties laten zien (afhankelijk van je instellingen)
Als je Alles afwijzen kiest, gebruiken we cookies niet voor deze aanvullende doeleinden.

Niet-gepersonaliseerde content wordt beïnvloed door factoren zoals de content die je op dat moment bekijkt, activiteit in je actieve zoeksessie en je locatie. Niet-gepersonaliseerde advertenties worden beïnvloed door de content die je op dat moment bekijkt en je algemene locatie. Gepersonaliseerde content en advertenties kunnen ook relevantere resultaten, aanbevelingen en op jou toegespitste advertenties omvatten die zijn gebaseerd op eerdere activiteit van deze browser, zoals uitgevoerde Google-zoekopdrachten. We gebruiken cookies en gegevens ook om te zorgen dat de functionaliteit geschikt is voor je leeftijd, als dit relevant is.

Selecteer Meer opties om meer informatie te bekijken, waaronder informatie over hoe je je privacyinstellingen beheert. Je kunt ook altijd naar g.co/privacytools gaan.
Meer opties

Nederlands
Privacybeleid Servicevoorwaarden
"
  ["response"]=>
  array(2) {
    ["code"]=>
    int(200)
    ["message"]=>
    string(2) "OK"
  }
  ["cookies"]=>
  array(2) {
    [0]=>
    object(WP_Http_Cookie)#16701 (7) {
      ["name"]=>
      string(13) "__Secure-ENID"
      ["value"]=>
      string(176) "6.SE=VgGXMbfmNLae_U49j2-xNyX9KSANb6dx9_kBZFAD-livbO1IFRxB0v04PdiGYCunZmFIdgJP9OAoodIXIXhoRNjx66Y0BlVOzVTDco86c0pQty6URPwm9u10tg4KQi3xaer_4s2q3OXLihut8AtZCDWLrFS_eCFQ8l-b67oyaCk"
      ["expires"]=>
      int(1694824318)
      ["path"]=>
      string(1) "/"
      ["domain"]=>
      string(10) "google.com"
      ["port"]=>
      NULL
      ["host_only"]=>
      bool(false)
    }
    [1]=>
    object(WP_Http_Cookie)#16700 (7) {
      ["name"]=>
      string(7) "CONSENT"
      ["value"]=>
      string(11) "PENDING 531"
      ["expires"]=>
      int(1723709620)
      ["path"]=>
      string(1) "/"
      ["domain"]=>
      string(10) "google.com"
      ["port"]=>
      NULL
      ["host_only"]=>
      bool(false)
    }
  }
  ["filename"]=>
  NULL
  ["http_response"]=>
  object(WP_HTTP_Requests_Response)#16713 (5) {
    ["response":protected]=>
    object(Requests_Response)#16710 (10) {
      ["body"]=>
      string(13943) "
Inloggen
Google
Voordat je verdergaat naar Google
We gebruiken cookies en gegevens voor het volgende:
Google-services leveren en onderhouden
Uitval bijhouden en bescherming bieden tegen spam, fraude en misbruik
Doelgroepbetrokkenheid en sitestatistieken meten om inzicht te krijgen in hoe onze services worden gebruikt en de kwaliteit van die services te verbeteren
Als je Alles accepteren kiest, gebruiken we cookies en gegevens ook voor het volgende:
Nieuwe services ontwikkelen en verbeteren
Advertenties laten zien en de effectiviteit ervan meten
Gepersonaliseerde content laten zien (afhankelijk van je instellingen)
Gepersonaliseerde advertenties laten zien (afhankelijk van je instellingen)
Als je Alles afwijzen kiest, gebruiken we cookies niet voor deze aanvullende doeleinden.

Niet-gepersonaliseerde content wordt beïnvloed door factoren zoals de content die je op dat moment bekijkt, activiteit in je actieve zoeksessie en je locatie. Niet-gepersonaliseerde advertenties worden beïnvloed door de content die je op dat moment bekijkt en je algemene locatie. Gepersonaliseerde content en advertenties kunnen ook relevantere resultaten, aanbevelingen en op jou toegespitste advertenties omvatten die zijn gebaseerd op eerdere activiteit van deze browser, zoals uitgevoerde Google-zoekopdrachten. We gebruiken cookies en gegevens ook om te zorgen dat de functionaliteit geschikt is voor je leeftijd, als dit relevant is.

Selecteer Meer opties om meer informatie te bekijken, waaronder informatie over hoe je je privacyinstellingen beheert. Je kunt ook altijd naar g.co/privacytools gaan.
Meer opties

Nederlands
Privacybeleid Servicevoorwaarden
"
      ["raw"]=>
      string(15256) "HTTP/1.1 200 OK
Content-Type: text/html; charset=utf-8
Vary: Sec-Fetch-Dest, Sec-Fetch-Mode, Sec-Fetch-Site
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Pragma: no-cache
Expires: Mon, 01 Jan 1990 00:00:00 GMT
Date: Tue, 16 Aug 2022 08:13:40 GMT
Content-Security-Policy: script-src 'report-sample' 'nonce-CbZa7ma1grUKc1m413SbUA' 'unsafe-inline';object-src 'none';base-uri 'self';report-uri /_/ConsentHttp/cspreport;worker-src 'self'
Accept-CH: Sec-CH-UA-Arch, Sec-CH-UA-Bitness, Sec-CH-UA-Full-Version, Sec-CH-UA-Full-Version-List, Sec-CH-UA-Model, Sec-CH-UA-Platform, Sec-CH-UA-Platform-Version
Cross-Origin-Opener-Policy: unsafe-none
Content-Security-Policy-Report-Only: require-trusted-types-for 'script';report-uri /_/ConsentHttp/cspreport
Cross-Origin-Resource-Policy: same-site
Permissions-Policy: ch-ua-arch=*, ch-ua-bitness=*, ch-ua-full-version=*, ch-ua-full-version-list=*, ch-ua-model=*, ch-ua-platform=*, ch-ua-platform-version=*
Content-Encoding: gzip
Server: ESF
X-XSS-Protection: 0
X-Frame-Options: SAMEORIGIN
X-Content-Type-Options: nosniff
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Connection: close
Transfer-Encoding: chunked

Inloggen
Google
Voordat je verdergaat naar Google
We gebruiken cookies en gegevens voor het volgende:
Google-services leveren en onderhouden
Uitval bijhouden en bescherming bieden tegen spam, fraude en misbruik
Doelgroepbetrokkenheid en sitestatistieken meten om inzicht te krijgen in hoe onze services worden gebruikt en de kwaliteit van die services te verbeteren
Als je Alles accepteren kiest, gebruiken we cookies en gegevens ook voor het volgende:
Nieuwe services ontwikkelen en verbeteren
Advertenties laten zien en de effectiviteit ervan meten
Gepersonaliseerde content laten zien (afhankelijk van je instellingen)
Gepersonaliseerde advertenties laten zien (afhankelijk van je instellingen)
Als je Alles afwijzen kiest, gebruiken we cookies niet voor deze aanvullende doeleinden.

Niet-gepersonaliseerde content wordt beïnvloed door factoren zoals de content die je op dat moment bekijkt, activiteit in je actieve zoeksessie en je locatie. Niet-gepersonaliseerde advertenties worden beïnvloed door de content die je op dat moment bekijkt en je algemene locatie. Gepersonaliseerde content en advertenties kunnen ook relevantere resultaten, aanbevelingen en op jou toegespitste advertenties omvatten die zijn gebaseerd op eerdere activiteit van deze browser, zoals uitgevoerde Google-zoekopdrachten. We gebruiken cookies en gegevens ook om te zorgen dat de functionaliteit geschikt is voor je leeftijd, als dit relevant is.

Selecteer Meer opties om meer informatie te bekijken, waaronder informatie over hoe je je privacyinstellingen beheert. Je kunt ook altijd naar g.co/privacytools gaan.
Meer opties

Nederlands
Privacybeleid Servicevoorwaarden
"
      ["headers"]=>
      object(Requests_Response_Headers)#16703 (1) {
        ["data":protected]=>
        array(18) {
          ["content-type"]=>
          array(1) {
            [0]=>
            string(24) "text/html; charset=utf-8"
          }
          ["vary"]=>
          array(1) {
            [0]=>
            string(46) "Sec-Fetch-Dest, Sec-Fetch-Mode, Sec-Fetch-Site"
          }
          ["cache-control"]=>
          array(1) {
            [0]=>
            string(46) "no-cache, no-store, max-age=0, must-revalidate"
          }
          ["pragma"]=>
          array(1) {
            [0]=>
            string(8) "no-cache"
          }
          ["expires"]=>
          array(1) {
            [0]=>
            string(29) "Mon, 01 Jan 1990 00:00:00 GMT"
          }
          ["date"]=>
          array(1) {
            [0]=>
            string(29) "Tue, 16 Aug 2022 08:13:40 GMT"
          }
          ["content-security-policy"]=>
          array(1) {
            [0]=>
            string(161) "script-src 'report-sample' 'nonce-CbZa7ma1grUKc1m413SbUA' 'unsafe-inline';object-src 'none';base-uri 'self';report-uri /_/ConsentHttp/cspreport;worker-src 'self'"
          }
          ["accept-ch"]=>
          array(1) {
            [0]=>
            string(151) "Sec-CH-UA-Arch, Sec-CH-UA-Bitness, Sec-CH-UA-Full-Version, Sec-CH-UA-Full-Version-List, Sec-CH-UA-Model, Sec-CH-UA-Platform, Sec-CH-UA-Platform-Version"
          }
          ["cross-origin-opener-policy"]=>
          array(1) {
            [0]=>
            string(11) "unsafe-none"
          }
          ["content-security-policy-report-only"]=>
          array(1) {
            [0]=>
            string(70) "require-trusted-types-for 'script';report-uri /_/ConsentHttp/cspreport"
          }
          ["cross-origin-resource-policy"]=>
          array(1) {
            [0]=>
            string(9) "same-site"
          }
          ["permissions-policy"]=>
          array(1) {
            [0]=>
            string(137) "ch-ua-arch=*, ch-ua-bitness=*, ch-ua-full-version=*, ch-ua-full-version-list=*, ch-ua-model=*, ch-ua-platform=*, ch-ua-platform-version=*"
          }
          ["content-encoding"]=>
          array(1) {
            [0]=>
            string(4) "gzip"
          }
          ["server"]=>
          array(1) {
            [0]=>
            string(3) "ESF"
          }
          ["x-xss-protection"]=>
          array(1) {
            [0]=>
            string(1) "0"
          }
          ["x-frame-options"]=>
          array(1) {
            [0]=>
            string(10) "SAMEORIGIN"
          }
          ["x-content-type-options"]=>
          array(1) {
            [0]=>
            string(7) "nosniff"
          }
          ["alt-svc"]=>
          array(1) {
            [0]=>
            string(162) "h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43""
          }
        }
      }
      ["status_code"]=>
      int(200)
      ["protocol_version"]=>
      float(1.1)
      ["success"]=>
      bool(true)
      ["redirects"]=>
      int(1)
      ["url"]=>
      string(129) "https://consent.google.com/ml?continue=https://www.google.com/maps?cid%3D4493464801819550785&gl=NL&m=0&pc=m&uxe=eomtm&hl=nl&src=1"
      ["history"]=>
      array(1) {
        [0]=>
        object(Requests_Response)#16712 (10) {
          ["body"]=>
          string(350) "

302 Moved

The document has moved
here.

"
          ["raw"]=>
          string(1863) "HTTP/1.1 302 Found
Location: https://consent.google.com/ml?continue=https://www.google.com/maps?cid%3D4493464801819550785&gl=NL&m=0&pc=m&uxe=eomtm&hl=nl&src=1
Cache-Control: private
Timing-Allow-Origin: https://www.google.com
Content-Type: text/html; charset=UTF-8
Content-Security-Policy-Report-Only: object-src 'none';base-uri 'self';script-src 'nonce-jhJkak4vlSBfyfk4tTJ8tQ' 'strict-dynamic' 'report-sample' 'unsafe-eval' 'unsafe-inline' https: http:;report-uri https://csp.withgoogle.com/csp/gws/maps-tactile
Cross-Origin-Opener-Policy: same-origin-allow-popups; report-to="gws"
Report-To: {"group":"gws","max_age":2592000,"endpoints":[{"url":"https://csp.withgoogle.com/csp/report-to/gws/maps-tactile"}]}
P3P: CP="This is not a P3P policy! See g.co/p3phelp for more info."
Date: Tue, 16 Aug 2022 08:13:40 GMT
Server: gws
Content-Length: 350
X-XSS-Protection: 0
X-Frame-Options: SAMEORIGIN
Set-Cookie: __Secure-ENID=6.SE=VgGXMbfmNLae_U49j2-xNyX9KSANb6dx9_kBZFAD-livbO1IFRxB0v04PdiGYCunZmFIdgJP9OAoodIXIXhoRNjx66Y0BlVOzVTDco86c0pQty6URPwm9u10tg4KQi3xaer_4s2q3OXLihut8AtZCDWLrFS_eCFQ8l-b67oyaCk; expires=Sat, 16-Sep-2023 00:31:58 GMT; path=/; domain=.google.com; Secure; HttpOnly; SameSite=lax
Set-Cookie: CONSENT=PENDING+531; expires=Thu, 15-Aug-2024 08:13:40 GMT; path=/; domain=.google.com; Secure
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Connection: close



302 Moved

The document has moved
here.

"
          ["headers"]=>
          object(Requests_Response_Headers)#16711 (1) {
            ["data":protected]=>
            array(15) {
              ["location"]=>
              array(1) {
                [0]=>
                string(129) "https://consent.google.com/ml?continue=https://www.google.com/maps?cid%3D4493464801819550785&gl=NL&m=0&pc=m&uxe=eomtm&hl=nl&src=1"
              }
              ["cache-control"]=>
              array(1) {
                [0]=>
                string(7) "private"
              }
              ["timing-allow-origin"]=>
              array(1) {
                [0]=>
                string(22) "https://www.google.com"
              }
              ["content-type"]=>
              array(1) {
                [0]=>
                string(24) "text/html; charset=UTF-8"
              }
              ["content-security-policy-report-only"]=>
              array(1) {
                [0]=>
                string(210) "object-src 'none';base-uri 'self';script-src 'nonce-jhJkak4vlSBfyfk4tTJ8tQ' 'strict-dynamic' 'report-sample' 'unsafe-eval' 'unsafe-inline' https: http:;report-uri https://csp.withgoogle.com/csp/gws/maps-tactile"
              }
              ["cross-origin-opener-policy"]=>
              array(1) {
                [0]=>
                string(41) "same-origin-allow-popups; report-to="gws""
              }
              ["report-to"]=>
              array(1) {
                [0]=>
                string(115) "{"group":"gws","max_age":2592000,"endpoints":[{"url":"https://csp.withgoogle.com/csp/report-to/gws/maps-tactile"}]}"
              }
              ["p3p"]=>
              array(1) {
                [0]=>
                string(62) "CP="This is not a P3P policy! See g.co/p3phelp for more info.""
              }
              ["date"]=>
              array(1) {
                [0]=>
                string(29) "Tue, 16 Aug 2022 08:13:40 GMT"
              }
              ["server"]=>
              array(1) {
                [0]=>
                string(3) "gws"
              }
              ["content-length"]=>
              array(1) {
                [0]=>
                string(3) "350"
              }
              ["x-xss-protection"]=>
              array(1) {
                [0]=>
                string(1) "0"
              }
              ["x-frame-options"]=>
              array(1) {
                [0]=>
                string(10) "SAMEORIGIN"
              }
              ["set-cookie"]=>
              array(2) {
                [0]=>
                string(289) "__Secure-ENID=6.SE=VgGXMbfmNLae_U49j2-xNyX9KSANb6dx9_kBZFAD-livbO1IFRxB0v04PdiGYCunZmFIdgJP9OAoodIXIXhoRNjx66Y0BlVOzVTDco86c0pQty6URPwm9u10tg4KQi3xaer_4s2q3OXLihut8AtZCDWLrFS_eCFQ8l-b67oyaCk; expires=Sat, 16-Sep-2023 00:31:58 GMT; path=/; domain=.google.com; Secure; HttpOnly; SameSite=lax"
                [1]=>
                string(94) "CONSENT=PENDING+531; expires=Thu, 15-Aug-2024 08:13:40 GMT; path=/; domain=.google.com; Secure"
              }
              ["alt-svc"]=>
              array(1) {
                [0]=>
                string(162) "h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43""
              }
            }
          }
          ["status_code"]=>
          int(302)
          ["protocol_version"]=>
          float(1.1)
          ["success"]=>
          bool(false)
          ["redirects"]=>
          int(0)
          ["url"]=>
          string(51) "https://www.google.com/maps?cid=4493464801819550785"
          ["history"]=>
          array(0) {
          }
          ["cookies"]=>
          object(Requests_Cookie_Jar)#16714 (1) {
            ["cookies":protected]=>
            array(2) {
              ["__Secure-ENID"]=>
              object(Requests_Cookie)#16707 (5) {
                ["name"]=>
                string(13) "__Secure-ENID"
                ["value"]=>
                string(176) "6.SE=VgGXMbfmNLae_U49j2-xNyX9KSANb6dx9_kBZFAD-livbO1IFRxB0v04PdiGYCunZmFIdgJP9OAoodIXIXhoRNjx66Y0BlVOzVTDco86c0pQty6URPwm9u10tg4KQi3xaer_4s2q3OXLihut8AtZCDWLrFS_eCFQ8l-b67oyaCk"
                ["attributes"]=>
                object(Requests_Utility_CaseInsensitiveDictionary)#16708 (1) {
                  ["data":protected]=>
                  array(6) {
                    ["expires"]=>
                    int(1694824318)
                    ["path"]=>
                    string(1) "/"
                    ["domain"]=>
                    string(10) "google.com"
                    ["secure"]=>
                    bool(true)
                    ["httponly"]=>
                    bool(true)
                    ["samesite"]=>
                    string(3) "lax"
                  }
                }
                ["flags"]=>
                array(4) {
                  ["creation"]=>
                  int(1660637620)
                  ["last-access"]=>
                  int(1660637620)
                  ["persistent"]=>
                  bool(false)
                  ["host-only"]=>
                  bool(false)
                }
                ["reference_time"]=>
                int(1660637620)
              }
              ["CONSENT"]=>
              object(Requests_Cookie)#16706 (5) {
                ["name"]=>
                string(7) "CONSENT"
                ["value"]=>
                string(11) "PENDING+531"
                ["attributes"]=>
                object(Requests_Utility_CaseInsensitiveDictionary)#16705 (1) {
                  ["data":protected]=>
                  array(4) {
                    ["expires"]=>
                    int(1723709620)
                    ["path"]=>
                    string(1) "/"
                    ["domain"]=>
                    string(10) "google.com"
                    ["secure"]=>
                    bool(true)
                  }
                }
                ["flags"]=>
                array(4) {
                  ["creation"]=>
                  int(1660637620)
                  ["last-access"]=>
                  int(1660637620)
                  ["persistent"]=>
                  bool(false)
                  ["host-only"]=>
                  bool(false)
                }
                ["reference_time"]=>
                int(1660637620)
              }
            }
          }
        }
      }
      ["cookies"]=>
      object(Requests_Cookie_Jar)#16714 (1) {
        ["cookies":protected]=>
        array(2) {
          ["__Secure-ENID"]=>
          object(Requests_Cookie)#16707 (5) {
            ["name"]=>
            string(13) "__Secure-ENID"
            ["value"]=>
            string(176) "6.SE=VgGXMbfmNLae_U49j2-xNyX9KSANb6dx9_kBZFAD-livbO1IFRxB0v04PdiGYCunZmFIdgJP9OAoodIXIXhoRNjx66Y0BlVOzVTDco86c0pQty6URPwm9u10tg4KQi3xaer_4s2q3OXLihut8AtZCDWLrFS_eCFQ8l-b67oyaCk"
            ["attributes"]=>
            object(Requests_Utility_CaseInsensitiveDictionary)#16708 (1) {
              ["data":protected]=>
              array(6) {
                ["expires"]=>
                int(1694824318)
                ["path"]=>
                string(1) "/"
                ["domain"]=>
                string(10) "google.com"
                ["secure"]=>
                bool(true)
                ["httponly"]=>
                bool(true)
                ["samesite"]=>
                string(3) "lax"
              }
            }
            ["flags"]=>
            array(4) {
              ["creation"]=>
              int(1660637620)
              ["last-access"]=>
              int(1660637620)
              ["persistent"]=>
              bool(false)
              ["host-only"]=>
              bool(false)
            }
            ["reference_time"]=>
            int(1660637620)
          }
          ["CONSENT"]=>
          object(Requests_Cookie)#16706 (5) {
            ["name"]=>
            string(7) "CONSENT"
            ["value"]=>
            string(11) "PENDING+531"
            ["attributes"]=>
            object(Requests_Utility_CaseInsensitiveDictionary)#16705 (1) {
              ["data":protected]=>
              array(4) {
                ["expires"]=>
                int(1723709620)
                ["path"]=>
                string(1) "/"
                ["domain"]=>
                string(10) "google.com"
                ["secure"]=>
                bool(true)
              }
            }
            ["flags"]=>
            array(4) {
              ["creation"]=>
              int(1660637620)
              ["last-access"]=>
              int(1660637620)
              ["persistent"]=>
              bool(false)
              ["host-only"]=>
              bool(false)
            }
            ["reference_time"]=>
            int(1660637620)
          }
        }
      }
    }
    ["filename":protected]=>
    NULL
    ["data"]=>
    NULL
    ["headers"]=>
    NULL
    ["status"]=>
    NULL
  }
}

So, in above example, the var_dump outputs an array of data, but not for the given URL. It shows data from the (302) redirected URL.

How can I achieve the same result as I did with cURL and CURLOPT_FOLLOWLOCATION?

Not sure if this helps, but you should correct the 'sslverify' => 'false' - because it should actually be 'sslverify' => false, i.e. not the string 'false' which is equivalent to a true. Also, redirections can be followed by setting the redirection to a non-zero number, which you've already done it, so what's the full output of var_dump($result); ? (just replace sensitive URLs and other data with dummy ones) — Sally CJ
– Sally CJ, Commented Aug 16, 2022 at 6:10
Note though that not all HTTP transports support redirections, but as far as I know, the core ones (curl and streams) do support redirections - and the default transport is cURL, but then the CURLOPT_FOLLOWLOCATION is (always) set to false. — Sally CJ
– Sally CJ, Commented Aug 16, 2022 at 6:16
@bosco cURL is the default transport, and so long as redirection is enabled, then redirection will be followed - internally via WordPress' own redirects handler, even if CURLOPT_FOLLOWLOCATION is false. So I don't know why would it not work when the code in question is good (except the sslverify part). But I'd probably try adding the returnFormat parameter as a query string. — Sally CJ
– Sally CJ, Commented Aug 16, 2022 at 8:12
@SallyCJ, you were right about the 'false' after 'sslverify'. I've fixed this and edited my question, but that wasn't the issue. In my edited question I've now posted a URL with the corresponding var_dump response I get back using wp_remote_get. As you can see the response is of the 302 redirected URL instead of the targeted URL. Exactly that response is happening in my cURL request too when I set CURLOPT_FOLLOWLOCATION to false. — DeltaG
– DeltaG, Commented Aug 16, 2022 at 8:29
@DeltaG the response looked good to me - WordPress did follow the redirect there (check the items keyed redirects and history respectively). But it's probably a Google consent issue... Or you didn't send the correct parameters and thus the second page halted with a message instead (no Location header). Try comparing both the request and response when using the curl_xx functions and when using wp_remote_get(), and you'd know what is/are missing. Have you tried adding the user-agent header to your wp_remote_get() args? — Sally CJ
– Sally CJ, Commented Aug 16, 2022 at 8:45

DeltaG · Accepted Answer · 2022-08-16 10:59:41Z

2

So it looks like I've managed to fix my own problem here. I don't know if this is the best thinkable solution to this exact problem, but it does work!

My assumptions about CURLOPT_FOLLOWLOCATION seem correct. As Sally CJ also explained in the comments, the default setting in WP is always set to false. That's why the data always contained contents from the (302) redirect instead of the actual targeted URL.

I've found this old thread and answer where the underlying CURLOPT_FOLLOWLOCATION is changed through the http_api_curl action. Using this action to override the default setting was the key for fixing my problem with wp_remote_get. With the function enabled (see below), my returned data is exactly the same as when using cURL.

function __set_curl_to_follow( &$handle ) {
    curl_setopt( $handle, CURLOPT_FOLLOWLOCATION, true );  
}
add_action( 'http_api_curl', '__set_curl_to_follow' );

Special thanks to Sally CJ and bosco for thinking with me on this!

answered Aug 16, 2022 at 10:59

DeltaG

9195 silver badges17 bronze badges

You're very welcome, but actually, just now I tried the exact same wp_remote_get() code in question and also used the same Google maps URL, and everything worked fine for me, and even after setting redirection to 0, I still received the correct response (a Google map page), both without having to force CURLOPT_FOLLOWLOCATION to true. So I wondered what exactly happened in your case, or the real reason why that cURL option had to be true. As for the sample response though, it looked like a cookie issue, so not sure if WordPress was at fault. 🤔

Sally CJ
– Sally CJ

2022-08-17 00:49:56 +00:00
Commented Aug 17, 2022 at 0:49
Nonetheless, I'm glad you managed to find the filter on your own - which BTW, I deliberately didn't mention in the comments, because in normal situations, using the filter isn't necessary for the purpose of following redirections - so if I were you, I'd try to investigate further to check if this was a core bug.

Sally CJ
– Sally CJ

2022-08-17 00:52:06 +00:00
Commented Aug 17, 2022 at 0:52
I'm glad you got it working! I never would have thought that such a thing would be necessary - definitely super curious as to what's going on there. If you do pursue it any further, please report back with your findings :)

bosco
– bosco

2022-08-18 21:04:04 +00:00
Commented Aug 18, 2022 at 21:04

Add a comment |

Stack Exchange Network

How to use the HTTP API (wp_remote_get) instead of cURL?

1 Answer 1

Your Answer

Linked

Hot Network Questions

How to use the HTTP API (wp_remote_get) instead of cURL?

1 Answer 1

Your Answer

Sign up or log in

Post as a guest

Linked

Related

Hot Network Questions