HTTP Stream using Axios (Node JS)

Question

I'm trying to stream price data via HTTP and I use axios to make normal REST API requests, but I don't know how to handle 'Transfer Encoding': 'chunked' type of requests.

This code just hangs and doesn't produce any error so assume it's working but not able to process the response:

const { data } = await axios.get(`https://stream.example.com`, {headers: 
{Authorization: `Bearer ${token}`, 'Content-Type': 'application/octet- 
stream'}})

console.log(data) // execution hangs before reaching here

Appreciate your help.

WORKING SOLUTION: As pointed out from the answer below, we need to add a responseType: stream as an axios option and also add an event listener on the response.

Working code:

const response = await axios.get(`https://stream.example.com`, {
  headers: {Authorization: `Bearer ${token}`}, 
  responseType: 'stream'
});

const stream = response.data
stream.on('data', data => { 
  data = data.toString()
  console.log(data) 
})

HTTP streams are far simpler than websockets, and can take advantage of HTTP/2 — Evert
– Evert, Commented Mar 19, 2022 at 20:56

jfriend00 · Accepted Answer · 2022-03-19 17:19:22Z

46

FYI, sending the content-type header for a GET request is meaningless. The content-type header applies to the BODY of the http request and there is no body for a GET request.

With the axios() library, if you want to get direct access to the response stream, you use the responseType option to tell Axios that you want access to the raw response stream:

const response = await axios.get('https://stream.example.com', {
    headers: {Authorization: `Bearer ${token}`, 
    responseType: 'stream'
});

const stream = response.data;

stream.on('data', data => {
    console.log(data);
});

stream.on('end', () => {
    console.log("stream done");
});

Axios document reference here.

edited Mar 19, 2022 at 17:19

answered Mar 19, 2022 at 0:14

jfriend00

711k104 gold badges1.1k silver badges1k bronze badges

Sign up to request clarification or add additional context in comments.

9 Comments

Jat90 Over a year ago

Thanks, I'm now able to print the response, which has a 200 HTTP Code, and the stream is kept alive. How do I process new data as it comes in? Is there some event handler like stream.on('message', function() { //do something} ) ? I can't find it in the docs

jfriend00 Over a year ago

It is presumably an http.IncomingMessage object which is a sub-class of the standard nodejs stream.readable. You would use stream.on('data', data => { console.log(data) }); to get notified of incoming data on the stream. The data arrives in arbitrary chunks (perhaps even split among different data events so you will have to know what the format is and parse it appropriately.

Jat90 Over a year ago

That worked! Adding the event listener on stream as you described did the trick. Thanks

CR Sardar Over a year ago

It does NOT work, it gives "xhr.js:211 The provided value 'stream' is not a valid enum value of type XMLHttpRequestResponseType."

jfriend00 Over a year ago

@CRSardar - Please post your own question and show your own code. This concept works and worked for the question here. Your situation is apparently different so we would need to see your exact code in order to help you. Your particular error sounds like you are using TypeScript and/or sounds like perhaps you are not using axios or not using the right TypeScript typedefs for axios. But, let's not fix or debate that here. Please post your own question and show your own code. Also a downvote here is perhaps not really useful to anyone. Your situation is different and needs its own question.

|

gerardnll · Accepted Answer · 2024-02-06 10:18:38Z

8

Since NodeJS 10 you can also use asynchronous iteration to read from streams. Read more here.

const response = await axios.get('https://stream.example.com', {
    headers: {Authorization: `Bearer ${token}`, 
    responseType: 'stream'
});

const stream = response.data;

for await (const chunk of stream) {
    console.log(chunk);
}

answered Feb 6, 2024 at 10:18

gerardnll

2,2951 gold badge18 silver badges18 bronze badges

Comments

usersina · Accepted Answer · 2023-09-14 18:17:27Z

4

I've had problems getting it to work with axios (mainly this). Therefore I ended up using the fetch API, allowing me do this elegantly in client side code:

const stream = await generateStream()
for await (const chunk of stream) {
  console.log(chunk)
}

Here's the full answer that uses the fetch API from another similar thread. It includes the a full example.

answered Sep 14, 2023 at 18:17

usersina

1,9651 gold badge24 silver badges43 bronze badges

Collectives™ on Stack Overflow

HTTP Stream using Axios (Node JS)

3 Answers 3

9 Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

9 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related