High CPU usage while reading an HLS stream using OpenCV GStreamer

Question

I have the following code that creates a gstreamer pipeline to process an HLS stream.

cap = cv2.VideoCapture(
    f"souphttpsrc is_live=true location={hls_stream_link} ! hlsdemux ! 
    queue ! decodebin ! videorate ! video/x-raw,framerate=1/1 ! 
    videoconvert ! appsink max-buffers=1 drop=true sync=false",
cv2.CAP_GSTREAMER)

I now read the stream like this:

while True:
    success, frame = cap.read()
    time.sleep(1.0)

Note, I read the stream at 1 FPS and have the properties max-buffers=1 drop=true sync=false. By doing this, I always grab the latest frame from the stream the buffer has to offer.

The issue with this is the CPU usage is extremely high, spiking to 120% at times on my i7 machine. Removing the sleep would make it worse.

Any solutions or ideas as to why processing an HLS stream is so CPU intensive would be fantastic. Also, ideas on how to reduce usage would also be great.

Florian Zwoch · Accepted Answer · 2019-03-26 10:10:44Z

1

You are decoding the complete stream. When done in software this can be quite costly. One potential optimization step I see here is to add a GstPadProbe before the decoder and parse a bit into the bit stream and drop all samples that are not IDR. Then you will only send IDR frames to the decoder. What would reduce the load of the decoder, however you will end up with a new frame only after each IDR-frame interval, which is usually every 2 seconds (but may vary between each).

answered Mar 26, 2019 at 10:10

Florian Zwoch

7,5782 gold badges15 silver badges24 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

farza Over a year ago

Thanks so much for the answer, that makes a TON of sense.

farza Over a year ago

Any idea what this would look like using OpenCV for the CLI? I'm quite new to this whole world.

Florian Zwoch Over a year ago

I have no idea about opencv and the interaction with GStreamer. From a CLI point of view you cannot do it. It requires some code logic usually done in an application or library code. It can be done in python too. You need to some real code work with the GStreamer API.

farza Over a year ago

Is there a specific reason why the stream needs to be decoded frame by frame no matter what? Why is it difficult or odd to simply "pick out" frames?

Florian Zwoch Over a year ago

That's how majority of video codecs work. Frames depend on their predecessors (that makes it so effective). But in order to decode a specific frame you need all of its depended frames decoded too or the frame makes no sense. The exception are IDR frames which mark the beginning of a group of pictures and can be decoded by themselves.

Collectives™ on Stack Overflow

High CPU usage while reading an HLS stream using OpenCV GStreamer

1 Answer 1

5 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

5 Comments

Your Answer

Sign up or log in

Post as a guest

Related