Kinesis WebRTC C SDK - Convert Frames to Images

Question

I managed to wrap kvsWebRTCClientViewer in a python C extension and I also managed to expose frame data to python via callback function as bytes array but I don't understand how to convert these frames to images. Does it require gstreamer pipeline?

Following is C function to expose frame data to python:

VOID handleFrame(UINT64 customData, PFrame pFrame)
{
    PyObject* retval;
    long result;
    
    UNUSED_PARAM(customData);
    DLOGV("Frame received. TrackId: %" PRIu64 ", Size: %u, Flags %u", pFrame->trackId, pFrame->size, pFrame->flags);
    PSampleStreamingSession pSampleStreamingSession = (PSampleStreamingSession) customData;
    if (pSampleStreamingSession->firstFrame) {
        pSampleStreamingSession->firstFrame = FALSE;
        pSampleStreamingSession->startUpLatency = (GETTIME() - pSampleStreamingSession->offerReceiveTime) / HUNDREDS_OF_NANOS_IN_A_MILLISECOND;
        printf("Start up latency from offer to first frame: %" PRIu64 "ms\n", pSampleStreamingSession->startUpLatency);
    }
    
    retval = PyObject_CallFunction(frame_callable, "y", pFrame->frameData);
    
    if (retval && PyLong_Check(retval))
        result = (int)PyLong_AsLong(retval);
    else
        result = -1;
        
    Py_XDECREF(retval);
}

And following is python code to start viewer and handle frames:

from samples import libkvsWebrtcClientViewer

def frame_handler(frame):
    print("Frame Type: ", type(frame))

libkvsWebrtcClientViewer.startViewer(frame_handler, "channel-name")

Any help in this regard is highly appreciated.

Farooq Zaman · Accepted Answer · 2023-03-08 08:27:15Z

0

Managed to fix this issue by attaching GStreamer pipeline to the video stream and storing output as HLS stream.

appsrc name=appsrc-video ! h264parse ! queue max-size-buffers=1000 leaky=downstream ! identity name=\"identity-elem\" signal-handoffs=TRUE ! mpegtsmux ! hlssink location=/tmp/hls/segment_%05d.ts playlist-location=/tmp/hls/playlist.m3u8 target-duration=300

Then I opened HLS stream in python using OpenCV VideoCapture method.

camera = cv2.VideoCapture('/tmp/hls/segment_00000.ts')

answered Mar 8, 2023 at 8:27

Farooq Zaman

5071 gold badge6 silver badges22 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Kinesis WebRTC C SDK - Convert Frames to Images

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related