Detailed explanation of how to implement real-time video data interaction between C# and Python

When we were doing RTSP|RTMP playback, we met many developers. Most of their visual algorithms were run under python and needed to efficiently implement video data interaction between C# and Python. The commonly used methods are as follows:

Method 1: Transfer video data through HTTP request

Server side (Python)

Create an HTTP server using web frameworks such as Flask or Django to provide video data through the API interface.

from flask import Flask, send_file, request, jsonify  
import cv2  
import io  
import base64  
  
app = Flask(__name__)  
  
def get_video_frame():  
    # Read video frames    cap = ('your_video.mp4')  
    ret, frame = ()  
    ()  
    if not ret:  
        return None, "No more frames"  
  
    # Convert to RGB format    frame_rgb = (frame, cv2.COLOR_BGR2RGB)  
  
    # Convert to byte stream    ret, buffer = ('.jpg', frame_rgb)  
    if not ret:  
        return None, "Could not encode frame"  
  
    # Convert a byte stream to a base64 string    img_str = base64.b64encode(buffer).decode('utf-8')  
    return img_str, 200  
  
@('/video_frame', methods=['GET'])  
def video_frame():  
    img_str, status_code = get_video_frame()  
    if img_str is None:  
        return jsonify({"error": status_code}), status_code  
    return jsonify({"image": img_str})  
  
if __name__ == '__main__':  
    (debug=True)

Client (C#)

Use HttpClient to get the video frame from the Python server and display it in PictureBox.

using System;  
using ;  
using ;  
using ;  
using ;  
using ;  
  
public class VideoForm : Form  
{  
    private PictureBox pictureBox;  
    private HttpClient httpClient;  
  
    public VideoForm()  
    {  
        pictureBox = new PictureBox  
        {  
            Dock = ,  
            SizeMode =   
        };  
        (pictureBox);  
  
        httpClient = new HttpClient();  
        Timer timer = new Timer();  
         = 100; // Adjust the interval as needed  
         += Timer_Tick;  
        ();  
    }  
  
    private async void Timer_Tick(object sender, EventArgs e)  
    {  
        try  
        {  
            HttpResponseMessage response = await ("http://localhost:5000/video_frame");  
            ();  
            string jsonResponse = await ();  
            dynamic jsonData = (jsonResponse);  
            string base64String = ;  
            byte[] imageBytes = Convert.FromBase64String(base64String);  
            using (MemoryStream ms = new MemoryStream(imageBytes))  
            {  
                Image image = (ms);  
                 = image;  
            }  
        }  
        catch (Exception ex)  
        {  
            ($"Error: {}");  
        }  
    }  
  
    [STAThread]  
    public static void Main()  
    {  
        ();  
        (new VideoForm());  
    }  
}

Method 2: Transfer video data through ZeroMQ

ZeroMQ is a high-performance asynchronous message library suitable for applications requiring low latency and high throughput.

Server side (Python)

usepyzmqLibrary sends video frames.

import zmq  
import cv2  
import numpy as np  
  
context = ()  
socket = ()  
("tcp://*:5555")  
  
cap = ('your_video.mp4')  
  
while True:  
    ret, frame = ()  
    if not ret:  
        break  
    frame_rgb = (frame, cv2.COLOR_BGR2RGB)  
    frame_bytes = frame_rgb.tobytes()  
    socket.send_string("FRAME", )  
    (frame_bytes, flags=0, copy=False)

Client (C#)

useNetMQThe library receives video frames.

using System;  
using ;  
using ;  
using ;  
using ;  
using ;  
using NetMQ;  
using ;  
  
public class VideoForm : Form  
{  
    private PictureBox pictureBox;  
    private SubscriberSocket subscriber;  
  
    public VideoForm()  
    {  
        pictureBox = new PictureBox  
        {  
            Dock = ,  
            SizeMode =   
        };  
        (pictureBox);  
  
        var address = "tcp://localhost:5555";  
        using (var context = ())  
        {  
            subscriber = ();  
            (address);  
            ("FRAME");  
  
            (() => ReceiveFramesAsync(subscriber));  
        }  
    }  
  
    private async Task ReceiveFramesAsync(SubscriberSocket subscriber)  
    {  
        while (true)  
        {  
            var topic = await ();  
            var frameBytes = await ();  
  
            if (topic == "FRAME")  
            {  
                int width = 640; // Adjust based on your video resolution  
                int height = 480; // Adjust based on your video resolution  
                int stride = width * 3;  
                Bitmap bitmap = new Bitmap(width, height, PixelFormat.Format24bppRgb);  
                BitmapData bitmapData = (new Rectangle(0, 0, width, height), , );  
                (frameBytes, 0, bitmapData.Scan0, );  
                (bitmapData);  
  
                 = bitmap;  
            }  
        }  
    }  
  
    [STAThread]  
    public static void Main()  
    {  
        ();  
        (new VideoForm());  
    }  
}

Method 3: Shared memory or file

If C# and Python are running on the same machine, data exchange can be performed via shared memory or file system. This method is relatively simple, but its performance may not be as good as the previous two methods.

The above is a detailed explanation of how to implement real-time video data interaction between C# and Python. For more information about C# and Python video data interaction, please follow my other related articles!