视频转换：h.264从UDP套接字到Mat对象

Question

我有一个带有picam的Raspberry pi，我想要将视频流式传输到服务器来处理它并运行对象检测软件。我通过UDP套接字将视频流发送到服务器并重新获取我无法转换为Mats的字节数组。

python发件人：

import socket
import time
import picamera
import sys

# msg = b'test'
addr = ("192.168.1.79", 7777)
start = time.time()
# Connect a client socket to my_server:8000 (change my_server to the
# hostname of your server)
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
s.connect(('192.168.1.79', 7777))
con = s.makefile('wb');
print("Conectado")
# Make a file-like object out of the connection
# connection = s.makefile('wb')
try:
    with picamera.PiCamera() as camera:
        camera.resolution = (640, 480)
        camera.framerate = 24
        print("Cargando camara")
        camera.start_preview()
        time.sleep(2)
        print("Enviando")
        camera.start_recording(con, format='h264')
        camera.wait_recording(10)
        camera.stop_recording()        
finally:
    con.close()
    s.close()

c＃接收器：

using Emgu.CV;
using Emgu.CV.UI;
using System;
using System.Drawing;
using System.Net;
using System.Net.Sockets;


namespace EgmuCv {
    class Decoder{
        private const int listenPort = 7777;


        public static void streamToImage(ImageBox img) {
            UdpClient listener = new UdpClient(listenPort);
            IPEndPoint groupEP = new IPEndPoint(IPAddress.Any, listenPort);
            Mat mat = new Mat(new Size(640, 480), Emgu.CV.CvEnum.DepthType.Cv8U, 3);

            try {
                while (true) {
                    Console.WriteLine("Waiting for broadcast");
                    byte[] bytes = listener.Receive(ref groupEP);
                    // CvInvoke.Imdecode(bytes, Emgu.CV.CvEnum.ImreadModes.ReducedColor8, mat);
                    /** Code to turn bytes into Mat **/
                    // img.Image = mat;
                }
            } catch (SocketException e) {
                Console.WriteLine(e);
            } finally {
                listener.Close();
            }

        }
    }
}

Answer 1

经过一些研究，我发现h.264 shold没有流式传输UDP但TCP，如果你想在对象检测程序中使用视频，h.264根本不是一个好主意。一个更好的想法是使用TCP套接字发送单个MJPEG帧，稍微慢一点，但是使用640X480 @ 30这比使用更多并且转换为Mat对象方式更容易