如何对来自 RTSP 源的流进行硬件解码

Question

我已经使用 ffmpeg 设置了 RTSP 流：

ffmpeg -loglevel warning -re -stream_loop -1 -i path/to/file.mp4 -c:v libx264 -preset superfast -tune zerolatency -pix_fmt yuv420p -b:v 600k -max_muxing_queue_size 1024 -profile:v baseline -an -filter:v scale=1280:-1 -f rtsp rtsp://<IP>:8554/test

然后我尝试使用以下 gstreamer 管道访问该流：

gst-launch-1.0 rtspsrc location=rtsp://<IP>:8554/test drop-on-latency=true ! rtph264depay ! h264parse ! queue ! nvv4l2decoder ! queue ! nvvidconv ! 'video/x-raw, format=(string)RGBA' ! fakesink

我使用 fakesink 只是为了查看管道是否工作以及 NVDEC 是否已启用（我使用 Jetson Orin Nano 来运行该管道）。但是，当运行此管道时，我收到以下错误：

libEGL warning: DRI3: failed to query the version
libEGL warning: DRI2: failed to authenticate
Setting pipeline to PAUSED ...
Opening in BLOCKING MODE 
Pipeline is live and does not need PREROLL ...
Progress: (open) Opening Stream
Progress: (connect) Connecting to rtsp://192.168.12.13:8554/flare
Progress: (open) Retrieving server options
Progress: (open) Retrieving media info
Progress: (request) SETUP stream 0
Progress: (open) Opened Stream
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Progress: (request) Sending PLAY request
Progress: (request) Sending PLAY request
Progress: (request) Sent PLAY request
NvMMLiteOpen : Block : BlockType = 261 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
NvMMLiteBlockCreate : Block : BlockType = 261 
Stream format not found, dropping the frame
Stream format not found, dropping the frame

如您所见，我收到了几条类型为

Stream format not found, dropping the frame

的消息

为了调试，我更改了 RTSP 源，并使用以下命令创建了 RTSP 服务器：（编译了 test-launch-c:

./test-launch "filesrc location=video.m4v ! qtdemux ! rtph264pay name=pay0 pt=96 "

video.m4v

采用 H264 编码。

现在，使用新的 RTSP 源运行以前的管道，我没有错误。

为什么会发生这种情况以及如何更改失败的 gstreamer 管道以适应不同的源格式？

测试已更新

作为

SeB

的观察者，我在这两种情况下都在调试模式下运行之前的管道，以下是我的发现：

FFMPEG Stream（有问题的）

/GstPipeline:pipeline0/nvv4l2decoder:nvv4l2decoder0.GstPad:sink: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, pixel-aspect-ratio=(fraction)1/1, width=(int)1280, height=(int)720, framerate=(fraction)30/1, interlace-mode=(string)progressive, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, parsed=(boolean)true, profile=(string)constrained-baseline, level=(string)3.1

RTSP 流（工作正常）

/GstPipeline:pipeline0/nvv4l2decoder:nvv4l2decoder0.GstPad:sink: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, pixel-aspect-ratio=(fraction)1/1, width=(int)596, height=(int)336, framerate=(fraction)30000/1001, interlace-mode=(string)progressive, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, parsed=(boolean)true, profile=(string)high, level=(string)3

我看到工作管道中的profile和level分别是“high”和“3”，所以我尝试将我之前的管道更改为：

gst-launch-1.0 rtspsrc location=rtsp://192.168.18.10:8554/test drop-on-latency=true ! rtph264depay ! h264parse ! video/x-h264,profile=high,level=3 ! queue ! nvv4l2decoder enable-max-performance=1 ! queue ! nvvidconv ! 'video/x-raw, format=(string)RGBA' ! fakesink

虽然我不再看到错误：

libEGL warning: DRI3: failed to query the version
libEGL warning: DRI2: failed to authenticate
Setting pipeline to PAUSED ...
Opening in BLOCKING MODE 
Pipeline is live and does not need PREROLL ...
Progress: (open) Opening Stream
Progress: (connect) Connecting to rtsp://192.168.18.10:8554/test
Progress: (open) Retrieving server options
Progress: (open) Retrieving media info
Progress: (request) SETUP stream 0
Progress: (open) Opened Stream
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Progress: (request) Sending PLAY request
Progress: (request) Sending PLAY request
Progress: (request) Sent PLAY request
NvMMLiteOpen : Block : BlockType = 261 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
NvMMLiteBlockCreate : Block : BlockType = 261

解码器未被使用。

我需要添加更多上限或更改它们的应用位置吗？

Answer 1

您可以调整 ffmpeg 命令以使用级别 3.1 的高配置文件：

... -profile:v high -level:v 3.1 ...

另请参阅：https://en.wikipedia.org/wiki/Advanced_Video_Coding

切换到 gstreamer RTSP 服务器也可能是一个更好的选择，特别是如果稍后迁移到另一个 Orin 模块，这可能会利用 HW NVENC。