MediaPipe Live Stream 用于手势模型不接受视频数据

Question

Google docs 不清楚如何在检测手势时为 Live Stream 模式提供视频数据。

它只是在文档中说

data=numpy_frame_from_opencv

，没有实际的例子。

我收到此错误：

TypeError: __init__(): incompatible constructor arguments. The following argument types are supported:
    1. mediapipe.python._framework_bindings.image.Image(image_format: mediapipe::ImageFormat_Format, data: numpy.ndarray[numpy.uint8])
    2. mediapipe.python._framework_bindings.image.Image(image_format: mediapipe::ImageFormat_Format, data: numpy.ndarray[numpy.uint16])
    3. mediapipe.python._framework_bindings.image.Image(image_format: mediapipe::ImageFormat_Format, data: numpy.ndarray[numpy.float32])

这是我的主要功能。我正在使用经过训练的模型here

def capture_video(camera_id=0, width=640, height=480):
  cap = cv2.VideoCapture(camera_id)
  cap.set(cv2.CAP_PROP_FRAME_WIDTH, width)
  cap.set(cv2.CAP_PROP_FRAME_HEIGHT, height)

  with vision.GestureRecognizer.create_from_options(options) as recognizer:
    while cap.isOpened():
        success, image = cap.read()
        if not success:
            print("Ignoring empty camera frame.")
            continue

        # Convert the BGR image to RGB
        rgb_image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
        height, width, _ = rgb_image.shape

        # Create the mp.Image object with the correct data type
        mp_image = mp.Image(
            image_format=mp.ImageFormat.SRGB,
            data=rgb_image.astype(np.uint8).flatten(),  # Ensure data type is uint8
            width=width,
            height=height
        )

        # Process the image for gesture recognition
        recognizer.recognize_async(mp_image, int(time.time() * 1000))

        # For visualization and gesture handling (implement as needed)

        # Display the image
        cv2.imshow('MediaPipe Gesture Recognition', image)

        # Exit on 'q' press
        if cv2.waitKey(5) & 0xFF == ord('q'):
            break

  cap.release()

Answer 1

您是否尝试将 rgb_image 传递给 mp.Image 中的数据？