Android - 人脸检测

Question

使用

MlKitAnalyzer

进行人脸检测时，有没有办法获取图像数据？我能够绘制边界框并获取所有轮廓/地标，但现在我需要对检测到的面部进行 rgb 计算，为此我找不到访问 imageProxy 的方法。任何帮助将不胜感激。 https://developer.android.com/reference/androidx/camera/mlkit/vision/MlKitAnalyzer.Result

cameraController.setImageAnalysisAnalyzer(cameraExecutor, MlKitAnalyzer(
            listOf(faceDetectionComponent.getFaceDetector()),
            COORDINATE_SYSTEM_VIEW_REFERENCED,
            cameraExecutor
        ) { result ->
           //draw the bounding box - works well!

           //need access to image data ??
          }

我需要对检测到的脸部进行以下 rgb 计算，但 imageProxy 或与图像相关的任何其他数据似乎在

MlKitAnalyzer.Result

中不可用

fun Bitmap.calculateAverageColor(): RGBData {
    var r = 0
    var g = 0
    var b = 0
    val height = this.height
    val width = this.width
    var n = 0.0
    val pixels = IntArray(width * height)
    this.getPixels(pixels, 0, width, 0, 0, width, height)
    var i = 0
    while (i < pixels.size) {
        val color = pixels[i]
        r += Color.red(color)
        g += Color.green(color)
        b += Color.blue(color)
        n++
        i += PIXEL_SPACING
    }
    return RGBData(r.toDouble() / n, g.toDouble() / n, b.toDouble() / n)
}

Answer 1

尝试这个解决方案，

配置

MlKitAnalyzer

，以便您可以在结果回调中访问

ImageProxy

，并将

ImageProxy

转换为

Bitmap

并裁剪到面部区域。

cameraController.setImageAnalysisAnalyzer(cameraExecutor, MlKitAnalyzer(
    listOf(faceDetectionComponent.getFaceDetector()),
    COORDINATE_SYSTEM_VIEW_REFERENCED,
    cameraExecutor
) { result ->
    // Get the detected faces (assuming a single face for simplicity)
    val faceList = result.getValue(faceDetectionComponent.getFaceDetector())
    if (faceList != null && faceList.isNotEmpty()) {
        val face = faceList[0] // Get the first detected face
        val boundingBox = face.boundingBox

        // Access the ImageProxy
        val imageProxy = result.imageProxy
        if (imageProxy != null) {
            // Convert ImageProxy to Bitmap
            val bitmap = imageProxy.toBitmap()

            // Crop the Bitmap to the bounding box of the face
            val faceBitmap = Bitmap.createBitmap(
                bitmap,
                boundingBox.left,
                boundingBox.top,
                boundingBox.width(),
                boundingBox.height()
            )

            // Calculate the average color on the cropped face bitmap
            val avgColor = faceBitmap.calculateAverageColor()
            
            // Use avgColor as needed (e.g., log, display, etc.)

            // Don't forget to close the imageProxy
            imageProxy.close()
        }
    }
})

要将

ImageProxy

转换为

Bitmap

，这里有一个实用函数：

fun ImageProxy.toBitmap(): Bitmap {
    val yBuffer = planes[0].buffer
    val uBuffer = planes[1].buffer
    val vBuffer = planes[2].buffer

    val ySize = yBuffer.remaining()
    val uSize = uBuffer.remaining()
    val vSize = vBuffer.remaining()

    val nv21 = ByteArray(ySize + uSize + vSize)
    yBuffer.get(nv21, 0, ySize)
    vBuffer.get(nv21, ySize, vSize)
    uBuffer.get(nv21, ySize + vSize, uSize)

    val yuvImage = YuvImage(nv21, ImageFormat.NV21, width, height, null)
    val out = ByteArrayOutputStream()
    yuvImage.compressToJpeg(Rect(0, 0, width, height), 100, out)
    val imageBytes = out.toByteArray()
    return BitmapFactory.decodeByteArray(imageBytes, 0, imageBytes.size)
}