防止 SpriteKit Metal Renderer 超出纹理大小限制

Question

我有 SpriteKit 节点，我使用 SKEffectNode 在其上应用核心图像滤镜。精灵是用户在场景中添加的图像，可以是任意大小。一些过滤器会改变渲染精灵的大小。当该大小超过 Metal 允许的限制时，应用程序会崩溃并出现如下错误：

-[MTLTextureDescriptorInternal validateWithDevice:]:1357: failed assertion `Texture Descriptor Validation
MTLTextureDescriptor has width (9050) greater than the maximum allowed size of 8192.
MTLTextureDescriptor has height (9050) greater than the maximum allowed size of 8192.

如何防止我实时进行的任何图像处理超出 Metal 的限制？

这是 SpriteKit 代码：

var mySprite: SKSpriteNode!
var myEffectNode: SKEffectNode!
var sliderInput: Double = 0

override func didMove(to view: SKView) {    
    // Sprite from an image
    // The user picks the image, it could be of any size 
    mySprite = SKSpriteNode(imageNamed: "myImage")
    mySprite.name = "mySprite"
        
    myEffectNode = SKEffectNode()
    myEffectNode.addChild(mySprite)
    
    addChild(myEffectNode)
}

// Called by SwiftUI
// The filter changes dynamically in real-time

func updateEffects() {
    myEffectNode.filter = CIFilter(name: "CIZoomBlur", parameters: [
        "inputAmount": sliderInput
    ])
}

所需的伪代码：

func carefullyUpdateEffects() {
    // Compute the image processing
    // Get the texture size limit on this device
    // Check if the expected output of the processing exceeds the limit
        // If no, proceed and render
        // If yes, handle case
}

Answer 1

我已经取得了进展或者想分享。

问题

核心图像过滤器返回的结果可能会在渲染时超出 Metal 对每个纹理的大小限制。

示例：1024x1024 图像上的

CIFilter(name: "CIZoomBlur", parameters: ["inputAmount": 140])

会生成 17325*17325 的图像。当该过滤器将其结果返回给 SpriteKit 时，渲染器崩溃。

如何在发送到渲染器之前获得过滤器的预期输出？

解决方案

子类

CIFilter

 并在该自定义类中进行检查。在那里，我们可以重写 

outputImage

 属性，该属性的类型为 

CIImage

。 CIImage 是一个 Core Image 对象，它表示图像，但在明确要求之前不会渲染。因此，我们可以在输出发送到渲染器之前检查其

extent.size

。

下面的自定义类是一个有效的解决方案，可以防止 SpriteKit 的渲染器在应用 Core Image 过滤器后崩溃。它基于这个

answer，它是为了链接 SpriteKit 中的 Core Image 过滤器而编写的。一石X鸟！

import Core Image

class ChainFilters: CIFilter {
    let chainedFilters: [CIFilter]
    let metalSizeLimit: CGFloat = 8000 /// TBD. Get the texture limit of the device's GPU family, and substract some safety margin
    @objc dynamic var inputImage: CIImage?
    
    init(filters: [CIFilter]) {
        self.chainedFilters = filters
        super.init()
    }
    
    /// Override `outputImage` to:
    /// - Chain multiple filters
    /// - Check the output result of each filter before it is passed on
    override var outputImage: CIImage? {
        get {
            let imageKey = "inputImage"
            var workingImage = self.inputImage
            for filter in chainedFilters {
                assert(filter.inputKeys.contains(imageKey))
                filter.setValue(workingImage, forKey: imageKey)
                guard let result = filter.outputImage else {
                    assertionFailure("Filter failed: \(filter.name)")
                    return nil
                }
                
                /// Start Metal limit test
                /// We check the `extent` property of the working image, which is a `CIImage`
                /// A CIImage is an object that represents an image but is not rendered until explicitly asked to
                
                if (result.extent.size.width > metalSizeLimit || result.extent.size.height > metalSizeLimit) {
                    print("Input size = \(workingImage?.extent.size ?? CGSize.zero)")
                    print("Output size = \(result.extent.size)")
                    return workingImage
                }
                /// End Metal limit test
                
                workingImage = result
            }
            /// Here the output image is returned, to be rendered in SpriteKit or elsewhere
            return workingImage
        }
    }
    
    required init?(coder aDecoder: NSCoder) { fatalError("init(coder:) has not been implemented") }
}

SpriteKit 中的使用示例：

var myDynamicValue: Double = 0

let filters: [CIFilter] = [
    CIFilter(name: "CIZoomBlur", parameters: ["inputAmount": myDynamicValue]),
    CIFilter(name: "CIPixellate", parameters: ["inputScale": 8])
].compactMap { $0 }
        
let appliedFilters = ChainFilters(filters: filters)

myEffectNode.filter = appliedFilters

要做的事

改进 CIFilter 子类内部的输出检查。例如，当超出限制时，返回一个没有使过滤器超出限制的值的结果，而不是返回未过滤的基础输入图像。