0

我正在尝试将像素缓冲区从实时视频源修改AVFoundation为流式传输OpenTok's API。但是每当我尝试这样做并通过 OpenTok 提供它时consumeFrame，它就会崩溃。

我这样做是为了可以应用不同的实时视频效果（滤镜、贴纸等）。我尝试CGImage->CVPixelBuffer使用不同的方法进行转换，但没有任何效果。

func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection) {
    if !capturing || videoCaptureConsumer == nil {
        return
    }

    guard let imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer)
        else {
            print("Error acquiring sample buffer")
            return
    }

    guard let videoInput = videoInput
        else {
            print("Capturer does not have a valid input")
            return
    }

    let time = CMSampleBufferGetPresentationTimeStamp(sampleBuffer)
    videoFrame.clearPlanes()

    videoFrame.timestamp = time
    let height = UInt32(CVPixelBufferGetHeight(imageBuffer))
    let width = UInt32(CVPixelBufferGetWidth(imageBuffer))

    if width != captureWidth || height != captureHeight {
        updateCaptureFormat(width: width, height: height)
    }

    // This is where I convert CVImageBuffer->CIImage, modify it, turn it into CGImage, then CGImage->CVPixelBuffer
    guard let finalImage = makeBigEyes(imageBuffer) else { return }

    CVPixelBufferLockBaseAddress(finalImage, CVPixelBufferLockFlags(rawValue: 0))

    videoFrame.format?.estimatedCaptureDelay = 10
    videoFrame.orientation = .left

    videoFrame.clearPlanes()
    videoFrame.planes?.addPointer(CVPixelBufferGetBaseAddress(finalImage))

    delegate?.finishPreparingFrame(videoFrame)

    videoCaptureConsumer!.consumeFrame(videoFrame)

    CVPixelBufferUnlockBaseAddress(finalImage, CVPixelBufferLockFlags(rawValue: 0))
}

这是我的CGImage->CVPixelBuffer方法：

    func buffer(from image: UIImage) -> CVPixelBuffer? {
    let attrs = [kCVPixelBufferCGImageCompatibilityKey: kCFBooleanTrue, kCVPixelBufferCGBitmapContextCompatibilityKey: kCFBooleanTrue] as CFDictionary
    var pixelBuffer : CVPixelBuffer?
    let status = CVPixelBufferCreate(kCFAllocatorDefault, Int(image.size.width), Int(image.size.height), kCVPixelFormatType_32ARGB, attrs, &pixelBuffer)
    guard (status == kCVReturnSuccess) else {
        return nil
    }

    CVPixelBufferLockBaseAddress(pixelBuffer!, CVPixelBufferLockFlags(rawValue: 0))
    let pixelData = CVPixelBufferGetBaseAddress(pixelBuffer!)

    let rgbColorSpace = CGColorSpaceCreateDeviceRGB()
    let context = CGContext(data: pixelData, width: Int(image.size.width), height: Int(image.size.height), bitsPerComponent: 8, bytesPerRow: CVPixelBufferGetBytesPerRow(pixelBuffer!), space: rgbColorSpace, bitmapInfo: CGImageAlphaInfo.noneSkipFirst.rawValue)

    context?.translateBy(x: 0, y: image.size.height)
    context?.scaleBy(x: 1.0, y: -1.0)

    UIGraphicsPushContext(context!)
    image.draw(in: CGRect(x: 0, y: 0, width: image.size.width, height: image.size.height))
    UIGraphicsPopContext()
    CVPixelBufferUnlockBaseAddress(pixelBuffer!, CVPixelBufferLockFlags(rawValue: 0))

    return pixelBuffer
}

我在第一帧收到此错误：

* 由于未捕获的异常“NSRangeException”而终止应用程序，原因：“* -[NSConcretePointerArray pointerAtIndex:]：尝试访问索引 1 处的指针超出边界 1”

如果你做到了这一步，谢谢你的阅读。我已经在这个问题上停留了一段时间，所以任何类型的指针都将不胜感激。谢谢。

4

1 回答 1

2

由于您要将相机（NV12）帧转换为 RGB，因此您需要设置pixelFormat为OTPixelFormatARGBonvideoFrame.format

于 2019-01-22T20:46:38.797 回答