-
Enhance RAW image processing with Core Image
Harness the power of version 9 of the Core Image RAW processing APIs to dramatically improve image quality in your apps, with improved sharpness and more defined color, while using the Apple Neural Engine for optimal performance. Take advantage of the CIRAWFilter API to let your users edit RAW photos by changing exposure, noise reduction, sharpness, contrast and more. And explore new CIImageProcessor APIs that optimize performance by giving you precise control over tile sizing and buffer management.
Chapters
- 0:00 - Introduction
- 0:52 - How Core Image supports RAW
- 2:48 - The evolution of RAW support
- 3:33 - RAW 9 overview
- 3:56 - RAW 9 quality improvements
- 5:50 - Enable and edit RAW 9 with CIRAWFilter API
- 8:33 - RAW 9 performance overview
- 9:19 - Interactive editing
- 10:52 - Exporting to other formats
- 11:50 - New CIImageProcessor features
Resources
Related Videos
WWDC22
WWDC21
-
Search this video…
-
-
11:08 - Contact for exports
let exportCtx = CIContext(options : [ .cacheIntermediate : false, .memoryLimit : 512 ]) -
12:23 - CIImageProcessor with explicit output tile sizes
import CoreImage class MyProcessor: CIImageProcessorKernel { override class func roi(forInput input: Int32, arguments: [String : Any]?, outputRect: CGRect) -> CGRect { return outputRect } override class func process(with inputs: [CIImageProcessorInput]?, arguments: [String : Any]?, output: CIImageProcessorOutput) throws { guard let input = inputs?.first, let iBuffer = input.pixelBuffer, let oBuffer = output.pixelBuffer else { return } let iRegion = input.region let oRegion = output.region // controlled by Core Image // MyCopyBuffer(iBuffer,iRegion, oBuffer,oRegion) } } let extent = inImg.extent let tileSize = 512.0 // whatever tile size you want var tiles: [CIVector] = [] for y in stride(from: extent.minY, to: extent.maxY, by: tileSize) { for x in stride(from: extent.minX, to: extent.maxX, by: tileSize) { let tile = CGRect(x: x, y: y, width: min(tileSize, extent.maxX - x), height: min(tileSize, extent.maxY - y)) tiles.append(CIVector(cgRect: tile)) } } let result = try MyProcessor.apply(withTiledExtent: tiles, inputs: [inImg], arguments: [:]) -
14:24 - CIImageProcessor using temporary PixelBuffer
import CoreImage class MyProcessor: CIImageProcessorKernel { override class func process(with inputs: [CIImageProcessorInput]?, arguments: [String: Any]?, output: CIImageProcessorOutput) throws { guard let input = inputs?.first, let srcPixelBuffer = input.pixelBuffer, let dstPixelBuffer = output.pixelBuffer else { return } // Get a scratch buffer from Core Image's cache guard let scratch = output.temporaryPixelBuffer(identifier : "myScratch", format: kCVPixelFormatType_64RGBAHalf, width: Int(output.region.width), height: Int(output.region.height), pixelBufferAttributes: nil) else { return } // Step 1: copy input CVPixelBuffer → scratch // Step 2: process pixels in scratch // Step 3: copy scratch → output CVPixelBuffer } }
-
-
- 0:00 - Introduction
Enhancements to Core Image and its support for RAW image files across Apple platforms.
- 0:52 - How Core Image supports RAW
An overview of the RAW processing stages, including demosaic, denoising, convolution, and color adjustments, and their support in Apple's operating systems and developer frameworks.
- 2:48 - The evolution of RAW support
A look at how Apple's RAW processing pipeline has evolved across nine versions, now supporting 784 camera models.
- 3:33 - RAW 9 overview
Explore the new RAW 9 processing pipeline, which leverages a Core ML model to significantly improve demosaic and denoise for the highest image quality.
- 3:56 - RAW 9 quality improvements
A side-by-side comparison of RAW 8 and RAW 9 demonstrating significant improvements in sharpness, color accuracy, and noise reduction.
- 5:50 - Enable and edit RAW 9 with CIRAWFilter API
How to opt in to RAW 9 using CIRAWFilter's decoderVersion property, check which camera models are supported, and use the CIRAWFilter editing properties to give people full control over how their RAW images appear.
- 8:33 - RAW 9 performance overview
Best practices for optimizing performance when using RAW 9.
- 9:19 - Interactive editing
Best practices for rendering a RAW file repeatedly at screen resolution, including using the scaleFactor property, enabling cacheIntermediates, the Extended Virtual Addressing entitlement, and rendering to Metal-backed MTKViews.
- 10:52 - Exporting to other formats
Best practices for exporting multiple RAW files to HEIF or JPEG at full resolution, including disabling cacheIntermediates, tuning the memoryLimit option, and using Core Image's heifRepresentation and jpegRepresentation methods.
- 11:50 - New CIImageProcessor features
Discover new API features added to CIImageProcessor, including explicit output tile sizes and temporary buffers.