Write the Kernel Code

The code that performs per-pixel processing resides in a file with the .cikernel extension. You can include more than one kernel routine in this file. You can also include other routines if you want to make your code modular. You specify a kernel using a subset of OpenGL Shading Language (glslang) and the Core Image extensions to it. See Core Image Kernel Language Reference for information on allowable elements of the language.

A kernel routine signature must return a vector (vec4) that contains the result of mapping the source pixel to a destination pixel. Core Image invokes a kernel routine once for each pixel. Keep in mind that your code can’t accumulate knowledge from pixel to pixel. A good strategy when writing your code is to move as much invariant calculation as possible from the actual kernel and place it in the Objective-C portion of the filter.

Listing 9-1 shows the kernel routine for a haze removal filter. A detailed explanation for each numbered line of code follows the listing. (There are examples of other pixel-processing routines in Kernel Routine Examples and in Image Unit Tutorial.)

Listing 9-1  A kernel routine for the haze removal filter

kernel vec4 myHazeRemovalKernel(sampler src,             // 1

                     __color color,

                    float distance,

                    float slope)

{

    vec4   t;

    float  d;

    d = destCoord().y * slope  +  distance;              // 2

    t = unpremultiply(sample(src, samplerCoord(src)));   // 3

    t = (t - d*color) / (1.0-d);                         // 4

    return premultiply(t);                               // 5

}

Here’s what the code does:

1. Takes four input parameters and returns a vector. When you declare the interface for the filter, you must make sure to declare the same number of input parameters as you specify in the kernel. The kernel must return a vec4 data type.

2. Calculates a value based on the y-value of the destination coordinate and the slope and distance input parameters. The destCoord routine (provided by Core Image) returns the position, in working space coordinates, of the pixel currently being computed.

3. Gets the pixel value, in sampler space, of the sampler src that is associated with the current output pixel after any transformation matrix associated with the src is applied. Recall that Core Image uses color components with premultiplied alpha values. Before processing, you need to unpremultiply the color values you receive from the sampler.

4. Calculates the output vector by applying the haze removal formula, which incorporates the slope and distance calculations and adjusts for color.

5. Returns a vec4 vector, as required. The kernel performs a premultiplication operation before returning the result because Core Image uses color components with premultiplied alpha values.

Use Quartz Composer to Test the Kernel Routine

Quartz Composer is an easy-to-use development tool that you can use to test kernel routines.

To download Quartz Composer

1. Open Xcode.

2. Choose Xcode > Open Developer Tool > More Developer Tools...

   Choosing this item will take you to developer.apple.com.

3. Sign in to developer.apple.com.

   You should then see the Downloads for Apple Developers webpage.

4. Download the Graphics Tools for Xcode package, which contains Quartz Composer.

Quartz Composer provides a patch—Core Image Filter—into which you can place your kernel routine.You simply open the Inspector for the Core Image Filter patch, and either paste or type your code into the text field, as shown in Figure 9-2.

After you enter the code, the patch’s input ports are automatically created according to the prototype of the kernel function, as you can see in Figure 9-3. The patch always has a single output port, which represents the resulting image produced by the kernel.

The simple composition shown in Figure 9-3 imports an image file using the Image Importer patch, processes it through the kernel, then renders the result onscreen using the Billboard patch. Your kernel can use more than one image or, if it generates output, it might not require any input images.

The composition you build to test your kernel can be more complex than that shown in Figure 9-3. For example, you might want to chain your kernel routine with other built-in Core Image filters or with other kernel routines. Quartz Composer provides many other patches that you can use in the course of testing your kernel routine.

Declare an Interface for the Filter

The .h file for the filter contains the interface that specifies the filter inputs, as shown in Listing 9-2. The haze removal kernel has four input parameters: a source, color, distance, and slope. The interface for the filter must also contain these input parameters. The input parameters must be in the same order as specified for the filter, and the data types must be compatible between the two.

Listing 9-2  Code that declares the interface for a haze removal filter

@interface MyHazeFilter: CIFilter

{

    CIImage   *inputImage;

    CIColor   *inputColor;

    NSNumber  *inputDistance;

    NSNumber  *inputSlope;

}

@end

Write an Init Method for the CIKernel Object

The implementation file for the filter contains a method that initializes a Core Image kernel object (CIKernel) with the kernel routine specified in the .cikernel file. A .cikernel file can contain more than one kernel routine. A detailed explanation for each numbered line of code appears following the listing.

Listing 9-3  An init method that initializes the kernel

static CIKernel *hazeRemovalKernel = nil;

- (id)init

{

    if(hazeRemovalKernel == nil)                                         // 1

    {

        NSBundle    *bundle = [NSBundle bundleForClass: [self class]];   // 2

        NSString    *code = [NSString stringWithContentsOfFile: [bundle

                                pathForResource: @"MyHazeRemoval"

                                ofType: @"cikernel"]];                   // 3

        NSArray     *kernels = [CIKernel kernelsWithString: code];       // 4

        hazeRemovalKernel = kernels[0];                                  // 5

    }

    return [super init];

}

Here’s what the code does:

1. Checks whether the CIKernel object is already initialized.

2. Returns the bundle that dynamically loads the CIFilter class.

3. Returns a string created from the file name at the specified path, which in this case is the MyHazeRemoval.cikernel file.

4. Creates a CIKernel object from the string specified by the code argument. Each routine in the .cikernel file that is marked as a kernel is returned in the kernels array. This example has only one kernel in the .cikernel file, so the array contains only one item.

5. Sets hazeRemovalKernel to the first kernel in the kernels array. If the .cikernel file contains more than one kernel, you would also initialize those kernels in this routine.

Write a Custom Attributes Method

A customAttributes method allows clients of the filter to obtain the filter attributes such as the input parameters, default values, and minimum and maximum values. (See CIFilter Class Reference for a complete list of attributes.) A filter is not required to provide any information about an attribute other than its class, but a filter must behave in a reasonable manner if attributes are not present.

Typically, these are the attributes that your customAttributes method would return:

• Input and output parameters

• Attribute class for each parameter you supply (mandatory)

• Minimum, maximum, and default values for each parameter (optional)

• Other information as appropriate, such as slider minimum and maximum values (optional)

Listing 9-4 shows the customAttributes method for the Haze filter. The input parameters inputDistance and inputSlope each have minimum, maximum, slider minimum, slider maximum, default and identity values set. The slider minimum and maximum values are used to set up the sliders shown in Figure 9-1. The inputColor parameter has a default value set.

Listing 9-4  The customAttributes method for the Haze filter

- (NSDictionary *)customAttributes

{

    return @{

        @"inputDistance" :  @{

            kCIAttributeMin       : @0.0,

            kCIAttributeMax       : @1.0,

            kCIAttributeSliderMin : @0.0,

            kCIAttributeSliderMax : @0.7,

            kCIAttributeDefault   : @0.2,

            kCIAttributeIdentity  : @0.0,

            kCIAttributeType      : kCIAttributeTypeScalar

            },

        @"inputSlope" : @{

            kCIAttributeSliderMin : @-0.01,

            kCIAttributeSliderMax : @0.01,

            kCIAttributeDefault   : @0.00,

            kCIAttributeIdentity  : @0.00,

            kCIAttributeType      : kCIAttributeTypeScalar

            },

         kCIInputColorKey : @{

         kCIAttributeDefault : [CIColor colorWithRed:1.0

                                               green:1.0

                                                blue:1.0

                                               alpha:1.0]

           },

   };

}

Write an Output Image Method

An outputImage method creates a CISampler object for each input image (or image mask), creates a CIFilterShape object (if appropriate), and applies the kernel method. Listing 9-5 shows an outputImage method for the haze removal filter. The first thing the code does is to set up a sampler to fetch pixels from the input image. Because this filter uses only one input image, the code sets up only one sampler.

The code calls the apply:arguments:options: method of CIFilter to produce a CIImage object. The first parameter to the apply method is the CIKernel object that contains the haze removal kernel function. (See Write the Kernel Code.) Recall that the haze removal kernel function takes four arguments: a sampler, a color, a distance, and the slope. These arguments are passed as the next four parameters to the apply:arguments:options: method in Listing 9-5. The remaining arguments to the apply method specify options (key-value pairs) that control how Core Image should evaluate the function. You can pass one of three keys: kCIApplyOptionExtent, kCIApplyOptionDefinition, or kCIApplyOptionUserInfo. This example uses the kCIApplyOptionDefinition key to specify the domain of definition (DOD) of the output image. See CIFilter Class Reference for a description of these keys and for more information on using the apply:arguments:options: method.

The final argument nil, specifies the end of the options list.

Listing 9-5  A method that returns the image output from a haze removal filter

- (CIImage *)outputImage

{

    CISampler *src = [CISampler samplerWithImage: inputImage];

    return [self apply: hazeRemovalKernel, src, inputColor, inputDistance,

        inputSlope, kCIApplyOptionDefinition, [src definition], nil];

}

Listing 9-5 is a simple example. The implementation for your outputImage method needs to be tailored to your filter. If your filter requires loop-invariant calculations, you would include them in the outputImage method rather than in the kernel.

Register the Filter

Ideally, you’ll package the filter as an image unit, regardless of whether you plan to distribute the filter to others or use it only in your own app. If you plan to package this filter as an image unit, you’ll register your filter using the CIPlugInRegistration protocol described in Packaging and Loading Image Units. You can skip the rest of this section.

If for some reason you do not want to package the filter as an image unit (which is not recommended), you’ll need to register your filter using the registration method of the CIFilter class described shown in Listing 9-6. The initialize method calls registerFilterName:constructor:classAttributes:. You should register only the display name (kCIAttributeFilterDisplayName) and the filter categories (kCIAttributeFilterCategories). All other filters attributes should be specified in the customAttributes method. (See Write a Custom Attributes Method).

The filter name is the string for creating the haze removal filter when you want to use it. The constructor object specified implements the filterWithName: method (see Write a Method to Create Instances of the Filter). The filter class attributes are specified as an NSDictionary object. The display name—what you’d show in the user interface—for this filter is Haze Remover.

Listing 9-6  Registering a filter that is not part of an image unit

+ (void)initialize

{

    [CIFilter registerFilterName: @"MyHazeRemover"

                     constructor: self

                 classAttributes:

     @{kCIAttributeFilterDisplayName : @"Haze Remover",

       kCIAttributeFilterCategories : @[

               kCICategoryColorAdjustment, kCICategoryVideo,

               kCICategoryStillImage, kCICategoryInterlaced,

               kCICategoryNonSquarePixels]}

     ];

}

Write a Method to Create Instances of the Filter

If you plan to use this filter only in your own app, then you’ll need to implement a filterWithName: method as described in this section. If you plan to package this filter as an image unit for use by third-party developers, then you can skip this section because your packaged filters can use the filterWithName: method provided by the CIFilter class.

The filterWithName: method shown in Listing 9-7 creates instances of the filter when they are requested.

Listing 9-7  A method that creates instance of a filter

+ (CIFilter *)filterWithName: (NSString *)name

{

    CIFilter  *filter;

    filter = [[self alloc] init];

    return filter;

}

After you follow these steps to create a filter, you can use the filter in your own app. See Using Your Own Custom Filter for details. If you want to make a filter or set of filters available as a plug-in for other apps, see Packaging and Loading Image Units.

A Simple ROI Function

If your ROI function does not require data to be passed to it in the userInfo parameter, then you don’t need to include that argument, as shown in Listing 9-9. The code in Listing 9-9 outsets the sampler by one pixel, which is a calculation used by an edge-finding filter or any 3x3 convolution.

Listing 9-9  A simple ROI function

CIKernelROICallback callback = ^(int index, CGRect rect) {

    return CGRectInset(rect, -1.0, -1.0);

};

Note that this function ignores the index value. If your kernel uses only one sampler, then you can ignore the index. If your kernel uses more than one sampler, you must make sure that you return the ROI that’s appropriate for the specified sampler. You’ll see how to do that in the sections that follow.

An ROI Function for a Glass Distortion Filter

Listing 9-10 shows an ROI function for a glass distortion filter. This function returns an ROI for two samplers. Sampler 0 represents the image to distort and sampler 1 represents the texture used for the glass.

As with other uses of blocks (Objective-C) or closures (Swift), an ROI callback can capture state from the context in which is defined. You can use this behavior to provide additional parameters to your routine, as seen in this example: an external value scale controls the inset applied by the ROI function. (When using the older setROISelector: API, you provide such values with the kCIApplyOptionUserInfo key in the options dictionary you pass to the apply:arguments:options: method.

All of the glass texture (sampler 1) needs to be referenced because the filter uses the texture as a rectangular pattern. As a result, the function returns an infinite rectangle as the ROI. An infinite rectangle is a convention that indicates to use all of a sampler. (The constant CGRectInfinite is defined in the Quartz 2D API.)

Listing 9-10  An ROI function for a glass distortion filter

float scale = 1.0f;

CIKernelROICallback distortionCallback = ^(int index, CGRect rect) {

    if (index == 0) {

        CGFloat s = scale * 0.5f;

        return CGRectInset(rect, -s,-s);

    }

    return CGRectInfinite;

};

An ROI Function for an Environment Map

Listing 9-11 shows an ROI function that returns the ROI for a kernel that uses three samplers, one of which is an environment map. The ROI for sampler 0 and sampler 1 coincide with the DOD. For that reason, the code returns the destination rectangle passed to it for samplers other than sampler 2.

Sampler 2 uses captured values that specify the size of the environment map to create the rectangle that specifies the region of interest.

Listing 9-11  Supplying a routine that calculates the region of interest

CGRect sampler2ROI = CGRectMake(0, 0, envMapWidth, envMapHeight);

CIKernelROICallback envMapROICallback = ^(int index, CGRect rect) {

    if (samplerIndex == 2) {

        return sampler2ROI;

    }

    return destination;

};

Specifying Sampler Order

As you saw from the previous examples, a sampler has an index associated with it. When you supply an ROI function, Core Image passes a sampler index to you. A sampler index is assigned on the basis of its order when passed to the apply method for the filter. You call apply from within the filter’s outputImage routine, as shown in Listing 9-12.

In this listing, notice especially the numbered lines of code that set up the samplers and show how to provide them to the kernel. A detailed explanation for each of these lines appears following Listing 9-12.

Listing 9-12  An output image routine for a filter that uses an environment map

- (CIImage *)outputImage

{

    int i;

    CISampler *src, *blur, *env;                                      // 1

    CIVector *envscale;

    CIKernel *kernel;

    src = [CISampler samplerWithImage:inputImage];                    // 2

    blur = [CISampler samplerWithImage:inputHeightImage];             // 3

    env = [CISampler samplerWithImage:inputEnvironmentMap];           // 4

    envscale = [CIVector vectorWithX:[inputEMapWidth floatValue]

                     Y:[inputEMapHeight floatValue]];

    i = [inputKind intValue];

    if ([inputHeightInAlpha boolValue]) {

        i += 8;

    }

    kernel = roundLayerKernels[i];

    return [kernel applyWithExtent: [self extent]                      // 5

                       roiCallback: envMapROICallback                  // 6

                         arguments: @[                                 // 7

                               blur,

                               env,

                               @( pow(10.0, [inputSurfaceScale floatValue]) ),

                               envscale,

                               inputEMapOpacity,

                         ]];

 }

Here’s what the code does:

1. Declares variables for each of the three samplers that are needed for the kernel.

2. Sets up a sampler for the input image. The ROI for this sampler coincides with the DOD.

3. Sets up a sampler for an image used for input height. The ROI for this sampler coincides with the DOD.

4. Sets up a sampler for an environment map. The ROI for this sampler does not coincide with the DOD, which means you must supply an ROI function.

5. Applies the kernel to produce a Core Image image (CIImage object), using the options that follow.

6. The ROI function with the kernel that needs to use it.

7. The arguments to the kernel function, which must be type compatible with the function signature of the kernel function. (The kernel function source is not shown here; assume they are type compatible in this example).

   The order of the sampler arguments determine its index. The first sampler supplied to the kernel is index 0. In this case, that’s the src sampler. The second sampler supplied to the kernel—blur— is assigned index 1. The third sampler—env—is assigned index 2. It’s important to check your ROI function to make sure that you provide the appropriate ROI for each sampler.

Computing a Brightening Effect

Listing 9-14 computes a brightening effect. A detailed explanation for each numbered line of code appears following the listing.

Listing 9-14  A kernel routine that computes a brightening effect

kernel vec4 brightenEffect (sampler src, float k)

{

  vec4 currentSource = sample (src, samplerCoord (src));         // 1

  currentSource.rgb = currentSource.rgb + k * currentSource.a;   // 2

  return currentSource;                                          // 3

}

Here’s what the code does:

1. Looks up the source pixel in the sampler that is associated with the current output position.

2. Adds a bias to the pixel value. The bias is k scaled by the alpha value of the pixel to make sure the pixel value is premultiplied.

3. Returns the changed pixel.

Computing a Multiply Effect

Listing 9-15 shows a kernel routine that computes a multiply effect. The code looks up the source pixel in the sampler and then multiplies it by the value passed to the routine.

Listing 9-15  A kernel routine that computes a multiply effect

kernel vec4 multiplyEffect (sampler src, __color mul)

{

  return sample (src, samplerCoord (src)) * mul;

}

Computing a Hole Distortion

Listing 9-16 shows a kernel routine that computes a hole distortion. Note that there are many ways to compute a hole distortion. A detailed explanation for each numbered line of code appears following the listing.

Listing 9-16  A kernel routine that computes a hole distortion

kernel vec4 holeDistortion (sampler src, vec2 center, vec2 params)   // 1

{

  vec2 t1;

  float distance0, distance1;

  t1 = destCoord () - center;                                        // 2

  distance0 = dot (t1, t1);                                          // 3

  t1 = t1 * inversesqrt (distance0);                                 // 4

  distance0 = distance0 * inversesqrt (distance0) * params.x;        // 5

  distance1 = distance0 - (1.0 / distance0);                         // 6

  distance0 = (distance0 < 1.0 ? 0.0 : distance1) * params.y;        // 7

  t1 = t1 * distance0 + center;                                      // 8

  return sample (src, samplerTransform (src, t1));                   // 9

}

Here’s what the code does:

1. Takes three parameters—a sampler, a vector that specifies the center of the hole distortion, and the params vector, which contains (1/radius, radius).

2. Creates the vector t1 from the center to the current working coordinates.

3. Squares the distance from the center and assigns the value to the distance0 variable.

4. Normalizes t1. (Makes t1 a unit vector.)

5. Computes the parametric distance from the center (distance squared * 1/distance) * 1/radius. This value is 0 at the center and 1 where the distance is equal to the radius.

6. Creates a hole with the appropriate distortion around it. (x – 1/sqrt (x))

7. Makes sure that all pixels within the hole map from the pixel at the center, then scales up the distorted distance function by the radius.

8. Scales the vector to create the distortion and then adds the center back in.

9. Returns the distorted sample from the source texture.