gstreamer Wiki - VideoHackfest/pixman

Here are some notes on how to integrate YCbCr into pixman:

Converting chroma subsampled formats requires interpolation when converting to RGB. This conversion is
- pixman_filter_neareest:
  - for each destination pixel determine location in source. Then take the nearest luma/chroma samples and apply the matrix.
- pixman_filter_bilinear:
  - for each destination pixel determine location in source, then determine the four closest luma samples and the four closest chroma samples, then bilinearly filter those and apply the matrix.
- pixman_filter_convolution:

       for each destination pixel
           s = location in source

           find nearest luma sample 

           align midpoint of convolution kernel with that sample

           for each pixel in convolution kernel,           
               find nearest luma sample, apply weight, add to sum
               find nearest chroma sample, apply weight, add to sum
           
           apply the color matrix.

When YUV formats are added to cairo, cairo will need to set
- pixman_filter_bilinear on a YUV pattern, even when there is no transformation. For RGB formats with an identity transformation, bilinear is the same as nearest. But as mentioned above, this is not the case for YCbCr formats.
Planar formats will be handled with something like this:

         pixman_image_create_bits (format, char **planes, int **rowstrides);

If you pass NULL for plane data, the bits will be 0, not
- black. This is actually already the case because it's somewhat unclear what a premultiplied (0, 0, 0, 0) ARGB pixel actually is.
Chroma samples have a defined location; it may be different from
- video codec to video codec. I believe ds said that it would be fine to use an enum value for each format as opposed to trying to exhaustively describe all the YUV formats. This implies that the chroma shift is embedded in the name. Is there a standard way of referring to chroma shifts?

Proposed API:

typedef enum
{
    PIXMAN_COLOR_MATRIX_NONE,
    PIXMAN_COLOR_MATRIX_HD,
    PIXMAN_COLOR_MATRIX_SD,
    PIXMAN_COLOR_MATRIX_JPEG
} pixman_color_matrix_t:

/* - Default is MATRIX_NONE.
 * - If you set a YCrCb format, then you should also set
 *   a matrix, or things will not make sense
 */
void
pixman_image_set_color_matrix (pixman_image_t        *image,
                               pixman_color_matrix_t  matrix);

/* 'jpeg' means half-sited chroma positions
 *  'mpeg2' means co-sited positions
 */
typedef enum
{
    PIXMAN_420_planar_jpeg,
    PIXMAN_420_planar_mpeg2,
    PIXMAN_nv12_planar_jpeg,
    PIXMAN_nv21_planar_mpeg2,
    PIXMAN_yvyu_packed_jpeg,
    PIXMAN_yvyu_packed_mpeg2,
    PIXMAN_yuyv_packed_jpeg,
    PIXMAN_yuyv_packed_mpeg2,

    PIXMAN_y444,
} pixman_format_t;