zerodcetensordec

Tensor decoder element for Zero-DCE low-light image enhancement, and more generally for any image-to-image model emitting the image-rgb-hwc-out tensor: denoising, deblurring, super-resolution, style transfer.

Zero-DCE was introduced in "Zero-Reference Deep Curve Estimation for Low-Light Image Enhancement" by Chunle Guo, Chongyi Li, Jichang Guo, Chen Change Loy, Junhui Hou, Sam Kwong and Runmin Cong (CVPR 2020, https://arxiv.org/abs/2001.06826), and extended in "Learning to Enhance Low-Light Image via Zero-Reference Deep Curve Estimation" by Chongyi Li, Chunle Guo and Chen Change Loy (IEEE TPAMI 2021, https://arxiv.org/abs/2103.00860).

Unlike the other tensor decoders of this plugin, which attach analytics metadata to the frame they receive, this one replaces the frame content: the inferred image tensor is written over the RGB_F32LE frame, or denormalized to 8 bit samples over the RGB frame.

The tensor resolution is a property of the model instance and is not rescaled here, so the frame that reaches this element must already have been scaled to it before inference.

Model file can be found here : https://gitlab.collabora.com/gstreamer/onnx-models

 gst-launch-1.0 v4l2src device=/dev/video1 ! videoconvertscale ! \
     onnxinference model-file=/work/data/models/models/zero_dce++.onnx ! \
     zerodcetensordec ! videoconvert ! glimagesink

This takes a frame from a video device, performs low-light image enhancement via onnxinference on it and decodes the inferred tensor with zerodcetensordec into the output frame displayed by glimagesink.

GObject
    ╰──GInitiallyUnowned
        ╰──GstObject
            ╰──GstElement
                ╰──GstBaseTransform
                    ╰──zerodcetensordec

video/x-raw:
         format: { RGB_F32LE, RGB }
          width: [ 1, 2147483647 ]
         height: [ 1, 2147483647 ]
      framerate: [ 0/1, 2147483647/1 ]
        tensors: "tensorgroups\,\ image-rgb-hwc-out\=\(/uniquelist\)\{\ \(caps\)\"tensor/strided\\\,\\\ tensor-id\\\=\\\(string\\\)image-rgb-hwc-out\\\,\\\ dims\\\=\\\(int\\\)\\\<\\\ 1\\\,\\\ \\\[\\\ 0\\\,\\\ 2147483647\\\ \\\]\\\,\\\ \\\[\\\ 0\\\,\\\ 2147483647\\\ \\\]\\\,\\\ 3\\\ \\\>\\\,\\\ dims-order\\\=\\\(string\\\)row-major\\\,\\\ type\\\=\\\(string\\\)float32\"\,\ \(caps\)\"tensor/strided\\\,\\\ tensor-id\\\=\\\(string\\\)image-rgb-hwc-out\\\,\\\ dims\\\=\\\(int\\\)\\\<\\\ \\\[\\\ 0\\\,\\\ 2147483647\\\ \\\]\\\,\\\ \\\[\\\ 0\\\,\\\ 2147483647\\\ \\\]\\\,\\\ 3\\\ \\\>\\\,\\\ dims-order\\\=\\\(string\\\)row-major\\\,\\\ type\\\=\\\(string\\\)float32\"\ \}\;"

video/x-raw:
         format: { RGB_F32LE, RGB }
          width: [ 1, 2147483647 ]
         height: [ 1, 2147483647 ]
      framerate: [ 0/1, 2147483647/1 ]