classifiertensordec
Tensor decoder element for image classification models using a common output format: a vector of per-class confidence levels.
The tensor id selects how the values are interpreted: classification-generic-out holds
raw scores that are softmaxed here, classification-generic-softmaxed-out holds
probabilities that are used as they are. uint8 values are scaled to [0, 1] first.
The most confident class is attached to the buffer as classification metadata. Without
labels-file the classes are labelled by their index.
Test image file, model file and labels file can be found here : https://gitlab.collabora.com/gstreamer/onnx-models
gst-launch-1.0 filesrc location=onnx-models/images/bus.jpg \
! jpegdec ! videoconvertscale ! video/x-raw,width=224,height=224 \
! onnxinference execution-provider=cpu model-file=onnx-models/models/mobilenetv2-10.onnx \
! classifiertensordec labels-file=onnx-models/labels/synset.txt \
! fakesink
This takes a JPEG, classifies it via onnxinference and decodes the inferred tensor with
classifiertensordec.
Tensor format must be:
- Dims: [
class_count] or [batch-size,class_count] - Datatype: float32 or uint8
Tensor [M,N]
|Batch 0 | Class 0 confidence level | ... | Class N-1 confidence level |
| ... |
|Batch M-1 | Class 0 confidence level | ... | Class N-1 confidence level |
In-memory tensor format:
|Batch 0, Class 0 confidence level |
|Batch 0, ... |
|Batch 0, Class N-1 confidence level |
| ... |
|Batch M-1, Class 0 confidence level |
|Batch M-1, ... |
|Batch M-1, Class N-1 confidence level |
Hierarchy
GObject ╰──GInitiallyUnowned ╰──GstObject ╰──GstElement ╰──GstBaseTransform ╰──classifiertensordec
Factory details
Authors: – Jeremy Whiting
Classification: – Tensordecoder
Rank – primary
Plugin – rsanalytics
Package – gst-plugin-analytics
Pad Templates
sink
video/x-raw(ANY):
tensors: "tensorgroups\,\ classification-generic-out\=\(/uniquelist\)\{\ \(caps\)\"tensor/strided\\\,\\\ tensor-id\\\=\\\(string\\\)classification-generic-out\\\,\\\ dims\\\=\\\(int\\\)\\\<\\\ \\\[\\\ 0\\\,\\\ 1\\\ \\\]\\\,\\\ \\\[\\\ 1\\\,\\\ 2147483647\\\ \\\]\\\ \\\>\\\,\\\ dims-order\\\=\\\(string\\\)row-major\\\,\\\ type\\\=\\\(string\\\)\\\{\\\ float32\\\,\\\ uint8\\\ \\\}\\\;\\\ tensor/strided\\\,\\\ tensor-id\\\=\\\(string\\\)classification-generic-out\\\,\\\ dims\\\=\\\(int\\\)\\\<\\\ \\\[\\\ 1\\\,\\\ 2147483647\\\ \\\]\\\ \\\>\\\,\\\ dims-order\\\=\\\(string\\\)row-major\\\,\\\ type\\\=\\\(string\\\)\\\{\\\ float32\\\,\\\ uint8\\\ \\\}\"\ \}\;"
video/x-raw(ANY):
tensors: "tensorgroups\,\ classification-generic-softmaxed-out\=\(/uniquelist\)\{\ \(caps\)\"tensor/strided\\\,\\\ tensor-id\\\=\\\(string\\\)classification-generic-softmaxed-out\\\,\\\ dims\\\=\\\(int\\\)\\\<\\\ \\\[\\\ 0\\\,\\\ 1\\\ \\\]\\\,\\\ \\\[\\\ 1\\\,\\\ 2147483647\\\ \\\]\\\ \\\>\\\,\\\ dims-order\\\=\\\(string\\\)row-major\\\,\\\ type\\\=\\\(string\\\)\\\{\\\ float32\\\,\\\ uint8\\\ \\\}\\\;\\\ tensor/strided\\\,\\\ tensor-id\\\=\\\(string\\\)classification-generic-softmaxed-out\\\,\\\ dims\\\=\\\(int\\\)\\\<\\\ \\\[\\\ 1\\\,\\\ 2147483647\\\ \\\]\\\ \\\>\\\,\\\ dims-order\\\=\\\(string\\\)row-major\\\,\\\ type\\\=\\\(string\\\)\\\{\\\ float32\\\,\\\ uint8\\\ \\\}\"\ \}\;"
Properties
class-confidence-threshold
“class-confidence-threshold” gfloat
Classes with a confidence level inferior to this threshold will be excluded
Flags : Read / Write
Default value : 0
labels-file
“labels-file” gchararray
Path to a file containing class label following the COCO format
Flags : Read / Write
Default value : NULL
The results of the search are