llamacpp-imagetransform
llama.cpp-based element that passes an image through a vision LLM and forwards the generated text.
It is possible to configure a system prompt, various sampling parameters and to keep a history of the last inputs/outputs for producing more consistent outputs.
The element can be used for example for image descriptions, which can then be used for text to speech, or for understanding the content of a video frame by frame.
Models
The model and the multimodal projection model can be selected via the model-path and
mmproj-path properties. These expect local GGUF files, which can be downloaded e.g. from
Hugging Face. The model-path should point to the GGUF file of a
vision LLM and mmproj-path to the GGUF file of the corresponding multimodal projection model.
Models that are known to work well are for example
Keep in mind that all these models have safeguards integrated, which can lead to rejections.
Examples
Image description
gst-launch-1.0 filesrc location=photo.jpg ! image/jpeg ! llamacpp-imagetransform model-path=/path/to/Qwen3.5-9B.gguf mmproj-path=/path/to/mmproj-Qwen3.5-9B.gguf system-prompt="Describe the image in one sentence." ! fakesink
Hierarchy
GObject ╰──GInitiallyUnowned ╰──GstObject ╰──GstElement ╰──GstLlamaBaseTransform ╰──llamacpp-imagetransform
Factory details
Authors: – Sebastian Dröge
Classification: – Text/LLM/Video
Rank – none
Plugin – llamacpp
Package – gst-plugin-llamacpp
Pad Templates
sink
image/jpeg:
image/png:
video/x-raw:
format: RGB
width: [ 1, 2147483647 ]
height: [ 1, 2147483647 ]
Properties
mmproj-path
“mmproj-path” gchararray
Path to the GGUF multimodal projection model file
Flags : Read / Write
Default value : NULL
The results of the search are