gstreamer Wiki - GStreamerSummitGranCanaria2009Notes

GStreamer Summit Gran Canaria 2009

Notes taken by Thomas, but I reordered things a bit.

Toward GStreamer 1.0

There is a GStreamer 0.11 wiki page: ZeroPointEleven
There is a simple TODO list in the docs/random/part-TODO.txt
Concerns about the March 2010 timeline proposal
Desire to do more of the experimental work in 0.10 - introduce base classes, . Explore the API there better.
"Release 1.0 with new core, removed deprecated API and invite applications to start porting while we start adding features again" is not a good plan.
Needs to be less ambitious than with the 0.9 dev cycle, as we don't have the same core of hackers that can work on it full time
must fix items
- remove the existing deprecated API
- GstImplementsInterface needs to die
- Must have a porting guide
nice to have
- Add base classes for Audio/Video decoders (can happen in 0.10 master)
API that needs improving:
- XOverlay -> GstVideoEmbedInterface
Caps negotiation - switch to a iterator model
Reverse negotiation via buffer alloc is flawed. Use an upstream notification mechanism from downstream that 'something has changed'
2 opinions: define a timeline for GNOME 3.0 and approach it vs. choose our own 'deadline' rather than arbitrarily pick GNOME 3.0 which is too soon

Notes from ThomasVS:

Remove deprecated APIs

ugly api
- ImplementsInterface
- tuner
- XOverlay (Tim), slight adjusting
- PropertyProbe (Tim), ugly
caps nego
- n2 algorithms (Wim)
- upstream caps change with bufferalloc: change without dropping the buffer
- Jan and Wim agree on an iterator approach instead of generating huge lists
part-todo.txt
- easy stuff, search and replace
buffers
- subclassing doesn't work well
  - used now for special allocation; to verify you created the buffer
    - netbuffer
    - subbuffering
    - for metadata (not yet used that way)
  - plan
    - some way of tagging buffers
    - using something like a structure but with less overhead
    - prototype using just a GType, not subclassable, just a structure with a refcount
    - see: http://cgit.freedesktop.org/gstreamer/gstreamer/tree/docs/design/draft-buffer2.txt (tpm)
- video buffers: caps and metadata
  - concept
    - yuv
    - stride
    - chroma
    - scale
    - gamma
    - subrectangle for visible region
    - buffer row-stride
    - clean-region
    - interlace-info
    - clean up colorspaces (for example for hi-def)
    - chroma subsampling
    - interleaved/non-interleaved audio
  - caps
    - things that don't change per buffer
    - things needed for negotiating
    - hints on what the metadata could contain
  - tags
    - things that change per buffer
newsegment
- makes pipelines harder to do dynamically
  - it only comes at the start it's state that needs to be kept on pads very similar to the caps on the buffer (Wim), but you would lose speed that way (Edward) descriptor of the stream, for timing information caching in the pad: great putting it on every buffer: not so great elements mostly store it for clipping or demuxing makes sense to keep as an event (for example for GDP) "given a buffer, how do I get the timing info for that buffer" (Wim) parity error between segment and accumulator
timestamps
- rounding errors for nanoseconds
- use a clock-base
- clock-base would be per-stream, not global
- clock distribution
  - make it atomic to set clock and base time
- GST_CLOCK_TIME_NONE
  - shoving a signed int into a uint creates problems
    - bindings
    - needing to use macros
- negative timestamps
  - Jan has a use case for DVD's sending older data after sending time info
error reporting
- Internal flow error coming from the source
  - no way to know where it came from except looking from the log
- caps negotiation and intersection failing
  - Stefan would already be happy with something in the logs saying why
    - this is for app developers trying to figure out nego
  - Jan: should be possible to try and negotiate, get a failure message, and try again with a different decoder/element
    - this is for the app
- Laszlo dumps images from pipeline in debug mode
- Wim says gst-launch does this when it hits an error
caps names and mime type
- elements could post mime types as tags
- clear up when a name is not a mime type ?
bindings
- what problems do we create for bindings ?
  - should all functions that can fail give a GError ?
  - special constructors ?
  - C-based stuff like writability based on refcounting ?
    - writability and refcounting should be separate somehow ?
  - MiniObjects ?
ORC
- Should we endorse it ?
- Dave has already made GStreamer branches ?
- our requirements ?
- we think it's a really good way to go
- with liboil you needed to ifdef after adding a function to liboil for you
- orc is more like a vector shader language
- liboil will not be a hard requirement for 1.0 anymore
- maybe include orc in the tree to help it stabilize and make us more active on it ? Doesn't seem necessary
registry stuff
- Jan: can be done in 0.10
- exec a binary and communicate over a socket
- fixes scanning on OSX
- fixes Windows scanning where you don't have fork
- you can blacklist plugins
- (Note: this is already done in 0.10.26 (tpm))
property probe
- Tim knows what he wants to do but doesn't have an API yet
- Not sure he wants to tie it to elements
- ideally it will be hotpluggable
- maybe a centralized device manager with notifications
parse library
- pull it out of core
unify microlibraries
GstBus
- async messages
  - with futures mechanism
- when the metadata is posted needs changing
  - to when the data is being rendered
Infrastructure
- our own server ?
  - storage for media
  - buildmaster
  - custom web app
  - central chroot for making tarballs
API experiments (Wim)
- pad_alloc
- get_range
new web page
- Stefan wants to get feedback and deploy
caps
- stepped ranges (like multiple of 8)
- RGB caps should be more like YUV (get rid of the masks)
black metadata
- things that don't fit in the current tags
tags on pads
- tags should be sticky on pads too, like newsegment (Olivier)
preset
- discussion with Christian

TODO

wiki with tips
- stefan's tracer log
- cymacs log analyzer

Mine [Thomas]

log refcounting
stream time deprecated ?

Easier installations for other platforms

Windows and OS/X installs
One of the corporate partners could just do it if they think it's important
We should invite people to join the release process
It's hard to keep people interested in making Windows builds in the long term 'for the fun of it', so we end up with multiple out-of-date installer bundles from different people
Build slaves might help, to keep ensuring that everything continues to build on interesting platforms
wix might be a good option for getting to msi files for Windows
LRN uses MingW cross-compilation
Windows builds need to be handled by someone who really cares
Moovida uses Git GStreamer and their own build process
Songbird build and ship their own GStreamer as well. They use patched autotools/libtools + MS VC compiler. MikeS could comment more and provide good feedback
Tandberg / Ole Andre

Supporting KDE/Qt better

Need examples of GMainloop/QtMainloop examples
Widget examples
applies to OS/X and Windows too - provide more platform example code
KDE people say GStreamer backend for Phonon sucks
We don't know who is working on it
Video support in Phonon unimplemented
Need to get the word out and invite people into #gstreamer
There are several GStreamer phonon backends, and it's not clear what the 'current' one is.
Need a way to get in touch with the relevant developers
GStreamer is the default backend because it's the only one they can easily ship, but it doesn't work properly
Stefan to try and find some of the developers to get us in touch
kdenlive are using MLT
Edward had some discussions with Amarok developers
- didn't know there were GStreamer C++ bindings.
- A simple helper API that hides all the complex GStreamer API
Tim doesn't think that C++ bindings are convincing, at least not to Qt people, as they're not Qt APIs
There are some Qt bindings for GStreamer in KDE svn somewhere
Applying resources to GStreamer 1.0 will prevent a lot of effort on this from us
Poor communication - KDE devs that don't know GStreamer runs on Windows for e.g.
Oliviere: Last night Andrunko said he had once tried to build some QObject style APIs via GObject introspection, which may be a least-effort way to go.