Planet GStreamer

GStreamer: GStreamer Conference 2013: Registration now open, Call for Papers

2013-05-21T23:00:00+00:00

About the GStreamer Conference

The GStreamer Conference 2013 will take place from 22-23 October 2013 in Edinburgh (UK), and will be co-hosted with the Embedded Linux Conference Europe and the Automotive Linux Summit.

It is a conference for developers, decision-makers, and anyone else interested in the GStreamer multimedia framework and open source multimedia.

Registration now open

You can now register for the GStreamer Conference 2013 on the conference website.

Call for Papers

The initial submission needs to be only a couple of paragraphs describing the talk you want to give and the desired length of your talk (please allow for 5-10 minutes of questions at the end as well).

Talks can be on almost anything multimedia related, ranging from talks about applications to challenges in the lower levels in the stack or hardware.

Please send all proposals to gstreamer-conference@lists.freedesktop.org.

Thomas Vander Stichele: Organizing photo libraries

2013-05-18T11:51:02+00:00

The weather’s picking up so it’s time for spring cleaning around the house. When I moved back to Barcelona three years ago I took with me my old analogue photos and negatives, with the idea of sorting through them at some point and getting them digitized. And while I’m at it, maybe it’s time to pull all my various folders of photos together too and organize them.

Well, I finally started. I grouped the negatives, labeled them by year, put them in individual envelopes, and handed them off to a professional lab to scan them after doing a quick test run on one set (which turned out great, but it’s *really* annoying me that they scan to JPEG by default, charge 40% extra for TIFF, and use a non-multiple-of-8 resolution to scan at which means I can’t losslessly rotate the negatives. Yes, I’m anal.)

So now I pulled together all my various folders of photos, and before I start doing tagging and stuff like that, I want to organize them in a decent folder layout. Googling for ideas pretty much suggests that the way to go is

YYYY/MM/DD

with possibly some description together with the DD

I’m not really happy about that, however, because there are certain things I’d like to be able to do:

easily see where photos come from – did I make them ? did I get them from someone ? Did I download them from Facebook ?
Are these original files from a camera without editing ?
Are these the original scans ? From negatives ? From actual photos ? Or are they retouched, rotated, denoised, …
Are these photos SFW ? Can I point my media center slideshow to this directory and have it safely show any photos under it ? (What do you mean, you’ve never snowboarded at night in only your underwear, and mooning the photographer ?) Or maybe not even SFW, but simply watchable and reasonable quality or subject material?

I realize some of these issues can not be resolved simply with a directory layout. But I’m sure some of you must have had similar issues or come up with a slightly better layout ?

Point me in the right direction please.

Stefan Kost: 15 May 2013

2013-05-15T12:10:04+00:00

buzztard is now buzztrax

We applied as an organisation to take part in the Google Summer of Code program, but got rejected mainly due to the project name. As this was not the first time people where uncomfortable with the name, we renamed the project - buzztard is now called buzztrax. The homepage with the wiki and wordpress already got moved. The renamed codebase is online at github. The mailing lists have been migrated to buzztrax.org. A few things still need to be fixed (e.g. file releases).

Besides the renaming there are also some improvements on the code side. I am probably the last one to discover the g_signal_handlers_disconnect_by_* macros. Using these made the code a bit leaner. I also worked on the level-meter features. I did some cleanups on the widget. The syncing code is more efficient as we listen to sync-messages on the gstreamer side to avoid another thread round trip. Song rendering can disabled the level-meters for less noise on the screen and some performance savings. The song-rendering now uses the TOC support in gstreamer-1.0. That means that the labels of a song (intro, chorus, break, ...) will end up in wav and flac files right now. When other formats support toc, this will automatically work for those formats too. The flag in ogg muxing got fixed in upstream and now works for us again.

56 files changed, 715 insertions(+), 631 deletions(-)

Thomas Vander Stichele: morituri and Hidden Track One Audio

2013-05-10T20:08:31+00:00

I have tomorrow (saturday) blocked out for a whole day of morituri hacking as I will be home alone.

One of the things a lot of morituri users are puzzled by is its relentless drive to extract every single sample of audio from the CD. Currently, even if it’s a really short pre-gap, and most likely just an inaccurate master or burn, with no useful audio in it.

For me, that was a design goal of morituri – I want to be able to exactly reproduce a CD as is. That is to say, ripping a CD should extract *all* audio from the CD, and it should be possible to make a copy of that CD and then rip that copy, and end up with exactly the same result as from the original CD. (I’m sure there’s a fancy scientific term for that that I can’t remember right now)

To a lot of other people, it seems to be annoying and they don’t like having those small almost empty files lying around.

So I thought I’d do something about that, and that it might be useful as well to analyze my current collection of tracks and figure out what’s in there. Maybe I can find some hidden gems that I hadn’t noticed before?

So I added a quick task to morituri that calculates the maximum sample value (I didn’t want to use my own level element in GStreamer for this as I wanted to make sure it was actual digital zero; this should be done in an element instead though, but I preferred the five minute hack for this one).

And then I ran:

rip debug maxsample /mnt/nas/media/audio/rip/morituri/own/album/*/00*flac

Sadly, that turned up 0 as the biggest sample for all these tracks!

Wait, what? I spent all that time on getting those secret tracks ripped just to get none? That’s not possible! I know some of those tracks!

Maybe the algorithm is wrong. Nope, it works fine on all the regular tracks.

Oh, crap. Maybe morituri has been ripping silence all this time because my CD drive can’t get that data off. Yikes, that would be a bit of egg on my face.

No, it works if I check that Bloc Party track I know about.

Ten minutes of staring at the screen to realize that, while I was outputting names from a variable from the for loop over my arguments, the track I was actually passing to the task was always the first one. Duh. Problem solved.

As for what I found in my collection:

a cute radio jingle that brought back memories from a live bootleg I had made myself of Bloem. That’s from over ten years ago, but that must have been around the time I learned about the existence of HTOA and wanted to get one in
found unknown HTOA tracks on Art Brut’s Bang Bang Rock & Roll, Mew’s Half the world is watching me; not their best stuff
soundscapey or stagesetting tracks on QOTSA’s Songs for the Deaf, Motorpsycho’s Angels and Daemons at play And Blissard; not that worth it (the Blissard track was ok, but really quiet)
Pulp hid a single piano chord in a 2 second pre-gap on This is Hardcore; very curious. It’s not an intro to the first track, because it doesn’t fit with the sound at all.
Damien Rice hid a demo version of 9 Crimes (the first track) in the pregap; instead of piano and female vocals, he plays guitar and sings all the parts.
Got reacquainted with my favourite HTOA tracks: the orchestral quasi-wordless medley on the Luke Haines/Das Capital disc; the first Bloc Party album with a beautiful instrumental (up there with the hidden track at the end of Placebo’s first album; both bands delivering an atypical but stunning moodscape; the beautiful cover of Ben Kenobi’s Theme by Arab Strap on the Cherubs EP (no idea why that landed in my album dir, that needs to be fixed); the silly Soulwax skit for their second album.

Of course, Wikipedia has the last word on everything

I note that they think Pulp recorded a cymbal, not a piano. And now that I see the title of the QOTSA hidden track, I get the joke I think.

In total, on my album collection of 1564 full CD’s, I have 171 HTOA’s ripped, 138 tracks of pure digital silence, and only about 11 are actually useful tracks.

I expected to find more gems in my collection. I’ll go through ep’s, singles and compilations next just to be sure.

But with this code in hand, maybe it’s time to add something to morituri to save the silent HTOA tracks as pure .cue information.

Andy Wingo: generators in v8

Andy Wingo — 2013-05-08T19:31:42+00:00

Hey y'all, ES6 generators have landed in V8! Excellent!

Many of you know what that means already, but for those of you that don't, a little story.

A few months ago I was talking with Andrew Paprocki over at Bloomberg. They use JavaScript in all kinds of ways over there, and as is usually the case in JS, it often involves communicating with remote machines. This happens on the server side and on the client side. JavaScript has some great implementations, but as a language it doesn't make asynchronous communication particularly easy. Sure, you can do anything you want with node.js, but it's pretty easy to get stuck in callback hell waiting for data from the other side.

Of course if you do a lot of JS you learn ways to deal with it. The eponymous Callback Hell site lists some, and lately many people think of Promises as the answer. But it would be nice if sometimes you could write a function and have it just suspend in the middle, wait for your request to the remote computer to come through, then continue on.

So Andrew asked if me if we could somehow make asynchronous programming easier to write and read, maybe by implementing something like C#'s await operator in the V8 JavaScript engine. I told him that the way into V8 was through standards, but that fortunately the upcoming ECMAScript 6 standard was likely to include an equivalent to await: generators.

ES6 generators

Instead of returning a value, when you first call a generator, it returns an iterator:

// notice: function* instead of function
function* values() {
  for (var i = 0; i < arguments.length; i++) {
    yield arguments[i];
  }
}

var o = values(1, 2, 3);  // => [object Generator]

Calling next on the iterator resumes the generator, lets it run until the next yield or return, and then suspends it again, resulting in a value:

o.next(); // => { value: 1, done: false }
o.next(); // => { value: 2, done: false }
o.next(); // => { value: 3, done: false }
o.next(); // => { value: undefined, done: true }

Maybe you're the kind of person that likes imprecise, incomplete, overly abstract analogies. Yes? Well generators are like functions, with their bodies taken to the first derivative. Calling next integrates between two yield points. Chew on that truthy nugget!

asynchrony

Anyway! Supending execution, waiting for something to happen, then picking up where you left off: put these together and you have a nice facility for asynchronous programming. And happily, it works really well with promises, a tool for asynchrony that lots of JS programmers are using these days.

Q is a popular promises library for JS. There are some 250+ packages that depend on it in NPM, node's package manager. So cool, let's take their example of the "Pyramid of Doom" from the github page:

step1(function (value1) {
    step2(value1, function(value2) {
        step3(value2, function(value3) {
            step4(value3, function(value4) {
                // Do something with value4
            });
        });
    });
});

The promises solution does at least fix the pyramid problem:

Q.fcall(step1)
.then(step2)
.then(step3)
.then(step4)
.then(function (value4) {
    // Do something with value4
}, function (error) {
    // Handle any error from step1 through step4
})
.done();

But to my ignorant eye, some kind of solution involving a straight-line function would be even better. Remember what generators do: they suspend computation, wait for someone to pass back in a result, and then continue on. So whenever you would register a callback, whenever you would chain a then onto your promise, instead you suspend computation by yielding.

Q.async(function* () {
  try {
    var value1 = yield step1();
    var value2 = yield step2(value1);
    var value3 = yield step3(value2);
    var value4 = yield step4(value3);
    // Do something with value4
  } catch (e) {
    // Handle any error from step1 through step4
  }
});

And for a super-mega-bonus, we actually get to use try and catch to handle exceptions, just as Gods and Brendan intended.

Now I know you're a keen reader and there are two things that you probably noticed here. One is, where are the promises, and where are the callbacks? Who's making these promises anyway? Well you probably saw already in the second example that using promises well means that you have functions that return promises instead of functions that take callbacks. The form of functions like step1 and such are different when you use promises. So in a way the comparison between the pyramid and the promise isn't quite fair because the functions aren't quite the same. But we're all reasonable people so we'll deal with it.

Note that it's not actually necessary that any of the stepN functions return promises. The promises library will lift a value to a promise if needed.

The second and bigger question would be, how does the generator resume? Of course you've already seen the answer: the whole generator function is decorated by Q.async, which takes care of resuming the generator when the yielded promises are fulfilled.

You don't have to use generators of course, and using them with Q does mean you have to understand more things: not only standard JavaScript, but newfangled features, and promises to boot. Well, if it's not your thing, that's cool. But it seems to me that the widespread appreciation of await in C# bodes well for generators in JS.

ES6, unevenly distributed

Q.async has been part of Q for a while now, because Firefox has been shipping generators for some time.

Note however that the current ES6 draft specification for generators is slightly different from what Firefox ships: calling next on the iterator returns an object with value and done properties, whereas the SpiderMonkey JS engine used by Firefox uses an exception to indicate that the iterator is finished.

This change was the result of some discussions at the TC39 meeting in March, and hasn't made it into a draft specification or to the harmony:generators page yet. All could change, but there seems to be a consensus.

I made a patch to Q to allow it to work both with the old SpiderMonkey-style generators as well as with the new ES6 style, and something like it should go in soon.

give it to me already!

So yeah, generators in V8! I've been working closely with V8 hackers Michael Starzinger and Andreas Rossberg on the design and implementation of generators in V8, and I'm happy to say that it's all upstream now, modulo yield* which should go in soon. Along with other future ES6 features, it's all behind a runtime flag. Pass --harmony or --harmony-generators to your V8 to enable it.

Barring unforeseen issues, this will probably see the light in Chromium 29, corresponding to V8 3.19. For now though this hack is so hot out of the fire that it hasn't even had time to cool down and make it to the Chrome Canary channel yet. Perhaps within a few weeks; whenever the V8 dependency in Chrome gets updated to the 3.19 tree.

As far as Node goes, they usually track the latest stable V8 release, and so for them it will probably also be a few weeks before it goes in. You'll still have to run Node with the --harmony flag. However if you want a sneak preview of the future, I uploaded a branch of Node with V8 3.19 that you can build from source. It might mulch your cat, but life is not without risk on the bleeding_edge.

Finally for Q, as I said ES6 compatibility should come soon; track the progress or check out your own copy here.

final notes

Thanks to the V8 team for their support in this endeavor, especially to Michael Starzinger for enduring my constant questions. There's lots of interesting things in the V8 pipeline coming out of Munich. Thanks also to Andrew Paprocki and the Bloomberg crew for giving me the opportunity to fill out this missing piece of real-world ES6. The Bloomberg folks really get the web.

This has been a hack from Igalia, your friendly neighborhood browser experts. Have fun with generators, and happy hacking!

Thomas Vander Stichele: Votes for talks at open source conferences

2013-05-07T11:53:26+00:00

I’ve never been a fan of voting for talks, because it tends to be poorly implemented under the guise of democracy. Of course it’s easy for me to talk, I’ve never organized anything at that scale.

I’ll give two examples on why I feel this way, one of which triggering today’s blog post.

First off, my colleague Marek submitted a talk to Djangocon. The talk was about how to use feat (a toolkit we wrote for livetranscoding) to serve Django pages, but in such a way that they can use Deferreds to remove the concurrency bottleneck of “1 request at a time” per process running Django.

Personally, to me, this is one of the most irritating design choices of Django – from the ground up it was built synchronously (which could have been fine in most places). But the fact that, when you get a request, you have to always synchronously respond to it (and block every other request for that process in the meantime) is a design choice that could have easily been avoided.

In our particular use case, it was really painful. If our website has to do an API request to some other service we don’t control that can easily take 30 seconds, our process throughput suddenly becomes 2 pages per minute. All the while, the server is sitting there waiting.

Yes, you can throw RAM at the problem and start 30 times more processes; or thread out API requests; or farm it out to Celery, and do some back-and-forthing to see when the call’s done. Or do any other number of workarounds for a fundamental design choice.

Since we like Twisted, we preferred to throw Twisted at the problem, and ended up with something that worked.

Anyway, that’s a lot of setup to explain what the talk was about. Marek submitted the talk to DjangoCon, and honestly I didn’t expect it to get much traction because, when you’re inside Django, you think like Django, and you don’t really realize that this is a real problem. Most people who do realize it switch away to something else.

But to my surprise, Marek’s talk was the most-voted talk! I wish I could link to the results, but of course that vote site is no longer online.

I guess I expected that would mean he’d be presenting at DjangoCon this year. So I asked him today when his talk was, and he said “Oh that’s right. I did not get accepted.”

Well, that was a surprise. Of course, the organising committee reserves the right to decide on their own – maybe they just didn’t like the talk. But if you ask your potential visitors to vote, you’d expect the most-voted talk to make it on the schedule no ?

The feedback Marek got from them was surprising too, though. Their first response was that this talk was too similar to another talk, titled “How to combine JavaScript & Django in a smart way”. Now, I’m not a JavaScript expert, but from the title alone I can already tell that it’s very unlikely that these two talks have many similarities beyond the word ‘Django’.

After refuting that point, their second reason was that they wanted more experienced speakers (but they didn’t ask Marek for his experience), and their third reason was that the talk was in previous editions of DjangoCon US/EU (it’s unclear whether they meant his talk or the JavaScript one, but Marek’s definitely wasn’t, and we couldn’t find any mention of the other talk in previous conferences. I’m also not sure why that even matters one way or the other. This email thread was in Polish, so I have to rely on Marek’s interpretation of it)

Personally, my reaction would have been to complain to the organizers or Django maintainers. Marek’s flegmatic attitude was much better though – after such an exchange, he simply doesn’t want to have anything to do with the conference.

He’s probably right – it’s hard to argue with someone who doesn’t want to invite you and is lying about the reasons.

The second example is BCNDevCon, a great conference here in Barcelona, organized by a guy who used to work for Flumotion who I have enormous respect for. I’ve never seen anyone create such a big conference over so little time.

He believes strongly in the democratic aspect, and as far as I can tell constructs the schedule solely based on the votes.

Sadly I didn’t go to the last one, and the reason is simply because I felt that the talks that made it were too obviously corporate. A lot of talks were about Microsoft products, and you could tell that they won votes because people’s coworkers voted on talks. I’m not saying that’s necessarily wrong – given that he worked at our company and has friends here, I’m sure people working here presenting at his conference have also done vote tending. It’s natural to do so. But there should be a way to balance that out.

I think the idea of voting is good, but implementation matters too. Ideally, you would only want people that actually are going to show up to vote. I have no idea how you can ensure that, though. Do you ask people to pre-pay ? Do you ask them to commit to pay if at least 50% of their votes make it in the final schedule, kickstarter-style ?

These two examples are on opposite extremes of voting. One conference simply disregards completely what people vote on. If I had voted or bought a ticket, I would feel lied to. Why waste the time of so many people? The other conference puts so much stock in the vote, that I feel the final result was strongly affected. I seriously doubt all those Windows 8 voters actually showed up.

Does anyone have good experiences with conference voting that did work? Feel free to share!

Thomas Vander Stichele: If I was 16 years younger…

2013-05-03T21:30:25+00:00

I’d totally try and be the intern for pinboard.

The money is great for a summer job, but that’s not the important part. pinboard seems interesting, it’s a real service, and it’s (I assume) small enough to understand from top to bottom. Contrary to, say, a Google Summer of Code project, you get to touch a real existing service, and from what I can tell from the blog you get to do it with a smart and funny guy.

You’ve got five weeks left; even if you’re in the middle of exams right now, apply!

(And if you do, why not add the features to merge and rename tags while you’re at it?)

Jean-François Fortin Tam: No more stuck rendering dialogs!

2013-04-28T04:19:25+00:00

If you’ve tried rendering projects with Pitivi 0.15 or older, chances are you’ve encountered one of these dreadful situations where the rendering process would get stuck:

…at the beginning, with the progressbar saying it’s currently “estimating” — which was a lie that I corrected a little while ago.
…at the very end. Extra trolling points for having made you waste a huge amount of time to get a 0 bytes output file (if we’re lucky, that bug is gone).
…somewhere in the middle, because caps negotiation failed, some elements were not linked, GStreamer thinks you ran out of available RAM, or because you’ve been very naughty.

In any such case, the rendering dialog just sat there and smiled at you, as if everything was fine in the world. Well, no more:

Pitivi is going to give you the honest, brutal truth.

This is the result of a horrifying thought suddenly springing to my mind yesterday night: “Hey, what if the code was not even checking for errors in the pipeline when rendering?”

Indeed, it wasn’t. How silly is that! I have thus prepared a simple fix to improve the situation: catch pipeline error messages, abort the render (you really don’t want to ignore a GStreamer error) and display an error dialog. This will at least let people know that something is wrong and that they should start writing patches to GStreamer instead of accusing Pitivi of hurting kittens. You’d be surprised how many people can sit for hours in front of that stuck progressbar.

Before I commit the fix however, I would need your feedback on the usability of that dialog:

This is not terribly pretty, but it’s better than nothing. A few things to consider:

In that screenshot, all the text except the window title (“Error While Rendering Project”) comes from the GStreamer pipeline error message (the error and the error’s details). I know that the error details look ugly, but I suspect it wouldn’t be useful to GStreamer/Pitivi developers if we don’t have them “verbatim”. Maybe we could try to mangle the error details string (split using “:” and take only the first and last two items of the resulting list?) and encourage the user to run from a terminal to get better debug info, but that feels a bit backwards.
We should probably have some less-scary text to accompany the actual error details. Something that guides the user towards an action that can be done to address the problem (ex: reporting a bug). Maybe it can be placed between the header and the details (above the “qtdemux.c” line)? The problem is finding a universal text to be used.
If we consider the route where we suggest the user to report bugs, where should we point to? The Pitivi bugs investigation page? Pitivi bugzilla? GStreamer bugzilla? The distro’s bug tracker?
Let’s keep this simple, both visually and in terms of code/implementation.

What do you think? Is the current approach sufficient or is there something better that we can easily do?

Update: here’s an alternative dialog with some more comprehensible text, where the actual error (as seen in the previous screenshot) gets shoved under the rug by putting it in a GTK expander widget (clicking “Details” reveals the error’s details as above):

GStreamer: GStreamer Core and Plugins 1.0.7 bug-fix release

2013-04-26T14:00:00+00:00

The GStreamer team is pleased to announce another bug-fix release for the new API and ABI-stable 1.x series of the GStreamer multimedia framework.

Check out the release notes for GStreamer core, gst-plugins-base, gst-plugins-good, gst-plugins-ugly, gst-plugins-bad, or gst-libav, or download tarballs for gstreamer, gst-plugins-base, gst-plugins-good, gst-plugins-ugly, gst-plugins-bad, or gst-libav,

Improvements include:

better support for static plugins
streamsynchronizer is now a public element, useful in HLS pipelines for example
osxaudio plugin ported to 1.0
androidmedia plugin is ported to 1.0
internal libav snapshot in gst-libav has been updated to libav 0.8.6
Now uses frame based threading if possible and disables slice based threading until libav 0.9.x is used (as is the case in git master already) This provides potentially better performance and fixes some display corruptions caused by bugs in the slice based threading
a number of bug fixes

Christian Schaller: Brno GUADEC Call for Papers!

2013-04-19T10:29:01+00:00

I would like to give everyone a friendly reminder that Saturday the 27th of April is the official deadline for the GUADEC 2013 Call for Papers. So make sure to get your proposal submitted.

We hope to have a wide range of talks this year, including talks on related subjects such as Wayland and Multimedia, so don’t automatically assume that you will not get a talk approved because its not ‘pure GNOME’.

GUADEC this year will be in Brno in the Czech Republic, so I hope to see as many of you as possible here.

Michael Sheldon: Erudite now available for Symbian Belle

2013-04-19T09:51:34+00:00

Overview

Erudite makes it possible to use Amazon’s Cloud Reader on various mobile platforms which aren’t officially supported by Amazon. You can either read books online, or download them for reading offline. Your progress in a book is then also kept synchronised between your phone and other Kindle devices.

Symbian Belle support

Until recently Erudite only supported MeeGo Harmattan and Mer based phones (such as the Nokia N9), but now it’s also available for Symbian Belle phones as well. I’ll try and put together a build for Symbian Anna phones in the near future.

There’s a fairly comprehensive review of Erudite for Symbian over on All About Symbian: Erudite review.

Upcoming features

In the next release for both Symbian and MeeGo I’ll be focusing on orientation switching support, so users can optionally view their books in landscape mode, as well as investigating some apparent issues with very large books.

Download

Symbian Belle version — Nokia 701, Nokia C6-01, Nokia C7-00, Nokia N8-00, Nokia X7-00, Nokia E6-00, Nokia 808 Pureview, Nokia 603, Nokia Oro, Nokia E7-00, and Nokia 700
MeeGo Harmattan version — Nokia N9, Nokia N950

Andy Wingo: inside full-codegen, v8's baseline compiler

Andy Wingo — 2013-04-18T15:45:20+00:00

Greetings to all. This is another nargish article on the internals of the V8 JavaScript engine. If that's your thing, read on. Otherwise, here's an interesting interview with David Harvey. See you laters!

full-codegen

Today's topic is V8's baseline compiler, called "full-codegen" in the source. To recall, V8 has two compilers: one that does a quick-and-dirty job (full-codegen), and one that focuses on hot spots (Crankshaft).

When you work with V8, most of the attention goes to Crankshaft, and rightly so: Crankshaft is how V8 makes code go fast. But everything goes through full-codegen first, so it's still useful to know how full-codegen works, and necessary if you go to hack on V8 itself.

The goal of this article is to recall the full-codegen to mind, so as to refresh our internal "efficiency model" of how V8 runs your code (cf. Norvig's PAIP retrospective, #20), and to place it in a context of other engines and possible models.

stack machine

All JavaScript code executed by V8 goes through full-codegen, so full-codegen has to be very fast. This means that it doesn't have time to do very much optimization. Even if it did have time to do optimization (which it doesn't), full-codegen lacks the means to do so. Since the code hasn't run yet when full-codegen compiles it, it doesn't know anything about types or shapes of objects. And as in other JS engines, full-codegen only compiles a function at a time, so it can't see very far. (This lets it compile functions lazily, the first time they are executed. Overall syntactic validity is ensured by a quick "pre-parse" over the source.)

I have to admit my surprise however at seeing again how simple full-codegen actually is: it's a stack machine! You don't get much simpler than that. It's shocking in some ways that this is the technology that kicked off the JS performance wars, back in 2008. The article I wrote a couple years ago does mention this, but since then I have spent a lot of time in JSC and forgot how simple things could be.

Full-codegen does have a couple of standard improvements over the basic stack machine model, as I mentioned in the "two compilers" article. One is that the compiler threads a "context" through the compilation process, so that if (say) x++ is processed in "effect" context, it doesn't need to push the initial value of x on the stack as the result value. The other is that if the top-of-stack (ToS) value is cached in a register, so that even if we did need the result of x++, we could get it with a simple mov instruction. Besides these "effect" and "accumulator" (ToS) contexts, there are also contexts for pushing values on the stack, and for processing a value for "control" (as the test of a conditional branch).

and that's it!

That's the whole thing!

Still here?

What about type recording and all that fancy stuff, you say? Well, most of the fancy stuff is in crankshaft. Type recording happens in the inline caches that aren't really part of full-codegen proper.

Full-codegen does keep some counters on loops and function calls and such that it uses to determine when to tier up to Crankshaft, but that's not very interesting. (It is interesting to note that V8 used to determine what to optimizing using a statistical profiler, and that is no longer the case. It seems statistical profiling would be OK in the long run, but as far as I understand it didn't do a great job with really short-running code, and made it difficult to reproduce bugs.)

I think -- and here I'm just giving my impression, but fortunately I have readers that will call my bullshit -- I think that if you broke down basic JavaScript and ignored optimizable hot spots, an engine mostly does variable accesses, property accesses, and allocations. Those are the fundamental things to get right. In a browser you also have DOM operations, which Blink will speed up at some point; and there are some domain-specific things like string operations and regular expressions of course. But ignoring that, there are just variables, properties, and allocations.

We'll start with the last bit. Allocation is a function of your choice of value representation, your use of "hidden classes" ("maps" in V8; all engines do the same thing now), and otherwise it's more a property of the runtime than of the compiler. (Crankshaft can lower allocation cost by using unboxed values and inlining allocations, but that's Crankshaft, not full-codegen.)

Property access is mostly handled by inline caches, which also handle method dispatch and relate to hidden classes.

The only thing left for full-codegen to do is to handle variable accesses: to determine where to store local variables, and how to get at them. In this case, again there's not much magic: either variables are really local and don't need to be looked up by name at runtime, in which case they can be allocated on the stack, or they persist for longer or in more complicated scopes, in which case they go on the heap. Of course if a variable is never referenced, it doesn't need to be allocated at all.

So in summary, full-codegen is quick, and it's dirty. It does its job with the minimum possible effort and gets out of the way, giving Crankshaft a chance to focus on the hot spots.

comparison

How does this architecture compare with what other JS engines are doing, you ask?

I have very little experience with SpiderMonkey, but as far as I can tell, with every release they they get closer and closer to V8's model. Earlier this month Kannan Vijayan wrote about SpiderMonkey's new baseline compiler, which looks to be exactly comparable to full-codegen. It's a stack machine that emits native code. It looks a little different because it operates on bytecode and not the AST, as SpiderMonkey still has an interpreter hanging around.

JavaScriptCore, the WebKit JavaScript engine, is similar on the surface: it also has a baseline compiler and an optimizing compiler. (In fact it's getting another optimizing compiler soon, but that's a story for another article). But if you look deeper, I think JSC has diverged from V8's model in interesting ways.

Besides being a register machine, a model that has inherent advantages over a stack machine, JavaScriptCore also has a low-level interpreter that uses the same calling convention as the optimizing compiler. I don't think JSC is moving in V8's direction in this regard; in fact, they could remove their baseline compiler entirely, relying instead on the interpreter and the optimizing compiler. JSC also has some neat tricks related to scoping (lazy tear-off); again, a topic for some future article.

summary

The field is still a bit open as to what is the best approach to use for cold code. It seems that an interpreter could still be a win, because if not, why would SpiderMonkey keep one around, now that they have a baseline compiler? And why would JSC add a new one?

At the same time, it seems to me that one could do more compile-time analysis at little extra cost, to speed up full-codegen by allocating more temporaries into slots; you would push and pop at compile-time to avoid doing it at run-time. It's unclear whether it would be worth it, though, as that analysis would have to run on all code instead of just the hot spots.

Hopefully someone will embark on the experiment; it should just be a simple matter of programming :) Until next time, happy hacking!

Jean-François Fortin Tam: PiTiVi and the 2013 Summer of Code

2013-04-11T19:09:26+00:00

This year will be a little bit different. In a rather unexpected turn of events, PiTiVi has been accepted as a mentoring organization but GStreamer has not. Fear not however, as GStreamer has no better ally than the PiTiVi team when it comes to pushing our favorite multimedia framework to its limits and beyond. As you may know, PiTiVi makes heavy use of the GStreamer Editing Services library and, in turn, GNonLin and the rest of GStreamer. With the switch to GES and the irrevocable shedding of our old skin, any backend work done for the sake of the PiTiVi project ends up benefitting GStreamer and other projects.

One way to look at things is that there is no such thing as a PiTiVi backend anymore. PiTiVi is a frontend that pushes the latest and greatest open-source multimedia technologies forward.

With the GES port nearing completion, this is the first time that we can truly say there are three interrelated components to contribute to. This new reality sets the tone for a different way to look at PiTiVi project ideas this year: you can finally…

Choose your character class

Are you a ninja? A spellcaster? A tank? While most projects are a balance of backend and UI work, we know that some people prefer to lean more to one side or another of the continuum — that’s why I created a new visual notation for our ideas page this year. Instead of an “easy/hard” system (which would be inaccurate and misleading, as perceived difficulty is measured differently for everybody), we simply provided a visual indication of the expected involvement in the various components for a given project idea (for example, “PiTiVi: ◼◼◻◻◻ GES: ◼◼◼◼◼ GStreamer: ◼◼◼◻◻”). So if you were looking for something closer to a hardcore GStreamer GSoC project, you can spot ideas that might interest you here.

Not a programmer? You can help raise awareness about this. Maybe you know a brilliant hacker friend/relative or a top-notch computer science student waiting for a chance to make a big difference in the world. Tell that person about how cool and welcoming PiTiVi is and how getting involved is the best way to advance free, powerful and intuitive video editing for everyone!

Arun Raghavan: PulseAudio in GSoC 2013

2013-04-11T11:34:48+00:00

That’s right — PulseAudio will be participating in the Google Summer of Code again this year! We had a great set of students and projects last year, and you’ve already seen some their work in the last release.

There are some more details on how to get involved on the mailing list. We’re looking forward to having another set of smart and enthusiastic new contributors this year!

p.s.: Mentors and students from organisations (GStreamer and BlueZ, for example), do feel free to get in touch with us if you have ideas for projects related to PulseAudio that overlap with those other projects.

Stefan Kost: 9 Apr 2013

2013-04-09T20:31:51+00:00

buzztard & gstreamer hackfest

This month I mostly cleaned up small bits and pieces from the gstreamer-1.0 port. Most notably multitrack encoding works again. The handling of EOS and starting of the next track was racy. Speaking of the recording dialog - this one now has some basics for a silent mode implemented. For now it only disables the scrolling in the sequence view. Next thing would be to disable the level meters.

I hacked a bit more on the child-proxy iface. This now turned into utility functions that allows to write:
bt_child_proxy_set(obj, "prop1::prop2::prop3", val, NULL);
So what does this do?

GObject obj1, obj2;

  g_object_get(obj, "prop1", &obj1, NULL);

  g_object_get(obj1, "prop2", &obj2, NULL);

  g_object_set(obj2, "prop3", val, NULL);

  g_object_unref (obj2);

  g_object_unref (obj1);

This saved quite a few lines of C in buzztard. I wonder if this is something we want to add to glib? If we do I would go for a single ':' as a separator and we might also want to consider starting the property with one:
g_object_set(obj, ":prop1:prop2:prop3", val, NULL);
The leading ':' would help to quickly detect whether we need to split the property name and recurse into child objects. The whole scheme is backwards compatible as property names are not allowed to contain ':'.

In the end of march I attended the GStreamer hackfest in Milano. First I looked into a few tests - both on GStreamer and buzztard side. On the GStreamer side adder has some test fixes. On the buzztard side I improved my encoding tests. Wim gave me the crucial tip that fixed the dynamic adding/removing of analyzers while playing. Maybe I can try that for machines again. I showed the parser for controller-setups to some people and did smaller changes on it. I also discussed what we could do with gst-tracelib for gst-1.0 and started a new design-draft for it.

41 files changed, 871 insertions(+), 816 deletions(-)

Christian Schaller: No GStreamer in this years Google Summer of Code

2013-04-09T09:29:36+00:00

Some sad news, GStreamer did not get accepted to the 2013 Google Summer of Code. We don’t know the exact reasons, but all we can do at this point is redouble our efforts to get accepted again next year.

In the meantime not all hope is lost. You can still do a Google Summer of Code involving GStreamer with some of the other organisations that did get approved. For instance if you have an idea for a great multimedia desktop application using GStreamer you can try submitting that to for instance GNOME or KDE depending on your UI toolkit of choice as one example. In the past GNOME has also been open to hosting some pure GStreamer projects so you can always try submitting such too. In general I recommend people to take a look at the list of accepted organisations and see if there are projects which technologies would be relevant to the GSoC project you want to do and the apply with them. Also remember you can submit the same project to multiple organisations if it fits multiple projects.

Also the PiTiVi video editor did get approved so I strongly encourage you to take a look at their wonderful ideas page and submit a proposal to do a PiTiVi project.

Jean-François Fortin Tam: Le nipponophone de l’autobus

2013-04-05T23:08:17+00:00

Hier dans l’autobus, j’ai eu la chance de tomber sur un homme blanc parlant au téléphone… dans le langage du soleil levant. Après un très long moment de tergiversation mentale après la fin de son appel, j’osai:

«Bonjour monsieur, parlez-vous français? C’est pas tous les jours que j’entends parler japonais dans cet autobus, et vous le parlez très bien! Où l’avez-vous appris?»

«J’ai vécu au Japon pendant cinq ans!»

S’en suit une quinzaine de minutes de discussion où je découvris des choses qui changèrent certaines de mes perceptions, jusque-là biaisées par les ouï-dires.

D’abord, le taux de change entre le peso canadien et le Yen est plutôt désavantageux pour les canadiens en ce moment: l’économie japonaise, bien que mal en point du point de vue intérieur, reste assez forte sur le point de vue international, spécialement en comparaison avec l’Europe en crise depuis 2008-2009. Les investisseurs se replient alors sur une monnaie stable comme le Yen (ça, je dois dire que j’ai du mal à comprendre cette subtilité). Si l’Europe reprenait du poil de la bête, la valeur du Yen se rebalancerait possiblement.

Oui, le coût de la vie au Japon est élevé. C’est pourquoi il est préférable d’y travailler plutôt que d’être touriste, parce que les salaires sont en conséquence.

Surtout, je me demandais comment était l’accueil et l’intégration des étrangers s’établissant au japon. Certains reportages, combinés aux films comme «Stupeur et tremblements» (qui est, admettons-le, un peu dramatisé/comédique) et les opinions des gens autour de moi, me laissaient croire à une société fermée/repliée. Selon cet homme, il n’en est rien; il s’agit strictement du même phénomène que ce que l’on retrouve pratiquement partout dans le monde: si l’immigrant fait des efforts pour apprendre et maîtriser la langue locale, son intégration se fera avec succès alors que «les gens vont se précipiter avec joie pour vous aider», tout comme les Québécois francophones adoptent une attitude positive envers quiconque fait des efforts pour parler le Français.

Et vous? Habitez-vous au Japon ou connaissez-vous quelqu’un qui vous a fait témoignage de l’intégration des immigrants là-bas?

Andy Wingo: thoughts on blink

Andy Wingo — 2013-04-04T06:43:17+00:00

So Chromium forked WebKit again! You've probably seen some articles on it already, but Alex Russell's piece is pretty good.

In retrospect the split was easy to see coming, but I can't help but feel bittersweet about it. The announcement and Alex's article both cite technical reasons for the fork, but to me it seems to be ultimately a human failure. So the Chromium team saw problems in WebKit and agreed on a solution that will help them achieve their goals faster: that's great! But why were they unable to do so within WebKit itself?

There are technical differences, yes, but those could be resolved by human action (as they now will be, by humans, under the Blink aegis). No, to me the fundamental problem was a lack of the conditions for consensus within the WebKit project, and for that there is ample blame to go around.

In the best of events this split will end up with both Blink and WebKit as more technically consistent, nimble projects. I would also hope that this split serves as a wakeup call to some of the decision-making processes within WebKit. However it is easy to be pessimistic in this regard -- the Apple-Google dynamic has been great for WebKit as an open project, opening up discussions and collaboration possibilities, especially among folks that don't work for the two major corporations. Both projects will have to actively make an effort to remain open, because passivity will favor the structural tendency to work in-house.

I should probably mention the obvious thing, that Igalia will work with both projects, of course. We'll get our WebKit patches into Blink, and vice versa. The message for customers doesn't change much, either -- most customers have already chosen a particular WebKit port, and those that used Chromium will now use Chromium on Blink.

For those projects that haven't chosen a WebKit port yet, the fundamental decision has always been WebKit2/JSC versus Chromium/V8, and that hasn't changed too much from this announcement. It's true that WebKit2 is a less mature multi-process implementation than Chromium, but it is getting a lot of investment these days, so it's fair to assume that technically that the two approaches will work just as well within a few months. The same goes for JSC versus V8.

Finally, a meta-note. Sometimes these press releases catch us up in their whirlwind and we're left with a feeling of anxiety: "Google just released AngularJS, now I need to rewrite all my apps!?" Well, maybe ;) But I find a bit of dispassion is appropriate after a piece of news like this. Good luck to both projects, and see you on the mailing lists.

Bastien Nocera: Geocluing the desktop, slowly

Bastien Nocera — 2013-04-04T02:40:56+00:00

Over the past couple of months, Satabdi has been working during her Outreach Program for Women on geocode-glib, and Zeeshan more recently joined us to help with cleaning up some of the code.

As Satabdi's program is now finished (though not her involvement!), and a new GNOME development cycle has started, I'll try to explain where geocode-glib fits in, and answer some questions on the future of Geoclue.

(Reverse-)Geocoding

First and foremost, geocode-glib is a geocoding and reverse geocoding library. It uses Yahoo! web services for this, though we're investigating using Nominatim for this in the future.

This replaces the convoluted implementations for those two services in geoclue (3 if you include Address as a service). The API should be mostly stable now, and we'll soon start porting a few applications to it (Evolution and Empathy come to mind).

The library also includes a GeocodeLocation object. This will be useful later.

GeoIP

geocode-glib, thanks to Satabdi's work, includes a GeoIP server, to be installed on GNOME servers eventually, which uses data from MaxMind to locate a user connected to the Internet from the IP address. We also have a client library to access this server.

This is usually good enough to locate a user in a city, or a country, which would help us with many integration points in GNOME, such as the upcoming Date and Time panel re-design.

But this code isn't really for you, app developers.

Geoclue

A fine project, but the codebase is showing its age (dbus-glib!), and the decision, well, not to take decisions on which backends to use for various services make it fragile. This is a maintenance problem, both for in terms of making sure all the services are kept working, and that geoclue itself is kept stable.

So we'll start a reimplementation of Geoclue. The goals are:

Trimmed down API, just for positioning
Smaller, but more integrated, selection of ways to get the positioning (GPS from your integrated WWAN modem, Wi-Fi AP data, IP address, no discrete GPS or manual location)
Power-saving, by aggregating requests from all the applications
and finally, privacy, where only applications that you allow to request your location can get it, and only with an accuracy as needed for the application.

The code currently in geocode-glib for IP geocoding will likely move there when the project has been kickstarted.

I hope this clears any misconceptions people might have about geocode-glib, or, more likely, about the future of geoclue.

Jean-François Fortin Tam: GStreamer Hackfest 2013: Moving Images

2013-04-03T16:33:09+00:00

I’m back from this year’s GStreamer hackfest, which was fantastic as usual — an intersection of great minds, big challenges, flaky Wi-Fi and good food. Christian already did a generic summary, so I’ll be narrating from the GNonLin/GES/PiTiVi perspective. See the end of this blog post for a nice video retrospective.

Edward provided an initial patch to improve the behavior of timestamps and seeking in GNonLin, while Nicolas “Stormer” Dufresne fixed two bugs causing deadlocks. Nicolas spent a lot of time discussing with Wim Taymans, Edward Hervey, Sebastian Dröge and other hackfesters about the architecture of GNonLin in light of GStreamer 1.x. He also fixed looping for the Ogg demuxer in pull mode and, with some help from Mathieu “Forest Ranger” Duponchelle, fleshed out the design for a new tree data structure for GNonLin.

Mathieu the Moustached Avenger worked on implementing keyframes in GES, paving the way for him to create a user interface to animate any effect property in PiTiVi. That user interface will most likely depend on him working on the clutter timeline canvas, so I’m looking forward to improvements in that area.

Thibault “Keyboard Crusher” Saunier finished the implementation of GES Containers and clip groups, then worked on implementing — at long last — audio mixing in GES. This is an essential feature of multitrack audio/video editing, and I’m really happy to see that feature make its comeback for the next release of PiTiVi. This work will also depend on Mathieu’s keyframes UI. A proper reimplementation of video mixing remains to be done, however.

There are lots of outstanding things to solve in GNonLin and GES. Nicolas has a bunch of ideas for things to improve and redesign in GNonLin and I expect much collaboration between Thibault and him to optimize the entire stack for better reliability and performance (for example, adding caps filters to allow realtime downscaling of videos to improve preview performance, configurable downstream buffering for playback to avoid frame drops in CPU-intensive parts of a timeline, etc.). GNonLin and GES have much potential to allow us to be a lot smarter than before.

Personally, I spent most of my time testing, discussing and hacking on some new features for PiTiVi.

I added a button in the timeline toolbar that toggles the “gapless mode” (automatic ripple edits), which makes your clips behave like magnets and prevents needing to re-arrange them manually all the time. The feature works and will be merged after a customary code review.
I made some progress on the custom effects UI branch. Once it’s complete, you will be able to easily create custom user interfaces for effects that require it, simply by using a glade/gtkbuilder .ui file (or, if you prefer, a set of widgets from your own python module). Of course, for the majority of effects, our automatically generated user interfaces are still good enough, so we can keep using them and avoid unnecessary work.

(See my previous blog post for a situation report on where we stood with PiTiVi before the hackfest)

I also spent a bit of time setting up my film making gear and shooting various interesting moments of the hackfest. Here’s my montage, which will tell the story much better than a long blog post. Hope you’ll like it:

I would like to thank Collabora for allowing many GStreamer contributors to attend the hackfest, which I consider vital to the health of the GStreamer community. I was happy to meet again with many friends and help push the Free multimedia stack forward. Props to Christian Schaller and Alessandro Decina for organizing the whole thing, too!

Aside from Collabora and Fluendo sponsoring two of our dinners (thanks!), I would also like to thank you, PiTiVi supporters, for making it possible for me to spend some money to thank GStreamer contributors with some food and beer — maximum boost to the GStreamer community! Full disclosure: I used 84 euros worth of PiTiVi donations for that purpose.

Phil Normand: GStreamer hackfest in Milan

2013-04-01T10:29:37+00:00

I spent Easter in Milan, attending the second official GStreamer hackfest. It has been a productive event and, as always, a pleasure to hang out with some members of the GStreamer core dev team fame :)

I focused on a few GStreamer bugs that were blocking some other WebKit bugs related to the media player and the WebAudio backend. I also had in mind to port the osxaudio sink to 1.0, a task started one week before the hackfest, but didn't have time to come back to it.

The first task I worked on was about cleaning up some code in uridecodebin related to URI protocols and on-disk buffering. In WebKit we use a custom HTTP(S) source element used by playbin when loading remote media files. The issue is that that element is also exposed to applications using WebKit itself so we made it handle a custom protocol, webkit+http://. But that exposed a bug in urideocdebin where some protocols, including http, are hardcoded in a white list used to enable on-disk buffering. Unfortunately the aforementioned bug is not solved yet even though some good progress was made!

The second task I worked on was first reported by Wim, the WebKit media player wasn't pausing when receiving buffering messages. The fix was quite simple. After that I finally added some basic codec installer support in WebKit. The GStreamer codec installer API is quite simple :)

Lastly I started working again on the WebAudio createMediaSourceElement support for the GStreamer/WebKit backend. The patch has been sleeping in Bugzilla since last Christmas. That feature is interesting for WebAudio applications that need to use audio data coming from media elements to eg, do stuff like equalizers. The current patch in bugzilla gathers the audio buffers in lists and copy them to the AudioBus when needed. Those memcpys could probably be avoided by pre-allocating the memory for the AudioBus and telling GStreamer to write directly there. So, with the help of my friend Alessandro Decina (who was doing some similar stuff on his Firefox branch) I started working on a simple GstAllocator that would be configured in the pipeline by intercepting allocation queries toward the appsinks used to gather the audio buffers. Well, this exposed some bugs on the GStreamer side! The deinterleave element always uses the default allocator when pushing data to its source pads. So I started a patch that performs allocation queries downstream and use the allocator returned, if any. Unfortunately my custom allocator doesn't work yet but I'll cary on work on this, the GstAllocator API is quite an interesting thing to play with :)

All in all it was a great and productive event. Christian wrote a nice report about the hackfest too. I also wanted to thank Collabora and Fluendo for sponsoring dinners. A special thank you as well for Igalia which covered my travel expenses and attendance to the hackfest.

Andy Wingo: sexuality and sexism

Andy Wingo — 2013-03-26T20:07:10+00:00

Greetings, dear readers. Today's article is not about compilers, but about the people that write and run them. Like me, and like you.

I write a lot about programming here because it's interesting to me and it makes me happy, but that's not the extent of my desires as a human. Among all the things, and perhaps even foremost among them, is the desire to live in a more beautiful world: a world of making and sharing, of nature abloom, and of people too: a world, in short, full of life.

Part of that life is sexual, and how wonderfully, playfully, rightfully so. But the world as a whole would be better if we kept sexuality out of programming and other male-dominated pursuits.

The reason for this is that sexuality (for example, in the form of sexual jokes) among a group of men creates a kind of "boy's club" atmosphere in which people that aren't in the straight male majority feel uncomfortable. A "boy's club" has a virtual "no girls or queers allowed" sign on it. It's just not a comfortable place to be, for non-club-members.

Of course, sometimes being uncomfortable is good. But being uncomfortable because of your gender is not one of those cases. And even, it must be said, sometime it goes beyond discomfort to danger -- conferences that women do not attend for fear of groping; things that women cannot say for fear of rape threats. There is no hyperbole here. It is an astonishing, indignation-provoking injustice.

How did it get this bad?

As usual, through small steps. One of the first is widespread toleration of unrelated sexuality in male-dominated fields: boy's clubs. So I think that we all -- and especially men, and especially people with respect within a community -- should actively discourage any of these factors that lead to "boy's clubs". A joke that "I'd fork that project" is not OK. It would be fine if it were just a lame joke; but it's not fine because it's part of this whole "boy's club" thing.

Incidentally, there is a name for the structural tendency to favor one gender over another in a group that isn't "boy's club", and it is "sexism". Sometimes people dismiss sexism in programming because, you know, "show me the code", but this misses the broader picture. I personally have profited immensely from the personal connections I have made at the many conferences that I have attended. I've even put up with some "boy's club" jokes on the same level as "fork-that-repo". I think a woman would find it more difficult to make some of these connections, and so would not end up producing the patches I do.

So, friends, if you are with me in recognizing this structural injustice called sexism, a stain upon our community of compiler hackers and users, I think this is an occasion in which imperative programming is acceptable and even appropriate. Learn to recognize "boy's clubs" and work constructively against them. Sex and tech is usually a bad idea, so point this out when you see it -- especially if you are a man -- and support others who do the same.

Jean-François Fortin Tam: La carte Opus et les points de service qui n’en sont pas vraiment

2013-03-26T03:25:05+00:00

Il se trouve que les cartes Opus expirent (simplement parce qu’ils ont décidé que ça serait cool que ça expire, même si c’est une carte anonyme à tarif régulier). Chez moi, la pharmacie du coin a toujours offert le service de recharge de titres de transports, et c’est donc avec un grosse absence d’excitation que j’ai reçu la nouvelle du CIT Richelain comme quoi «le CIT a maintenant un point de service sur son territoire! C’est la pharmacie Jean Coutu où vous faisiez déjà vos transactions depuis dix ans!».

Or, ma carte Opus du moment était sur le point d’expirer (en vérifiant le numéro de série sur le site web dédié). Le site en question nous assure à maintes reprises que le renouvellement obligatoire, étant un caprice des administrateurs de l’Opus, est entièrement gratuit. Dans la FAQ, on trouve ceci:

3. Où dois-je aller pour remplacer sans frais ma carte OPUS ?
Présentez-vous avec votre carte qui expire dans moins de trois mois ou qui est expirée depuis moins de 6 mois dans un centre de service de la STM ou dans une billetterie métropolitaine de l’AMT, ou dans un centre de service d’un CIT.

Pour être doublement certain, la même chose en d’autres termes:

8. Je ne veux pas payer les frais de 6 $, que dois-je faire pour éviter ces frais ?

Présentez-vous avec votre carte qui arrive à expiration dans moins de 3 mois ou qui est expirée depuis moins de 6 mois dans un centre de service de la STM ou dans une billetterie métropolitaine de l’AMT, ou dans un centre de service d’un CIT pour obtenir une nouvelle carte OPUS sans avoir à payer les frais de 6 $.

S’il y a des titres de transport valides sur votre carte, ceux-ci seront transférés sur votre nouvelle carte.
[...]

Telle ne fut pas ma surprise lorsque, après avoir échangé ma carte chez PJC, on me demande des frais de renouvellement de 6$. Je m’objecte:

«Uuuuuh. Non, pas s’pôsé. Le site officiel de la carte Opus dit que le renouvellement est gratuit dans un point de service du CIT et il n’est pas question que je paie pour le faire, sinon j’aurais conservé ma carte qui a encore trois mois d’usage avant expiration»

S’en suit des regards confus des deux caissières devant moi, un appel à la direction qui décide d’annuler les frais «pour cette fois» (vu que la procédure est déjà toute faite, qu’ils ont déjà exigé une montagne de mes renseignements personnels pour une carte anonyme et qu’ils ont déjà invalidé l’ancienne).

On me dit toutefois que si je ne veux pas payer de tels frais, il faudrait que j’aille à la billetterie de l’AMT à Montréal. Brillant. Il sert à quoi, votre point de service «officiel» sur la rive sud?

Pour vérifier, je téléphone au CIT/Gestrans… qui eux-mêmes n’étaient visiblement pas au courant du fait que le site de la carte Opus indique que le renouvellement est gratuit. Après m’avoir fait patienter, on me dit que non, il faut vraiment aller à la billetterie métropolitaine au centre-ville.

«Donc le point de service officiel sur le territoire du CIT n’est pas plus utile qu’avant?

— Ouaip.»

Regardons de nouveau le site d’information sur le renouvellement de la carte opus. L’explication du tonnerre qui nous dévoile un peu le fouillis dans lequel on se trouve:

4. Pourquoi doit-on se présenter dans un centre de service de la STM, dans une billetterie métropolitaine de l’AMT, ou un centre de service d’un CIT pour faire remplacer sans frais la carte OPUS qui arrive à expiration ou qui est expirée ?

Parce que les centres de service et les billetteries possèdent l’équipement requis pour effectuer tous les types de transaction (renouvellement, reconstitution, transferts des titres de transport restants, etc.) alors que dans un point de vente, le marchand n’est autorisé qu’à effectuer la vente des titres de transport et, par le fait même, vous demandera de payer les frais de 6 $ pour votre nouvelle carte OPUS.

Pourtant, selon l’annonce, c’est bien un point de service officiel dont on parle:

Le CIT est fier de vous annoncer l’ouverture d’un centre de service sur son territoire. [...] Le centre de service est situé au: Esc. André Lachapelle Inc. – Jean Coutu [...]

Plus amusant encore:

7. Puis-je faire renouveler ou échanger ma carte dans un point de vente (dépanneur, pharmacie), ou dans une distributrice de titres de transport ?
Aucun renouvellement ou échange ne peut se faire dans un point de vente, ou à une distributrice de titres de transport. Le renouvellement ou l’échange des cartes qui expirent dans moins de trois mois, ou qui sont expirées depuis moins de 6 mois, doivent se faire dans un centre de service de la STM ou dans une billetterie métropolitaine de l’AMT, ou dans un centre de service d’un CIT.

Bref, si la pharmacie du coin n’était pas effectivement un centre de service officiel du CIT, on n’aurait pas été en mesure de m’y renouveler ma carte (ce qui a pourtant été le cas)!

Mais monsieur, on peut pas vous le faire gratuitement parce qu’on peut pas, mais on peut, mais on peut pas. Enfin pas vraiment.

GStreamer: GStreamer OpenMAX IL wrapper plugin 1.0.0 release

2013-03-22T17:40:00+00:00

The GStreamer team is pleased to announce the first GStreamer OpenMAX IL wrapper plugin release for the new API and ABI-stable 1.x series of the GStreamer multimedia framework.

Check out the release notes for gst-omx, or download tarballs for gst-omx,

GStreamer: GStreamer Core and Plugins 1.0.6 bug-fix release

2013-03-22T16:00:00+00:00

The GStreamer team is pleased to announce a new bug-fix release for the new API and ABI-stable 1.x series of the GStreamer multimedia framework.

Improvements include:

build fixes for out-of-tree autogen.sh and automake 1.13, and recent kernel/video4linux
gobject-introspection fixes for bindings
videoscale: Correct DAR and border calculations
playbin: make sure converters are always plugged when needed, fixes not-negotiated errors with some sinks
playbin: fix subtitleoverlay caps handling to avoid not-negotiated errors when subtitle plugins are missing
adder: make "caps" property work properly
alsasink: don't use 100% CPU in some cases
reliability fixes for flushing seeks and shutdowns in queue and bastransform
appsrc: fix locking order
audiovisualisation fixes
deinterlace: fix infinite loop on EOS with non-default methods or fields
avidemux push mode fixes, make dv-in-avidemux work
level: send a final level message on EOS
osxvideosink fixes
ximagesrc: Set the pixel aspect ratio correctly in the output caps
v4l2: don't check stride for encoded formats
leak fixes in GstBin, pango, auparse, gdppay
qtdemux: skip disabled tracks and ignore chapter subtitle tracks
RTSP and RTP fixes
Opus audio decoder, encoder and RTP payloader fixes
codecparser fixes for H.264, MPEG-2 and VC-1 parsing
opensles, eglglessink, and decklink plugins ported to 1.0
libav: fix H.264 decoding errors in some files and update to 0.8.5 release
miscellaneous bug fixes

Zeeshan Ali: New in Boxes 3.8

2013-03-20T03:06:58+00:00

We just rolled out Boxes 3.7.92 and have now entered code freeze. This means that apart from translations and doc updates, Boxes is ready for the stable 3.8 release (due next Monday). So whats new you ask? Since Boxes 3.6, we have been hard at work, making Boxes an app that is so awesome that it makes you want to use virtual machines if you are not already. :) Boxes is now a lot more reliable, has improved and more impressive UI and has cool new features:

Add USB redirection support in new VMs and option to add it into existing VMs. i-e You can use your USB devices (camera, flash drive etc) plugged into host from the guest operating system.

Smartcard support. This means if you have an appropriate reader device, you can use that to read information from your smart cards (credit/bank cards etc) from your virtual machine boxes.
Automatically download and installation of all virtio and QXL drivers, and spice-vdagent as part of Windows XP and Windows 7 express installation. This means following working out of the box:

Copy&paste between host and VM.
Guest resolution automatically adjusted to fit the Boxes window.
Very efficient disk I/O (helps in reducing installation time e.g).

Here

oVirt support. This means you can now access all your virtual machines running on a (most likely remote) oVirt broker from Boxes by simply providing it's URL. Christophe blogged about it last year.
Improved selection view:

Option to pause VMs.
Options to select all, none and running boxes.
Allow accessing properties for non-running VMs.

Here

Allow tweaking properties for remote machines in the wizard.
UI looks a lot more like UI mockups and more consistent with Documents.
Add initial experience greeting. As a consequence, we don't auto-launch wizard on first time usage anymore.

Please ignore the notification about box deletion. You don't get that on first use, I promise!
Support for changing the CD-ROM device/ISO from properties. You can see this option in the screenshot above along with USB redirection options.
Selecting item when maximized automatically enters fullscreen view. You see this in action in the QXL demo screencast I pointed to above.
Mark recommended RAM and storage of guest OS (when known) in properties.
Allow search with multiple terms. A search like "foo bar" now means name containing "foo" and "bar". This is nicer, and what you expect from e.g. gnome-shell search.
Allow drag to unfullscreen Boxes.
Troubleshooting log viewer to system properties.
For Developers:
- We now make use of libosinfo for express/automated installations. Support for new OSes/distros can be added through libosinfo. No Vala or C hacking needed, only XSL/XML.
Loads of other fixes and minor improvements. For more details: http://git.gnome.org/browse/gnome-boxes/tree/NEWS

Hope you ~~like~~ love Boxes 3.8!

Stefan Kost: 14 Mar 2013

2013-03-14T20:13:40+00:00

buzztard

As I already mentioned in last months issue, I ported buzztard and gst-buzztard to use gstreamer-1.0. It will require 1.1.X for some fixes in gstreamer itself. So how did it went. The basic porting to make it compile against 1.0 took about 20 hours spread over a couple of days. It is quite straight forward. I updated the porting guide where things were not mentioned.

The more tricky part where the soft api changes. At this port I was running the apps with G_DEBUG=fatal_warnings. The property name changes here and there we easy to fix. Also request pad name changes (%d -> _%u) took not too long to handle. Other things like caps negotiation kept me busy for a few evenings. For once I was e.g. writing GST_AUDIO_NE ("S16") instead of GST_AUDIO_NE (S16) which leads to caps negotiation failing. Another recommendation is to avoid setting channel-mask=3 for stereo caps. It is purely optional, most elements drop it anyway and by leaving it out, you can merge the mono and stereo caps.

I ported freeverb in plugins-bad and made fixed to audiopanorama and adder. Adder and collectpads have a few more new tests upstream. Another things that took a while was getting seek-in-ready to work again. Buzztard needs to configure the playback segment from the application, as the sources happy beep for all eternity. In 0.10 I could send a seek to my bin in ready, in 1.0 it gets dropped (pads are flushing). Now I recursively iterate over sources and seek on all of them. This is only needed to get to playing. To get these things fixed I ported a few of my stand alone examples, but the majority of this is still left unported.

Now the basics seem to work. I am confident that I can fix the remaining issues in march and look forward to attend the hackfest in Milano over easter.

The porting for buzztard:
69 files changed, 1909 insertions(+), 1473 deletions(-)
The porting for gst-buzztard:
34 files changed, 1254 insertions(+), 1414 deletions(-)

The other thing I was doing this month was to experiment with embeddable scripting engines. I wrote a toy gtk app that opens a window, sets up the scripting engine, registers a function to get the window handle and start a script that calls the function to get the window and adds a vbox with two labels. I did that for lua, seed and gjs. For lua it took 30 min, for seek it took 30min, for gjs it took 2 hours and is still not working fully (the custom function returning a gobject).

69 files changed, 1907 insertions(+), 1507 deletions(-)

Michael Sheldon: Ogre3D on Jolla’s SailfishOS

2013-02-28T21:26:10+00:00

I spent a bit of time hacking around with the Sailfish SDK and managed to get a rough port of Ogre3D working:

Video of Ogre3D running under Sailfish

This is running within the emulator using the Mesa LLVM OpenGL ES 2.0 implementation, so naturally I won’t be able to say for certain if a port is possible for actual phones running Sailfish until I get my hands on some hardware, but it seems hopeful.

Andy Wingo: on generators

Andy Wingo — 2013-02-25T16:17:25+00:00

Hello! Is this dog?

Hi dog this is Andy, with some more hacking nargery. Today the topic is generators.

(shift k)

By way of introduction, you might recall (with a shudder?) a couple of articles I wrote in the past on delimited continuations. I even gave a talk on delimited continuations at FrOSCon last year, a talk that was so hot that thieves broke into the the FrOSCon video squad's room and stole their hard drives before the edited product could hit the tubes. Not kidding! The FrOSCon folks still have the tapes somewhere, but meanwhile time marches on without the videos; truly, the terrorists have won.

Anyway, the summary of all that is that Scheme's much-praised call-with-current-continuation isn't actually a great building block for composable abstractions, and that delimited continuations have much better properties.

So imagine my surprise when Oleg Kiselyov, in a mail to the Scheme standardization list, suggested a different replacement for call/cc:

My bigger suggestion is to remove call/cc and dynamic-wind from the base library into an optional feature. I'd like to add the note inviting the discussion for more appropriate abstractions to supersede call/cc -- such [as] generators, for example.

He elaborated in another mail:

Generators are a wide term, some versions of generators are equivalent (equi-expressible) to shift/reset. Roshan James and Amr Sabry have written a paper about all kinds of generators, arguing for the generator interface.

And you know, generators are indeed a very convenient abstraction for composing producers and consumers of values -- more so than the lower-level delimited continuations. Guile's failure to standardize a generator interface is an instance of a class of Scheme-related problems, in which generality is treated as more important than utility. It's true that you can do generators in Guile, but we shouldn't make the user build their own.

The rest of the programming world moved on and built generators into their languages. James and Sabry's paper is good because it looks at yield as a kind of mainstream form of delimited continuations, and uses that perspective to place the generators of common languages in a taxonomy. Some of them are as expressive as delimited continuations. There are are also two common restrictions on generators that trade off expressive power for ease of implementation:

Generators can be restricted to disallow backtracking, as in Python. This allows generators to yield without allocating memory for a new saved execution state, instead updating the stored continuation in-place. This is commonly called the one-shot restriction.
Generators can also restrict access to their continuation. Instead of reifying generator state as external iterators, internal iterators call their consumers directly. The first generators developed by Mary Shaw in the Alphard language were like this, as are the generators in Ruby. This can be called the linear-stack restriction, after an implementation technique that they enable.

With that long introduction out of the way, I wanted to take a look at the proposal for generators in the latest ECMAScript draft reports -- to examine their expressiveness, and to see what implementations might look like. ES6 specifies generators with the the first kind of restriction -- with external iterators, like Python.

Also, Kiselyov & co. have a hip new paper out that explores the expressiveness of linear-stack generators, Lazy v. Yield: Incremental, Linear Pretty-printing. I was quite skeptical of this paper's claim that internal iterators were significantly more efficient than external iterators. So this article will also look at the performance implications of the various forms of generators, including full resumable generators.

ES6 iterators

As you might know, there's an official ECMA committee beavering about on a new revision of the ECMAScript language, and they would like to include both iterators and generators.

For the purposes of ECMAScript 6 (ES6), an iterator is an object with a next method. Like this one:

var p = console.log;

function integers_from(n) {
  function next() {
    return this.n++;
  }
  return { n: n, next: next };
}

var ints = integers_from(0);
p(ints.next()); // prints 0
p(ints.next()); // prints 1

Here, int_gen is an iterator that returns successive integers from 0, all the way up to ~~infinity~~2^53.

To indicate the end of a sequence, the current ES6 drafts follow Python's example by having the next method throw a special exception. ES6 will probably have an isStopIteration() predicate in the @iter module, but for now I'll just compare directly.

function StopIteration() {}
function isStopIteration(x) { return x === StopIteration; }

function take_n(iter, max) {
  function next() {
    if (this.n++ < max)
      return iter.next();
    else
      throw StopIteration;
  }
  return { n: 0, next: next };
}

function fold(f, iter, seed) {
  var elt;
  while (1) {
    try {
      elt = iter.next();
    } catch (e) {
      if (isStopIteration (e))
        break;
      else
        throw e;
    }
    seed = f(elt, seed);
  }
  return seed;
}

function sum(a, b) { return a + b; }

p(fold(sum, take_n(integers_from(0), 10), 0));
// prints 45

Now, I don't think that we can praise this code for being particularly elegant. The end result is pretty good: we take a limited amount of a stream of integers, and combine them with a fold, without creating intermediate data structures. However the means of achieving the result -- the implementation of the producers and consumers -- are nasty enough to prevent most people from using this idiom.

To remedy this, ES6 does two things. One is some "sugar" over the use of iterators, the for-of form. With this sugar, we can rewrite fold much more succinctly:

function fold(f, iter, seed) {
  for (var x of iter)
    seed = f(x, seed);
  return seed;
}

The other thing offered by ES6 are generators. Generators are functions that can yield. They are defined by function*, and yield their values using iterators:

function* integers_from(n) {
  while (1)
    yield n++;
}

function* take_n(iter, max) {
  var n = 0;
  while (n++ < max)
    yield iter.next();
}

These definitions are mostly equivalent to the previous iterator definitions, and are much nicer to read and write.

I say "mostly equivalent" because generators are closer to coroutines in their semantics because they can also receive values or exceptions from their caller via the send and throw methods. This is very much as in Python. Task.js is a nice example of how a coroutine-like treatment of generators can help avoid the fraction-of-an-action callback mess in frameworks like ~~Twisted~~node.js.

There is also a yield* operator to delegate to another generator, as in Python's PEP 380.

OK, now that you know all the things, it's time to ask (and hopefully answer) a few questions.

design of ES6 iterators

Firstly, has the ES6 committee made the right decisions regarding the iterators proposal? For me, I think they did well in choosing external iterators over internal iterators, and the choice to duck-type the iterator objects is a good one. In a language like ES6, lacking delimited continuations or cross-method generators, external iterators are more expressive than internal iterators. The for-of sugar is pretty nice, too.

Of course, expressivity isn't the only thing. For iterators to be maximally useful they have to be fast, and here I think they could do better. Terminating iteration with a StopIteration exception is not so great. It will be hard for an optimizing compiler to rewrite the throw/catch loop terminating condition into a simple goto, and with that you lose some visibility about your loop termination condition. Of course you can assume that the throw case is not taken, but you usually don't know for sure that there are no more throw statements that you might have missed.

Dart did a better job here, I think. They split the next method into two: a moveNext method to advance the iterator, and a current getter for the current value. Like this:

function dart_integers_from(n) {
  function moveNext() {
    this.current++;
    return true;
  }
  return { current: n-1, moveNext: moveNext };
}

function dart_take_n(iter, max) {
  function moveNext() {
    if (this.n++ < max && iter.moveNext()) {
      this.current = iter.current;
      return true;
    }
    else
      return false;
  }
  return {
    n: 0,
    current: undefined,
    moveNext: moveNext
  };
}

function dart_fold(f, iter, seed) {
  while (iter.moveNext())
    seed = f(iter.current, seed);
}

I think it might be fair to say that it's easier to implement an ES6 iterator, but it is easier to use a Dart iterator. In this case, dart_fold is much nicer, and almost doesn't need any sugar at all.

Besides the aesthetics, Dart's moveNext is transparent to the compiler. A simple inlining operation makes the termination condition immediately apparent to the compiler, enabling range inference and other optimizations.

iterator performance

To measure the overhead of for-of-style iteration, we can run benchmarks on "desugared" versions of for-of loops and compare them to normal for loops. We'll throw in Dart-style iterators as well, and also a stateful closure iterator designed to simulate generators. (More on that later.)

I put up a test case on jsPerf. If you click through, you can run the benchmarks directly in your browser, and see results from other users. It's imprecise, but I believe it is accurate, more or less.

The summary of the results is that current engines are able to run a basic summing for loop in about five cycles per iteration (again, assuming 2.5GHz processors; the Epiphany 3.6 numbers, for example, are mine). That's suboptimal only by a factor of 2 or so; pretty good. Thigh fives all around!

However, performance with iterators is not so good. I was surprised to find that most of the other tests are suboptimal by about a factor of 20. I expected the engines to do a better job at inlining.

One data point that stands out of my limited data set is the development Epiphany 3.7.90 browser, which uses JavaScriptCore from WebKit trunk. This browser did significantly better with Dart-style iterators, something I suspect due to better inlining, though I haven't verified it.

Better inlining and stack allocation is already a development focus of modern JS engines. Dart-style iterators will get faster without being a specific focus of optimization. It seems like a much better strategy for the ES6 specification to piggy-back off this work and specify iterators in such a way that they will be fast by default, without requiring more work on the exception system, something that is universally reviled among implementors.

what about generators?

It's tough to know how ES6 generators will perform, because they're not widely implemented and don't have a straightforward "desugaring". As far as I know, there is only one practical implementation strategy for yield in JavaScript, and that is to save the function's stack and the set of live registers. Resuming a generator copies the activation back onto the stack and restores the registers. Any other strategy that involves higher-level control-flow rewriting would make debugging too difficult. This is not a terribly onerous requirement for an implementation. They already have to know exactly what is live and what is not, and can reconstruct a stack frame without too much difficulty.

In practice, invoking a generator is not much different from calling a stateful closure. For this reason, I added another test to the jsPerf benchmark I mentioned before that tests stateful closures. It does slightly worse than iterators that keep state in an object, but not much worse -- and there is no essential reason why it should be slow.

On the other hand, I don't think we can expect compilers to do a good job at inlining generators, especially if they are specified as terminating with a StopIteration. Terminating with an exception has its own quirks, as Chris Leary wrote a few years ago, though I think in his conclusion he is searching for virtues where there are none.

dogs are curious about multi-shot generators

So dog, that's ECMAScript. But since we started with Scheme, I'll close this parenthesis with a note on resumable, multi-shot generators. What is the performance impact of allowing generators to be fully resumable?

Being able to snapshot your program at any time is pretty cool, but it has an allocation cost. That's pretty much the dominating factor. All of the examples that we have seen so far execute (or should execute) without allocating any memory. But if your generators are resumable -- or even if they aren't, but you build them on top of full delimited continuations -- you can't re-use the storage of an old captured continuation when reifying a new one. A test of iterating through resumable generators is a test of short-lived allocations.

I ran this test in my Guile:

(define (stateful-coroutine proc)
  (let ((tag (make-prompt-tag)))
    (define (thunk)
      (proc (lambda (val) (abort-to-prompt tag val))))
    (lambda ()
      (call-with-prompt tag
                        thunk
                        (lambda (cont ret)
                          (set! thunk cont)
                          ret)))))

(define (integers-from n)
  (stateful-coroutine
   (lambda (yield)
     (let lp ((i n))
       (yield i)
       (lp (1+ i))))))

(define (sum-values iter n)
  (let lp ((i 0) (sum 0))
    (if (< i n)
        (lp (1+ i) (+ sum (iter)))
        sum)))

(sum-values (integers-from 0) 1000000)

I get the equivalent of 2 Ops/sec (in terms of the jsPerf results), allocating about 0.5 GB/s (288 bytes/iteration). This is similar to the allocation rate I get for JavaScriptCore, which is also not generational. V8's generational collector lets it sustain about 3.5 GB/s, at least in my tests. Test case here; multiply the 8-element test by 100 to yield approximate MB/s.

Anyway these numbers are much lower than the iterator tests. ES6 did right to specify one-shot generators as a primitive. In Guile we should perhaps consider implementing some form of one-shot generators that don't have an allocation overhead.

Finally, while I enjoyed the paper from Kiselyov and the gang, I don't see the point in praising simple generators when they don't offer a significant performance advantage. I think we will see external iterators in JavaScript achieve near-optimal performance within a couple years without paying the expressivity penalty of internal iterators.

Catch you next time, dog!

Andy Wingo: opengl particle simulation in guile

Andy Wingo — 2013-02-16T19:20:14+00:00

Hello, internets! Did you know that today marks two years of Guile 2? Word!

Guile 2 was a major upgrade to Guile's performance and expressiveness as a language, and I can say as a user that it is so nice to be able to rely on such a capable foundation for hacking the hack.

To celebrate, we organized a little birthday hack-feast -- a communal potluck of programs that Guilers brought together to share with each other. Like last year, we'll collect them all and publish a summary article on Planet GNU later on this week. This is an article about the hack that I worked on.

figl

Figl is a Guile binding to OpenGL and related libraries, written using Guile's dynamic FFI.

Figl is a collaboration between myself and Guile super-hacker Daniel Hartwig. It's pretty complete, with a generated binding for all of OpenGL 2.1 based on the specification files. It doesn't have a good web site yet -- it might change name, and probably we'll move to savannah -- but it does have great documentation in texinfo format, built with the source.

But that's not my potluck program. My program is a little demo particle simulation built on Figl.

Click to download video

Source available here. It's a fairly modern implementation using vertex buffer objects, doing client-side geometry updates.

To run, assuming you have some version of Guile 2.0 installed:

git clone git://gitorious.org/guile-figl/guile-figl.git
cd guile-figl
autoreconf -vif
./configure
make
# 5000 particles
./env guile examples/particle-system/vbo.scm 5000

You'll need OpenGL development packages installed -- for the.so links, not the header files.

performance

If you look closely on the video, you'll see that the program itself is rendering at 60 fps, using 260% CPU. (The CPU utilization drops as soon as the video starts, because of the screen capture program.) This is because the updates to the particle positions and to the vertices are done using as many processors as you have -- 4, in my case (2 real and 2 threads each). I've gotten it up to 310% or so, which is pretty good utilization considering that the draw itself is single-threaded and there are other things that need to use the CPU like the X server and the compositor.

Each particle has a position and a velocity, and is attracted to the center using a gravity-like force. If you consider that updating a particle's position should probably take about 100 cycles (for a square root, some divisions, and a few other floating point operations), then my 2.4GHz processors should allow for a particle simulation with 1.6 million particles, updated at 60fps. In that context, getting to 5K particles at 60 fps is not so impressive -- it's sub-optimal by a factor of 320.

This slowness is oddly exciting to me. It's been a while since I've felt Guile to be slow, and this is a great test case. The two major things to work on next in Guile are its newer register-based virtual machine, which should speed things up by a factor of 2 or so in this benchmark; and an ahead-of-time native compiler, which should add another factor of 5 perhaps. Optimizations to an ahead-of-time native compiler could add another factor of 2, and a JIT would probably make up the rest of the difference.

One thing I was really wondering about when making this experiment was how the garbage collector would affect the feeling of the animation. Back when I worked with the excellent folks at Oblong, I realized that graphics could be held to a much higher standard of smoothness than I was used to. Guile allocates floating-point numbers on the heap and uses the Boehm-Demers-Weiser conservative collector, and I was a bit skeptical that things could be smooth because of the pause times.

I was pleasantly surprised that the result was very smooth, without noticeable GC hiccups. However, GC is noticeable on an overloaded system. Since the simulation advances time by 1/60th of a second at each frame, regardless of the actual frame rate, differing frame processing times under load can show a sort of "sticky-zipper" phenomenon.

If I ever got to optimal update performance, I'd probably need to use a better graphics card than my Intel i965 embedded controller. And of course vertex shaders are really the way to go if I cared about the effect and not the compiler -- but I'm more interested in compilers than graphics, so I'm OK with the state of things.

Finally I would note that it has been pure pleasure to program with general, high-level macros, knowing that the optimizer would transform the code into really efficient low-level code. Of course I had to check the optimizations, with the ,optimize, command at the REPL. I even had to add a couple of transformations to the optimizer at one point, but I was able to do that without restarting the program, which was quite excellent.

So that's my happy birthday dish for the Guile 2 potluck. More on other programs after the hack-feast is over; until then, see you in #guile!

Stefan Kost: 14 Feb 2013

2013-02-14T20:57:17+00:00

buzztard 0.7 is out

We started the year with a release in the beginning of January. So far only a few issues were reported. Patches for these are on the 0.7.1 branch. One last minute change in 0.7 was the departure from trying to link/unlink elements in gstreamer while playing. I sadly have to conclude that in 0.10 this only works for some cases. It was working sort of okayish for buzztard, but the were still deadlocks or internal dataflow errors from time to time. One such scenario is that when you happen to reconnect elements while the song loops. These things are not easy to sync from the application level. Lets hope this is easier in 1.0. The sticky events should help with setting up the context from new elements.

On the mainline I bumped the required library version and removed the ifdefs for the backwards compatibility. As a first bigger change, I ported the settings from gconf to dconf. We had a settings abstraction to other gconf or play keyfiles, which we now removed. Altogether I kept to use an intermediate object that exposes settings as a gobject with properties for the keys. This allows to use notify to bind to property changes instead of using a custom api.

Then I started with porting gst-buzztard & buzztard to gstreamer-1.0. I made good progress. I will report the details in the next update.

77 files changed, 5892 insertions(+), 8131 deletions(-)

Guillaume Emont: Migrating to Jekyll

2013-02-09T23:00:00+00:00

I’ve decided to switch my blog from WordPress to a Jekyll-based system. When this article goes live, the migration should be done. I hope I managed to avoid flooding the planets that aggregate me with old articles (Planet Igalia, Planet GStreamer and Planet GNOME), if I failed, I apologise, feed aggregators are a bit like dark magic to me.

CC BY-NC-SA 2.0 by Kevin Lim

Why?

The main motivation behind that is that is that I am very tired of the WordPress editor. I am a programmer and spend my days using the same editor¹, which means that I have gotten rather efficient using it, or, at least, that’s the way of writing text that is the most comfortable to me.

In a stock WordPress installation, all you could do to edit your blog post in your own editor is to edit it in HTML and then cut and paste your result in the editor and publish. HTML. Eeek! I don’t want to be editing a tag soup, I want to focus on the content. There are better languages than HTML for that, such as reStructuredText or MarkDown. I did try WordPreSt a while ago but wasn’t satisfied with the work flow it provided, and if I remember correctly, I ended up having to correct things manually.

So, reading up a bit on the internets on what’s available, I got seduced by the concept of jekyll and its principle of offline generation of static pages, and the default workflow is very similar to the one I am used to when doing software development. Also, it seems to be actively maintained and used in GitHub Pages, which gives me a feeling that it’s here to stay. The only drawback that I see to it is that commenting cannot be handled by strictly static web pages. Most people using jekyll for blogging seem to resort to using external commenting services, such as Disqus and friends or even facebook, but I’d rather not depend on an external service for that. I’ve found a solution to that that seems decent to me, and I explain this in a later section.

Jekyll in itself is not a blogging system out of the box, but a good tool to make one. Since I didn’t feel like reinventing the wheel, I am using Jekyll-Bootstrap as a base for my blog.

Conversion of posts

I’d rather leave my old articles online, while still not have to manage a WordPress installation, so I definitely want to migrate my old articles to the new system. Luckily, jekyll comes out of the box with migration tools to make your life easy. I’ve used the tool that gets the content from a Wordpress export file and it went (mostly) like a breeze, as far as I could check. I only had to modify a bit the image caption code in one article. I still have some image files that aren’t where they should be in old posts, but that was already the case with WordPress. I will need to fix these articles at some point.

Static comments

The issue that I had left to deal with is to be able to have comments, and to import the comments I had in WordPress. I chose to use Jekyll::StaticComments (code here for that purpose. It is a plugin for Jekyll that takes comments in YAML files and renders them at the end of a post. That still left me with two problems: how do I migrate old comments from WordPress, and how do I let people add new comments?

Comment Migration

I’ve written a quick and dirty script with BeautifulSoup that gets the comment from a WordPress export file and generates a suitable tree of YAML files for use by Jekyll::StaticComments. I’ve put it online and called it wp2jekyll-comments.

Adding new comments

Jekyll:StaticComments comes with a PHP script that allows one to submit comments. Since I’m an awful geek, I don’t like much the idea of having to use PHP here, so I made my own version of it in Python. I’ve published it as jekyll-comments.

vim
↩

Andy Wingo: knocking on private back doors with the web browser

Andy Wingo — 2013-02-04T08:47:09+00:00

I woke up at five this morning with two things in my head.

One was, most unfortunately, Rebecca Black's Friday, except with "Monday" -- you know, "Monday, Monday, gettin' up on Monday, hack, hack, hack, hack, which thing should I hack?", et cetera.

The other was a faint echo of Patrick McKenzie's great article on the practical implications of the recent Rails vulnerabilities. In particular, he pointed out that development web servers running only on your local laptop could be accessed by malicious web pages.

Of course this is not new, strictly speaking, but it was surprising to me in a way that it shouldn't have been. For example, Guile has a command-line argument to run a REPL server on a local port. This is fantastic for interactive development and debugging, but it is also a vulnerability if you run a web server on the same machine. A malicious web page could request data from the default Guile listener port, which is a short hop away from rooting your machine.

I made a little test case this morning: a local port scanner. It tries to fetch http://localhost:port/favicon.ico for all ports between 1 and 10000 on your machine. (You can click that link; it doesn't run the scan until you click a button.)

If it finds a favicon, that indicates that there is a running local web application, which may or may not be vulnerable to CSRF attacks or other attacks like the Rails yaml attack.

If the request for a favicon times out, probably that port is vulnerable to some kind of attack, like the old form protocol attack.

If the request fails, the port may be closed, or the process listening at the port may have detected invalid input and closed the connection. We could run a timing loop to determine if a connection was made or not, but for now the port scanner doesn't output any information for this case.

In my case, the page finds that I have a probably vulnerable port 2628, which appears to be dictd.

Anyway, I hope that page is useful to someone. And if you are adding backdoors into your applications for development purposes, be careful!

Bastien Nocera: Power management in GNOME 3.8

Bastien Nocera — 2013-02-01T08:45:32+00:00

In the past couple of weeks, apart from reviewing very many patches for gnome-control-center (especially for new and re-designed panels), I've been working on updating the power management handling in GNOME.

Test suite

The first change is that we have a test suite (currently with 15 separate tests) to test interaction between gnome-settings-daemon's power management and various session and system components. This is thanks to Martin Pitt, and his work on python-dbusmock.

We'll try and add new tests as bug reports come in to avoid regressions, although some cases will remain untested because of limitations in our logging.

All clear

Screensaver and backlight interaction

With gnome-shell becoming the sole screensaver (after the removal of fallback mode, and the obsoletion of gnome-screensaver), we've been able to streamline the code handling the various screen backlight power levels.

Your screen will now turn off as soon as the screensaver kicks in, moving your mouse in the screensaver will turn it back on for 20 seconds before turning off again, and when to dim (if you've chosen so) is dependent on whether you're on battery or not, and the default idle time (eg. if your screen turns off after 5 minutes of inactivity, the screen will dim after 4). This makes the behaviour more consistent, and predictable, compared to the mish-mash of settings we had before, where some delays were available for change in the UI, and others only through GSettings or gnome-tweak-tool.

Those constants are separate from the code, and exported to the test suite so they are flexible and can be changed if the behaviour doesn't exactly match what users are expecting.

The other change relating to that, is that the screen shield will now always pop down when the screensaver kicks in (thanks to Giovanni for the gnome-shell work). This doesn't mean that you'll have to enter your password each time, but only after the "lock delay" if you've set one.

We've also added a number of nice touches, like the screen turning back on for a short period when you plug or unplug your laptop, made sure that your laptop screen gets turned off and your session locked when closing the lid and turn off the backlight for machines where suspend causes the backlight to come back on temporarily (as seen on MacBooks).

Very very idle

We've also added a long-requested feature: the ability to force logout after a period of idle. This is useful in kiosk and computer lab situations, and is only available through GSettings. As we've added support for this feature (warning prior to logging out, with the screen turning on for a couple of seconds when the warning shows up), we've realised that the infrastructure is the same for automatic suspend/hibernate situation. This means I expect to change the default "long idle" behaviour to suspending. This will still be changeable in the Power preferences. This should land after 3.7.5, and don't worry, we'll make this change very visible in the release notes :)

*I* am not suspending by default

Inhibit

But you don't want to suspend, you really don't.

GNOME supports the draft FreeDesktop "Idle inhibition" specification, as implemented by KDE, which hopefully means that more third-party applications should start behaving better when playing back films, in presentation mode, or for large overnight downloads. This should hopefully get out of draft status before the GNOME 3.8 release.

We also have a gnome-session-inhibit tool available in gnome-session for your scripting needs.

Colophon

All the changes mentioned should be available in GNOME 3.7.5, and I will be available to take complaints at FOSDEM this week-end.

Bastien Nocera: Office-runner 1.0

Bastien Nocera — 2013-01-28T21:45:24+00:00

Office Runner 1.0 is now available, downloadable at the usual location.

We had some pretty good comments on the original announcement, even though some people took this opportunity to start slinging insults, which is pretty uncool.

There's plenty of related power management work that's been happening in preparation of GNOME 3.8, and we're mostly waiting on GNOME Shell changes to land before I can talk to you about those. I hope it gets the same (positive) comments as Office Runner did, though I imagine some people will get angry about changes, as they are wont to do.

This version requires systemd for the suspend inhibition.

And don't forget to post your high scores ;)

Thomas Vander Stichele: measuring puppet

2013-01-24T19:58:17+00:00

For one of work’s projects, we’ll soon be working on scaling our platform more, which will require deploying a bunch more machines. For this project, we basically have a local dev platform, an online dev platform, a preproduction platform, and a production platform.

Right now, all these platforms have exactly one host. There are two puppetmasters, one for the dev platforms and one for the pre/pro platforms.

Since deploying a bunch more machines is going to require a lot more puppet running, I want to work on removing as much friction as I can from my puppet work. I do the runs manually as we upgrade platforms during deployment, and a run typically takes well over a minute. For me, that’s too long – it causes me to waste time, lose focus, task switch, and forget I should be following up on puppet runs. It makes finetuning puppet modules a chore as I hack on them.

So I wanted to start by trimming some of the obvious fat before I segment my puppet config into separately testable pieces. I would have expected puppet apply to actually have something to help with that, but it doesn’t.

After thinking it through, I realized I wanted some kind of tool that would timestamp output of puppet apply –debug so I could see which things it does take more time than others.

I wasn’t sure what to google for, but timestamp stdout seemed to bring up some results, and I hit on http://joeyh.name/code/moreutils/ which includes a tool called ‘ts’ and is a simple pipe filter that timestamps lines going to stdout.

That was almost good enough. What I really wanted though was to know how much time elapsed since printing the last line. My perl is rusty, but I managed to quickly cook up a patch that makes it print incremental timestamps.

Now I can do a puppet run like this:
puppet apply --modulepath=`pwd`/modules:`pwd`/dev/modules manifests/site.pp --debug | ts -i "%H:%M:%.S" | egrep -B 1 "^00:00:0[^0]" 00:00:00.001066 debug: Executing 'test -e ../commit && ( test xorigin/master == `cat ../commit` || ( git fetch -a; test `git rev-parse --verify origin/master^0 | head -n 1` == `cat ../commit` ) )' 00:00:02.646908 debug: Service[postfix](provider=redhat): Executing '/sbin/service postfix status' -- 00:00:00.000987 debug: Executing 'test -e ../commit && ( test xorigin/release-1.4.x == `cat ../commit` || ( git fetch -a; test `git rev-parse --verify origin/release-1.4.x^0 | head -n 1` == `cat ../commit` ) )' 00:00:01.871258 debug: Exec[git-checkout-/var/www/partner-test](provider=posix): Executing check 'test -e ../commit && ( test xorigin/release-1.4.x == `cat ../commit` || ( git fetch -a; test `git rev-parse --verify origin/release-1.4.x^0 | head -n 1` == `cat ../commit` ) )' 00:00:00.000942 debug: Executing 'test -e ../commit && ( test xorigin/release-1.4.x == `cat ../commit` || ( git fetch -a; test `git rev-parse --verify origin/release-1.4.x^0 | head -n 1` == `cat ../commit` ) )' 00:00:01.886606 debug: Prefetching aliases resources for mailalias -- 00:00:00.000957 debug: Executing '/usr/sbin/semanage fcontext -l | grep -q '^/home/git/dev(/.*)?'' 00:00:01.750281 debug: /Stage[main]/Dexter::Apache/Selinux::Set_fcontext[home-httpd]/Exec[semanage-/home/git/dev-httpd_sys_content_t]/unless: /usr/sbin/semanage: Broken pipe -- 00:00:00.000855 debug: Executing 'test -e ../commit && ( test xorigin/release-1.4.x == `cat ../commit` || ( git fetch -a; test `git rev-parse --verify origin/release-1.4.x^0 | head -n 1` == `cat ../commit` ) )' 00:00:02.064475 debug: /Schedule[puppet]: Skipping device resources because running on a host -- 00:00:00.001048 debug: Executing '/usr/sbin/semanage fcontext -l | grep -q '^/srv/merchant(/.*)?'' 00:00:01.750129 debug: /Stage[main]/Partner::Install/Selinux::Set_fcontext[srv-merchant-httpd]/Exec[semanage-/srv/merchant-httpd_sys_content_t]/unless: /usr/sbin/semanage: Broken pipe -- 00:00:00.000861 debug: Executing 'test -e ../commit && ( test xmaster == `cat ../commit` || ( git fetch -a; test `git rev-parse --verify master^0 | head -n 1` == `cat ../commit` ) )' 00:00:01.841316 debug: Exec[git-checkout-/var/www/merchant](provider=posix): Executing check 'test -e ../commit && ( test xorigin/release-1.4.x == `cat ../commit` || ( git fetch -a; test `git rev-parse --verify origin/release-1.4.x^0 | head -n 1` == `cat ../commit` ) )' 00:00:00.000955 debug: Executing 'test -e ../commit && ( test xorigin/release-1.4.x == `cat ../commit` || ( git fetch -a; test `git rev-parse --verify origin/release-1.4.x^0 | head -n 1` == `cat ../commit` ) )' 00:00:01.858206 debug: Service[httpd](provider=redhat): Executing '/sbin/service httpd status'

Some explanation is in order.

The puppet apply is straightforward if you know puppet a little – it will apply a manifest and spit out a lot of debug info.

The output gets piped into ts, which will do incremental timestamping (with -i, which is what my patch adds) according to the specified format (ts by default uses seconds precision, but can do microsecond precision if you use %.S in the format).

Then I grep for all lines that take at least 1 second to be output, and display the line before that too (since puppet is generating output either before or after a possibly long-running task, so either on the line before or the line that took too long).

In the first section, I doubt service postfix status is to blame, so it’s probably my convoluted git updating that takes too long. I need to rework that module so it doesn’t fetch on every run.

In the third section, semanage is to blame. Hm, maybe I need to find a different way to look up whether the particular fcontext rule I want to add is already there. I’ve considered converting it to facts, although that sounds like it would be stretching facts a little – that’s a lot of info to store in a fact.

The others are repeats of both, so I know where to start trimming the fat now!

And when all > 1 sec items are gone, time to shave off more below that.

If you want to try out ts with incremental timestamping, it’s available in the rebuilt moreutils rpm in my package repositories for CentOS 6 and F16/17/18.

If any puppetmaster (hah!) has good tips on how to debug and measure the catalog generation step (the one on the master), let me know!

Thomas Vander Stichele: mach 1.0.2 “ears” released

2013-01-22T21:32:41+00:00

Another Fedora, another mach release. This release fixes a minor bug and adds support for Fedora 18.

Get the source, update from my repository, or wait until updates hit the Fedora repository.

Happy packaging!

Andy Wingo: an opinionated guide to scheme implementations

Andy Wingo — 2013-01-07T13:43:35+00:00

Hark, the beloved programing language! But hark, also: for Scheme is a language of many implementations. If Scheme were a countryside, it would have its cosmopolitan cities, its hipster dives, its blue-collar factory towns, quaint villages, western movie sets full of façades and no buildings, shacks in the woods, and single-occupancy rent-by-the-hour slums. It's a confusing and delightful place, but you need a guide.

And so it is that there is a steady rivulet of folks coming into the #scheme IRC channel asking what implementation to use. Mostly the regulars are tired of the question, but when they do answer it's in very guarded terms -- because the topic is a bit touchy, and a bit tribal. There's a subtle tension between partisans of the different implementations, but people are polite enough to avoid controversial, categorical statements. Unfortunately the end result of this is usually silence in the channel.

Well I'm tired of it! All that silence is boring, and so I thought I'd give you my opinionated guide to Scheme implementations. This is necessarily subjective and colored, as I co-maintain the Guile implementation and do all of my hacking in Guile. But hey, to each folk their stroke, right? What I say might still be valuable to you.

So without further introduction, here are seven recommendations for seven use cases.

The embedded Scheme

So you have a big project written in C or C++ and you want to embed a Scheme interpreter, to allow you to extend this project from the inside in a higher-level, dynamic language. This is the sort of thing where Lua really shines. So what Scheme to use?

Use Chibi Scheme! It's small and from all accounts works great for static linking. Include a copy of it in your project, and save yourself the headache of adding an external dependency to something that changes.

Embedding Scheme is a great option if you need to ship an application on Windows, iOS, or Android. The other possibility is shipping a native binary created by a Scheme that compiles natively, so...

The Scheme that can make a binary executable

Sometimes what you want to do is just ship an executable, and not rely on the user to install a runtime for a particular Scheme system. Here you should use Chicken or Gambit. Both of these Schemes have a compiler that produces C, which it then passes on to your system's C compiler to generate an executable.

Another possibility is to use Racket, and use its raco exe and raco dist commands to package together a program along with a dynamically-linked runtime into a distributable directory.

The dynamically linked Scheme

What if you want to extend a project written in C, C++, or the like, but you want to dynamically link to a Scheme provided with the user's system instead of statically linking? Here I recommend Guile. Its C API and ABI are stable, parallel-installable, well-documented, and binary packages are available on all Free Software systems. It's LGPL, so it doesn't impose any licensing requirements on your C program, or on the Scheme that you write.

The Scheme with an integrated development environment

What if you really like IDEs? For great justice, download you a Racket! It has a great debugger, an excellent Scheme editor, on-line help, and it's easy to run on all popular platforms. Incidentally, I would say that Racket is probably the most newbie-friendly Scheme there is, something that stems from its long association with education in US high schools and undergraduate programs. But don't let the pedagogical side of things fool you into thinking it is underpowered: besides its usefulness for language research, Racket includes a package manager with lots of real-world modules, and lots of people swear by it for getting real work done.

The Scheme for SICP

Many people come by #scheme asking which Scheme they should use for following along with Structure and Interpretation of Computer Programs. SICP was originally written for MIT Scheme, but these days it's easier to use Neil Van Dyke's SICP mode for Racket. It has nice support for SICP's "picture langauge". So do that!

The server-side Scheme

So you write the intertubes, do you? Well here you have a plethora of options. While I write all of my server-side software in Guile, and I think it's a fine implementation for this purpose, let me recommend Chicken for your consideration. It has loads of modules ("eggs", they call them) for all kinds of tasks -- AWS integration, great HTTP support, zeromq, excellent support for systems-level programming, and they have a good community of server-side hackers.

The Scheme for interactive development with Emacs

Install Guile! All right, this point is a bit of an advertisement, but it's my blog so that's OK. So the thing you need to do is install Paredit and Geiser. That page I linked to gives you the procedure. From there you can build up your program incrementally, starting with debugging the null program. It's a nice experience.

You and your Scheme

These recommendations are starting points. Most of these systems are multi-purpose: you can use Gambit as an interpreter for source code, Racket for server-side programing, or Chicken in Emacs via SLIME. And there are a panoply of other fine Schemes: Chez, Gauche, Kawa, and the list goes on and on (and even farther beyond that). The important thing in the beginning is to make a good starting choice, invest some time with your choice, and then (possibly) change implementations if needed.

To choose an implementation is to choose a tribe. Since Scheme is so minimal, you begin to rely on extensions that are only present in your implementation, and so through code you bind yourself to a world of code, people, and practice, loosely bound to the rest of the Scheme world through a fictional first-person-plural. This is OK! Going deep into a relationship with an implementation is the only way to do great work. The looser ties to the rest of the Scheme world in the form of the standards, the literature, the IRC channel, and the mailing lists provide refreshing conversation among fellow travellers, not marching orders for a phalanx.

So don't fear implementation lock-in -- treat your implementation's facilities as strengths, until you have more experience and know more about how your needs are served by your Scheme.

Finally...

Just do it! You wouldn't think that this point needs elaboration, but I see so many people floundering and blathering without coding, so it bears repeating. Some people get this strange nostalgic "Scheme is an ideal that I cannot reach" mental block that is completely unproductive, not to mention lame. If you're thinking about using Scheme, do it! Install your Scheme, run your REPL, and start typing stuff at it. And try to write as much of your program in Scheme as possible. Prefer extension over embedding. Use an existing implementation rather than writing your own. Get good at using your REPL (or the IDE, in the case of Racket). Read your implementation's manual. Work with other people's code so you get an idea of what kind of code experienced Schemers write. Read the source code of your implementation.

See you in #scheme, and happy hacking!

Stefan Kost: 1 Jan 2013

2013-01-01T18:02:31+00:00

buzztard

Happy new 2013!

Besides a lot of testing and little bug fixes here and there I finished of a few half ready features. One is range-randomize (press Ctrl+Shift+R). In contrast to randomize which takes the parameters min-max value as bounds, this one picks the bounds from the first and last value in the selected range. The other one is that now one can rename machines also from the sequence headers. I was afraid that swapping the GtkLabel with a GtkEntry would use too much space and look ugly, but luckily one can turn off the frame. I still get a different background color though.

During testing I noticed some inconsistencies when renaming machines and found a rather bad data corruption. Certain interaction would create songs that won't read back fully. One can manually fix them up (it is xml after all). I fixed the bug and could even make the format a bit simpler. I had to touch quite a few places:
30 files changed, 246 insertions(+), 404 deletions(-)
Old songs can still be read, but songs written with the new version, also need the new version for reading. One lesson learned, the 'id' attribute in xml is special.

Another results from testing is that the shell based test scripts can now be run standalone to check a bunch of songs (do syntax check, print stats, convert or encode songs).

Finally I entered string freeze for 0.7. I already received the first updates for the translations, thanks a lot! I am looking forward to release it soon.

I uploaded some example song snippets to soundcloud.

438 files changed, 3138 insertions(+), 2800 deletions(-)

Bastien Nocera: Settings news

Bastien Nocera — 2012-12-21T15:22:48+00:00

GNOME 3.7.3 just got released earlier today, and includes some great new work. I won't be posting screenshots, because some of the UIs aren't final, and we'll be iterating until 3.8 is released (and it's my birthday ;).

Cleaning up

We've cleaned up gnome-settings-daemon plug-ins, and gnome-control-center panels, as well as removing the support code for GNOME Fallback, saving us around 10k lines of code.

gnome-control-center (now "Settings" in the UI) is faster to start, and gnome-settings-daemon require less code to write additional plug-ins.

New panels

3 new panels got added:

Search panel, to control the search output in gnome-shell, as well as control which directories and file types Tracker should index.
Notifications, to manage the notifications that will show up on your desktop. The filtering is done in gnome-shell itself, and would allow you to only show specific notifications in the lock screen for example. See Giovanni's post if your application uses notifications.
Privacy, which still requires quite a bit of work, would be the go-to place to ensure your identity isn't leaked on the network, or visible on your system. You can see how some of the features in the two aforementioned panels will also affect your privacy.

With the above panels merged, we're left with the re-design of the Power panel, which should mean the end of the "Screen & Brightness" panel (half of the settings went to the Privacy panel, the other half will go to the Power panel).

New backend features

First, users of Wacom tablets, you'll be happy to know there's now a button you can press to see, in an OSD, the current configuration of your tablet buttons. This feature has been long in the making, but the results are great. There's coverage for every tablet known to libwacom, and support for touchrings, touchstrips and modeswitch buttons. Select the button you want to use for the help in the Settings panel.

Secondly, we now support the draft "Idle Inhibition" specification from Freedesktop.org.

We also have some unfinished features.

The remote-display plugin will disable animations in the desktop when using the desktop over VNC or Spice.

And the cursor plugin will hide the mouse cursor until first used, or when using a touchscreen, similarly to what Windows 8 supports (it's the only system other than ours that supports both cursor pointers and touchscreens).

Those 2 plugins should hopefully be working by the time of the GNOME 3.8 release.

Until next time.

Arun Raghavan: PulseAudio 3.0

2012-12-18T07:57:46+00:00

Yay, we just released PulseAudio 3.0! I’m not going to rehash the changelog that you can find in the release announcement as well as the longer release notes.

I would like to thank the 36 contributors over the last 6 months who have made this release what it is and continue to demonstrate what a vibrant community we have!

Phil Normand: WebKitGTK+ Hackfest, 2012 edition

2012-12-15T15:12:37+00:00

There has been some media coverage of the hackfest already so I'll just focus on one area, Multimedia!

I think this fourth edition of the event was the first with so many hackers focusing on Multimedia:

Xabier and Zan worked on the media controls and finally landed the GStreamer 1.0 build support in our JHBuild moduleset so the buildbots now run the tests with a modern GStreamer setup
Jonathon focused on the Mediastream/WebRTC backend
Victor has been working on audio pitch preservation
yours truly dedicated most of his time on porting the WebAudio backend to GStreamer 1.0 and hunting two small bugs in the deinterleave and interleave elements

We also had a small session about the current status of Multimedia in WebKitGTK+ with a summary of what's been done recently, the work in progress and plans for the future. It's interesting because there is always a new W3C specification to implement and bugs to fix :)

A warm thank you to all the sponsors and organizers that made the event possible, it was really nice to meet new (and old) friends, hack on interesting tasks and keep pushing for a vibrant project such as WebKitGTK+ :)

Andy Wingo: corps: bespoke text codecs

Andy Wingo — 2012-12-12T16:57:56+00:00

Happy 12/12/12, peoples! In honor of repeated subsequences, today I'm happy to release a new set of compression tools, Corps.

Corps is a toolkit for generating custom text codecs, specialized to particular bodies of text. You give it an example corpus to analyze, and Corps can generate codecs based on what it finds.

For example, if you want to develop a compression algorithm that operates on JavaScript source text, you probably want to use a special code to represent the multi-character sequence function.

I say "probably", because in reality you don't know what substrings are most common. Corps uses the Re-Pair algorithm to build up an optimal token set. This algorithm treats all characters in the input as tokens, and recursively creates composite tokens from the most common pair of adjacent tokens, repeating the process until there are no more repeated pairs of tokens. For full details, see "Off-Line Dictionary-Based Compression" by Larsson and Moffat.

Corps is mostly useful for creating special-purpose fixed codecs based on a dictionary generated by its offline analysis. Although pre-generated codecs often do not compress as well as adaptive codecs like the one used by gzip, fixed codecs are typically faster as they can use the fixed code table to generate optimized encoders and decoders. Corps currently generates encoders in C and Scheme, and decoders in C, Scheme, and JavaScript.

Special-purpose codecs can also provide some interesting properties, such as the ability to decompress a substring of the original text.

get source

Corps is written for Guile version 2.0. See http://gnu.org/s/guile for more information on Guile and how to install it on your system.

To build Corps from git, do:

git clone git://gitorious.org/corps/corps.git
cd corps
./autogen.sh && ./configure && make && make check

You can install using make install, but it's often more convenient to just run corps uninstalled, using the env script.

stinky cheese

Corps includes a simple command-line tool called corps. For full documentation, run corps help. You can run it uninstalled by prefixing the ./env from the build tree.

As an example, say you want to build a database of wikipedia pages on cheese. Let's fetch a page:

$ curl -s 'http://en.wikipedia.org/wiki/Maroilles_(cheese)' > cheese
$ ls -l cheese
-rw-r--r-- 1 wingo wingo 43123 Dec 12 15:12 cheese

Now we analyze it to determine common substrings:

./env corps extract-all cheese > cheese-tokens

This generates a list of (string,frequency) pairs. An extract-all token set is usually quite large. We can pare it down to something manageable, the 500 most common substrings:

./env corps extract-subset cheese-tokens 500 cheese > 500-tokens

With this dictionary, we can huffman-code the page:

$ ./env corps encode -t 500-tokens -m huffman cheese cheese.huff
$ ls -l cheese.huff
-rw-r--r-- 1 wingo wingo 18060 Dec 12 16:09 cheese.huff

Well that's pretty cool -- it's less than half the size of the source text. We can also pare down this set of tokens to an appropriate number of tokens for a bytecode, and try doing a byte-encode of the file:

$ ./env corps extract-subset 500-tokens 254 cheese > 254-tokens
$ ./env corps encode -t 254-tokens cheese > cheese.bc
$ ls -l cheese.bc
-rw-r--r-- 1 wingo wingo 22260 Dec 12 16:19 cheese.bc

It's larger than the huffman-code, not only because of the smaller dictionary, but also because a bytecode is less dense. In practice though a bytecode is good enough while being very fast, so we'll continue with the bytecode.

Now let's generate a C encoder and decoder for this token set.

$ ./env corps generate-c-byte-encoder 254-tokens > encoder.inc.c
$ ./env corps generate-c-byte-decoder 254-tokens > decoder.inc.c
$ cp ~/src/corps/corps/decoder.c .
$ cp ~/src/corps/corps/encoder.c .
$ gcc -o decoder -O3 decoder.c
$ gcc -o encoder -O3 encoder.c
$ ls -l encoder decoder
-rwxr-xr-x 1 wingo wingo 13192 Dec 12 16:23 decoder
-rwxr-xr-x 1 wingo wingo 31048 Dec 12 16:23 encoder

Nice! We could use the corps tool to decode cheese.bc, but to vary things up we can use our zippy C decoder:

$ ./decoder < cheese.bc > cheese.out
$ cmp cheese.out cheese && echo 'excellent!'
excellent!

The performance of the C encoder is pretty good:

$ for ((x=0;x<1000;x++)) do cat cheese >> megacheese; done
$ time gzip -c < megacheese > megacheese.gz
real	0m1.418s
user	0m1.396s
sys	0m0.016s
$ time ./encoder < megacheese > megacheese.bc
real	0m0.523s
user	0m0.480s
sys	0m0.044s
$ ls -l megacheese*
-rw-r--r-- 1 wingo wingo 43123000 Dec 12 17:03 megacheese
-rw-r--r-- 1 wingo wingo 22370002 Dec 12 17:04 megacheese.bc
-rw-r--r-- 1 wingo wingo 11519311 Dec 12 17:04 megacheese.gz

Gzip gets better compression results for many reasons, but our quick-and-dirty bytecode compressor does well enough and is quite fast. The decoder is also quite good:

$ time ./decoder < megacheese.bc > /dev/null
real	0m0.179s
user	0m0.160s
sys	0m0.016s
$ time gunzip -c < megacheese.gz > /dev/null
real	0m0.294s
user	0m0.284s
sys	0m0.008s

Amusingly, for this text, gzipping the bytecoded file has quite an impact:

$ gzip -c < megacheese.bc > megacheese.bc.gz
$ ls -l megacheese*
-rw-r--r-- 1 wingo wingo 43123000 Dec 12 17:03 megacheese
-rw-r--r-- 1 wingo wingo 22370002 Dec 12 17:04 megacheese.bc
-rw-r--r-- 1 wingo wingo   175246 Dec 12 17:12 megacheese.bc.gz
-rw-r--r-- 1 wingo wingo 11519311 Dec 12 17:04 megacheese.gz

It so happens that byte-compressing the original text allows it to fit within the default gzip "window size" of 32 KB, letting gzip detect the thousandfold duplication of the source text. As a codec that works on bytes, gzip tends to work quite well on bytecoded files, and poorly on bit-coding schemes like huffman codes. A gzipped bytecoded file is usually smaller than a gzipped raw file and smaller than a gzipped huffman-coded file.

hack

I'll close with a link to the 254 most common substrings in one corpus of JavaScript code that I analyzed: here. I have a set of patches to integrate a codec optimized for JavaScript source into V8, to try to reduce the memory footprint of JS source code. More on that quixotic endeavor in some future post. Until then, happy scheming!

Andy Wingo: geneva

Andy Wingo — 2012-12-05T23:16:13+00:00

silence on the wire

It's been pretty quiet in this electro-rag, and for a simple but profound reason. I don't know about all of Maslow's hierarchy of needs, but for me there is definitely a hierarchy between housing and anything else.

Thing is, my partner and I drove up to Geneva almost three months ago, but we only moved in somewhere permanent last week. Everything was up in the air! Sublets and couches and moving vans and storage units. All the things in all the places.

In a way it should have been liberating, but I reckon I'm just a homebody: someone who likes to have a spritual and physical base to come back to. It's the classic introverted/extroverted "where do you get your energy" thing -- in private, or in society? Of course it's both, but for me the private side is important and necessary.

society, nations, states

Incidentally, October made ten years since I left the US. I lived in Namibia for a couple of years, moved to Spain in 2005, and as you know, left for Switzerland in September. In that time, a third of my life, I have not been able to vote for my local government. Representative democracy isn't very democratic, of course, but not even having that nominal connection to collective decision-making is a bit unmooring.

In Spain I have what is called "EU long-term residency". In 2003, the EU standardized visas for "third-country nationals": folks like me that do not have citizenship in a EU country. After 5 years, you automatically get country-specific "long-term residency", and can optionally apply for "EU long-term residency" (as I just did), which has the additional benefit of being transferable to other EU countries.

The idea of EU long-term residency was to make third-country nationals have the same rights as EU citizens, but several restrictions placed on the agreement by individual nations make it not very attractive compared to full nationality. EU citizens have the right to move around the EU, whereas third-country nationals have to apply to do so.

And in many countries, you can get nationality just as easy as long-term residency. In France for example, you only have to wait 5 years to be able to apply for nationality. In Spain, if you come from a Latin American country, you only have to wait 2 years (in theory; I have heard of unreasonable delays). As that is not my particular case, I would have to wait 10 years to get Spanish nationality, making this "EU long-term residency" attractive. However if I move away from Spain officially, I would have to "start over".

Man. The EU. What a bizarre institution. I think few normal citizens can describe the structure of the the EU -- you know, what's the deal with the parliament and the commisioners and the president and the central bank? The operations of the EU have very little to do with democracy. The tenuous accountability of the Members of European Parliament is mostly to political parties rather than to the people. Finance in rich countries basically runs the bank. The commission seems to run itself, with limited accountability to the governments of member states. To the extent that the EU is a democracy, it is its weakest form: vast, physically removed from its constituency, composed of representatives of representatives.

and yet

It's been interesting to see the contrast in Switzerland. Switzerland has its own currency, protectionist trade policies, and a smattering of EU-à-la-carte. So, yes, it is relatively easy for EU citizens to move to Switzerland, but on the other hand it's quite difficult if you are a "third-country national" like myself.

Switzerland is also very local. Most legislation involves the citizens directly, via referendum. The other day, voters here just approved a new constitution, by simple majority, for Geneva. Contrast this to the EU, which centralizes its power more every year.

There are good things and bad things about this. I suppose that specifically, it's mostly good for the Swiss, neutral for EU nationals, and worse for third-country nationals. Policies that enable local agricultural and industrial production are great for local workers, businesses, consumers (with Swiss salary), and the environment. These policies are quite difficult to have in the EU. Even the existence of the Swiss franc is great for local decision-making power, although its policies are mostly controlled by finance.

On the other hand, the immigration policy is quite reactionary, often bordering on xenophobia. In the EU, the institutions were able to achieve some limited forms of freedom of movement of persons, including third-country nationals, in exchange for the freedom of movement of capital. No such arrangement has been made in Switzerland. (They'll happily take your capital, of course.)

As you probably know, I mostly identify as an anarchist, a kind of libertarian socialism that is suspicious of hierarchical power structures. So Switzerland has its attractions in that sense. But I've had discussions with folks arguing that the EU has actually been the emancipatory institution, in contrast to reactionary governments, and there is something there. For all its decentralized, democratic principles, women did not get the right to vote in Swiss federal elections until 1971, and the last canton (local government) held out until 1990, when it was forced to allow women to vote by a federal court.

capital

There's no way around it: Geneva stinks of money.

An illustration, if you don't mind. When we first came here to visit in August, we got out of the car, went to a cash machine, and tried to get out 120 francs (approximately 120 dollars, or 100 euros). No. The minimum amount was 100 francs, and the next was 200. Well, OK. We got 200. It came out in two bills. The machine did not dispense any lower denomination. There was nothing wrong with that machine; it was marked as only dispensing 100-franc bills and higher.

On the other hand, unlike Spain, all shopkeepers carry around fat bankrolls and are quite happy to break a 100, even on a 50 cent purchase. You can do that only if you can find something that costs that little, of course.

If you walk around the center of town, it's designer tailors, caviar bars (not kidding), and private banks. It is truly disgusting. It makes me think of a butcher's counter: bleach covering a faint smell of blood, but without the redeeming earthiness. Perhaps this is just a personal analogy though.

That's the macro-level. Of course there are normal folks here too, but that's tempered by the cost: things are in general really expensive here. Starbucks drip coffee for 5 dollars. You can easily spend 70 dollars a person at a normal restaurant. A normal one-hour tram ticket is 5 dollars. Etc.

From what I can tell, the cost of things is not a problem if you have a Swiss salary. I'm in something of an irregular state: an American with Spanish residency, working for a Spanish company, living in Geneva (but actually France). Administrativia aside, it has been quite a shock coming from Spain to here. In Igalia, the cooperative I work at, we try to pay everyone the same when everything is going well, but when the budget is a little tighter salaries get scaled down to something approaching "the cost of living" (whatever that is).

We had to open a whole new round of discussions about how to determine compensation after I moved here. Geneva just didn't fit in anyone's conception about what was reasonable! It raises all kinds of issues: what does it mean to work in a cooperative with people in all different kinds of places? What does fair and equal compensation actually mean in San Francisco and A Coruña and Madrid and Geneva, and how do you calculate it? It looks like we've come to an interesting and probably unique solution there, so perhaps more on that as discussions progress.

borders

Of course it's strange to come to this capital of, well, capital with these values, but here I am. You end up talking about money a lot. Of course you need a place to put your money and take it out. For an anarchist I've got a lot of bank accounts: Spanish, Swiss, French, and US somewhere... For all the new EU regulations, cross-border ATM fees still make it attractive to have an account in the place where you need to withdraw money.

Same thing with mobile phone companies. As I said, we ended up moving to France. The rent is a lot cheaper, it's more compatible with our residency permits, and it's still only a 25 minute bike or tram into the center of Geneva so it's not totally in the boon-docks. But this makes me carry around two phones because I cross the border all the time, and then of course there's the old Spanish SIM I need to do something with. I've done the PIN-to-PUK dance multiple times, because I can't remember so many numbers.

It's a pretty strange identity to have: to have a house in France, but feel attached to Switzerland. It's tough to catch the dominant social story, of who is the "us" in this place. It's problem with Geneva in general, transient city that it is.

greenery

Well, this post grows long, and it's mostly a rant, right? And I needed to rant and be done with it. But I don't want to be too negative. We have an actual house, with a garden, and we're going to plant things and compost and such. It's warm and cozy inside and there are snowy mountains about, and there's a cosmopolitan city within a close if brisk bike ride. I have a French grocery store five minute's walk away. It's a dark season, but there is cheese and wine enough to carry me through to springtime :) So things are good. I'll still rant, but things are all right.

Bon. Catch you internauts later. Next time, with code!

Zeeshan Ali: Boxes video tutorial

2012-11-23T17:05:56+00:00

I have been meaning to do this for a while and now I finally managed to do it: Here you'll find tutorial on Boxes. I know the sound quality and volume isn't good and there is other mistakes/issues but I'm not too ashamed as this is the first time I've done this and tools that I needed/wanted were all breaking on my Fedora 18 laptop.

UPDATE: I uploaded an updated version of this tutorial with slightly improved audio. To improve it further, I'll have to re-record the audio or the whole video but right now I don't have time for that. Perhaps in a few weeks..

Felipe Contreras: FelipeC

2012-11-13T02:28:17+00:00

I’ve been involved in the git project for some time now, and through the years I’ve had to work with other DSCMs, like mercurial and monotone. Not really as a user, but investigating the inner workings of them, mostly for conversion purposes, and I’ve written tools that nobody used, like mtn2git, or pidgin-git-import, but eventually I decided; why not write something that everybody can use?

So, I present to you git-remote-hg, and git-remote-bzr. What do they do?

git clone "hg::http://selenic.com/repo/hello"
git clone "bzr::lp:gnuhello"

That’s right, you can clone both mercurial and bazaar repositories now with little to no effort.

The original repository will be tracked like any git repository:

% git remote show origin
* remote origin
  Fetch URL: hg::http://selenic.com/repo/hello
  Push  URL: hg::http://selenic.com/repo/hello
  HEAD branch: master
  Remote branches:hg and mercurial. 
    branches/default tracked
    master           tracked
  Local branch configured for 'git pull':
    master merges with remote master
  Local ref configured for 'git push':
    master pushes to master (create)

You can pull, and in fact push as well. Actually, you wouldn’t even notice that this remote is not a git repository. You can create and push new branches… everything.

You might be thinking that this code being so new might have many issues, and you might screw up a mercurial repository. While that is true to some extent, there’s already a project that tries to act as a bridge between mercurial and git: hg-git. This project has a long history, and a lot of people use it successfully. So, I took their test framework, cleaned it up, and used to test my git-remote-hg code. Eventually all the tests passed, and I cloned big mercurial repositories without any issue, and the result were exact replicas, and I know that because the SHA-1′s were the same

So you can even keep working on top of clones you have created with hg-git, and switch back and forth between git-remote-hg and hg-git as you wish. The advantage over hg-git, is that you don’t need to use the mercurial interface at all, you can use git

To enable the hg-git compatibility mode you would need this:

% git config --global remote-hg.hg-git-compat true

What about the others?

Surely I must not be the first one try something this cool, right? I don’t want to dwell into all the details why none of the other tools suited my needs, but let’s give a quick look:

hg-git

It’s for mercurial, not git. So you need to use it through the mercurial UI.

fast-export

This is a very nice tool, but only allows you to export stuff (fetch an hg repo), and you have to manually run the ‘git fast-import’ command, setup the marks, and so on. Also, it’s not very well maintained.

Another git-remote-hg

It needs hg-git.

Yet another git-remote-hg

You need quite a lot of patches on top of git, so your vanilla version of git is not going to cut it. In addition to that it doesn’t support as many features; no tags, no bookmarks. Plus it fails all my extensive tests for git-remote-hg.

git-hg

It needs separate tools, such as ‘hg convert’, and fast-export, and it doesn’t use remote helper infrastructure, so it’s not transparent: you have to run git-hg clone, git-hg fetch, and so on.

Another git-hg

It needs hg-git, and doesn’t use git’s remote helper infrastructure either: you have to do git hg-clone, git hg-pull, etc.

There might be others, but you get the point.

The situation in bazaar is not much better, but I’m not going to bother listing the tools.

More coolness

In fact, you can push and pull from and to mercurial and bazaar

% git remote -v
bzr	bzr::lp:~felipec/+junk/test (fetch)
bzr	bzr::lp:~felipec/+junk/test (push)
gh	gh:felipec/test.git (fetch)
gh	gh:felipec/test.git (push)
hg	hg::bb+ssh://felipec/test (fetch)
hg	hg::bb+ssh://felipec/test (push)

Isn’t that cool?

So, are there any drawbacks? Surely some information must be lost.

Nope, not really. At least not when using the hg-git compat mode, because all the extra information is saved in the commit messages.

Update: I forgot to mention the only bidirectionality problem: octopus merges. In git a merge can have any number of parent commits, but in mercurial only two are allowed. So you might have problem converting from a git repository to a mercurial one. It’s the only feature that hg-git has, that git-remote-hg doesn’t. As for bazaar, it also allows multiple parents, so it should work fine, but this hasn’t been thoroughly tested.

The only consideration is that in the case of mercurial branches are quite different from git branches, so you need to be really careful to get the behavior you want, which is why it’s best to just ignore mercurial branches, and use bookmarks instead. When you create git branches and push them to mercurial, bookmarks will be created on whatever branch is current.

Trying it out

Just copy them files (git-remote-hg, git-remote-bzr) to any location available in your $PATH (.e.g ~/bin), and start cloning

Soon they might be merged into upstream git, so they would be distributed as other contrib scripts (e.g. /usr/share/git), and eventually possibly enabled by default.

Enjoy!

Stefan Kost: 11 Nov 2012

2012-11-11T21:07:36+00:00

buzztard

Finally pushed the pattern control-source: 26 files changed, 1204 insertions, 1333 deletions. It looks like only a little gain in code size, but that is not true; we added more tests and splitting up a large class always comes with some boilerplate for the new class. There are still some opportunities for optimizations, which I'll probably try some soon.

In the UI I improved the interaction controller workflow. For controllers that implement control discovery the dialog from the context menu will also bind the new controller. The settings are now finally useful, as there one can do mass-learning (tweak all knows and then go through the list and name them).

Writing synth plugins for buzztard would be quite a bit of copy and paste. I chopped simsyn into pieces as reusable objects in libgst-buzztard. We have a few oscillators, envelope objects and the svf-filter. The gst elements would proxy some properties of the synth component. Unfortunately that means copy'n'paste of the property setup. I've tried this code, but it shows no effect :/

     GObjectClass * filter_klass = g_type_class_ref (GSTBT_TYPE_FILTER_SVF);

     g_object_class_install_property (gobject_class, PROP_RESONANCE,

        g_param_spec_override ("resonance", 

          g_object_class_find_property (filter_klass, "resonance")));

     g_type_class_unref (filter_klass);

A full standalone example can be found at here.

We also have a new element called wavereplay. This is mainly for testing the wavetable oscillator, but also has a nice property compared to tracker elements. It only has parameters to select the wave to play and it will play it from start to finish in syn with the song. That is if you seek to 2:00, the wave will play content from 2:00. This is unlike tracker elements that would only play what gets triggered from the new time position. One practical use is doing a mixdown of the music rendering with the vocal recordings. In such a song you load two wave (song, vocals), use two wavereplay elements, add effect elements to the one for the vocals and re-render the song to create the mixdown.

I thought I knew pretty much all about gobject. This month I discovered something old I did not know about. It is supported to calls g_object_class_install_property() for a property inherited from the baseclass to e.g. narrow down the value range. This is very useful for the "children" property of polyphonic elements (where children is the number of voices). The interface does only limit the property to the full int range. With this feature elements like e.g. sidsyn can override the range to 3...3 (sid has always 3 voices).

The last bigger change this month was a refactoring cleanup in sequence and song-info. This was quite a big of code shuffling, but now all the timing info is in song-info (which already has bpm an stuff). This change helped with an optimization in the pattern control-sources.

132 files changed, 2417 insertions(+), 2138 deletions(-)

Bastien Nocera: Jobs change

Bastien Nocera — 2012-11-09T20:18:28+00:00

Seeing as everyone is making job-related posts, I thought I'd join in the fun.

After 10 years of service, I've left Red Hat UK on the 31st of October to join Red Hat France on the 1st of November.

I now live in the birthplace of cinema and of French gastronomy, so poke me if you want to talk GNOME or Linux over beers. I've already found the English pub.

Arun Raghavan: PulseConf 2012: Report

2012-11-06T11:04:22+00:00

For those of you who missed my previous updates, we recently organised a PulseAudio miniconference in Copenhagen, Denmark last week. The organisation of all this was spearheaded by ALSA and PulseAudio hacker, David Henningsson. The good folks organising the Ubuntu Developer Summit / Linaro Connect were kind enough to allow us to colocate this event. A big thanks to both of them for making this possible!

The room where the first PulseAudio conference took place

The conference was attended by the four current active PulseAudio developers: Colin Guthrie, Tanu Kaskinen, David Henningsson, and myself. We were joined by long-time contributors Janos Kovacs and Jaska Uimonen from Intel, Luke Yelavich, Conor Curran and Michał Sawicz.

We started the conference at around 9:30 am on November 2nd, and actually managed to keep to the final schedule(!), so I’m going to break this report down into sub-topics for each item which will hopefully make for easier reading than an essay. I’ve also put up some photos from the conference on the Google+ event.

Mission and Vision

We started off with a broad topic — what each of our personal visions/goals for the project are. Interestingly, two main themes emerged: having the most seamless desktop user experience possible, and making sure we are well-suited to the embedded world.

Most of us expressed interest in making sure that users of various desktops had a smooth, hassle-free audio experience. In the ideal case, they would never need to find out what PulseAudio is!

Orthogonally, a number of us are also very interested in making PulseAudio a strong contender in the embedded space (mobile phones, tablets, set top boxes, cars, and so forth). While we already find PulseAudio being used in some of these, there are areas where we can do better (more in later topics).

There was some reservation expressed about other, less-used features such as network playback being ignored because of this focus. The conclusion after some discussion was that this would not be the case, as a number of embedded use-cases do make use of these and other “fringe” features.

Increasing patch bandwidth

Contributors to PulseAudio will be aware that our patch queue has been growing for the last few months due to lack of developer time. We discussed several ways to deal with this problem, the most promising of which was a periodic triage meeting.

We will be setting up a rotating schedule where each of us will organise a meeting every 2 weeks (the period might change as we implement things) where we can go over outstanding patches and hopefully clear backlog. Colin has agreed to set up the first of these.

Routing infrastructure

Next on the agenda was a presentation by Janos Kovacs about the work they’ve been doing at Intel with enhancing the PulseAudio’s routing infrastructure. These are being built from the perspective of IVI systems (i.e., cars) which typically have fairly complex use cases involving multiple concurrent devices and users. The slides for the talk will be put up here shortly (edit: slides are now available).

The talk was mingled with a Q&A type discussion with Janos and Jaska. The first item of discussion was consolidating Colin’s priority-based routing ideas into the proposed infrastructure. The broad thinking was that the ideas were broadly compatible and should be implementable in the new model.

There was also some discussion on merging the module-combine-sink functionality into PulseAudio’s core, in order to make 1:N routing easier. Some alternatives using te module-filter-* were proposed. Further discussion will likely be required before this is resolved.

The next steps for this work are for Jaska and Janos to break up the code into smaller logical bits so that we can start to review the concepts and code in detail and work towards eventually merging as much as makes sense upstream.

Low latency

This session was taken up against the background of improving latency for games on the desktop (although it does have other applications). The indicated required latency for games was given as 16 ms (corresponding to a frame rate of 60 fps). A number of ideas to deal with the problem were brought up.

Firstly, it was suggested that the maxlength buffer attribute when setting up streams could be used to signal a hard limit on stream latency — the client signals that it will prefer an underrun, over a latency above maxlength.

Another long-standing item was to investigate the cause of underruns as we lower latency on the stream — David has already begun taking this up on the LKML.

Finally, another long-standing issue is the buffer attribute adjustment done during stream setup. This is not very well-suited to low-latency applications. David and I will be looking at this in coming days.

Merging per-user and system modes

Tanu led the topic of finding a way to deal with use-cases such as mpd or multi-user systems, where access to the PulseAudio daemon of the active user by another user might be desired. Multiple suggestions were put forward, though a definite conclusion was not reached, as further thought is required.

Tanu’s suggestion was a split between a per-user daemon to manage tasks such as per-user configuration, and a system-wide daemon to manage the actual audio resources. The rationale being that the hardware itself is a common resource and could be handled by a non-user-specific daemon instance. This approach has the advantage of having a single entity in charge of the hardware, which keeps a part of the implementation simpler. The disadvantage is that we will either sacrifice security (arbitrary users can “eavesdrop” using the machine’s mic), or security infrastructure will need to be added to decide what users are allowed what access.

I suggested that since these are broadly fringe use-cases, we should document how users can configure the system by hand for these purposes, the crux of the argument being that our architecture should be dictated by the main use-cases, and not the ancillary ones. The disadvantage of this approach is, of course, that configuration is harder for the minority that wishes multi-user access to the hardware.

Colin suggested a mechanism for users to be able to request access from an “active” PulseAudio daemon, which could trigger approval by the corresponding “active” user. The communication mechanism could be the D-Bus system bus between user daemons, and Ștefan Săftescu’s Google Summer of Code work to allow desktop notifications to be triggered from PulseAudio could be used to get to request authorisation.

David suggested that we could use the per-user/system-wide split, modified somewhat to introduce the concept of a “system-wide” card. This would be a device that is configured as being available to the whole system, and thus explicitly marked as not having any privacy guarantees.

In both the above cases, discussion continued about deciding how the access control would be handled, and this remains open.

We will be continuing to look at this problem until consensus emerges.

Improving (laptop) surround sound

The next topic dealt with being able to deal with laptops with a built-in 2.1 channel set up. The background of this is that there are a number of laptops with stereo speakers and a subwoofer. These are usually used as stereo devices with the subwoofer implicitly being fed data by the audio controller in some hardware-dependent way.

The possibility of exposing this hardware more accurately was discussed. Some investigation is required to see how things are currently exposed for various hardware (my MacBook Pro exposes the subwoofer as a surround control, for example). We need to deal with correctly exposing the hardware at the ALSA layer, and then using that correctly in PulseAudio profiles.

This led to a discussion of how we could handle profiles for these. Ideally, we would have a stereo profile with the hardware dealing with upmixing, and a 2.1 profile that would be automatically triggered when a stream with an LFE channel was presented. This is a general problem while dealing with surround output on HDMI as well, and needs further thought as it complicates routing.

Testing

I gave a rousing speech about writing more tests using some of the new improvements to our testing framework. Much cheering and acknowledgement ensued.

Ed.: some literary liberties might have been taken in this section

Unified cross-distribution ALSA configuration

I missed a large part of this unfortunately, but the crux if the discussion was around unifying cross-distribution sound configuration for those who wish to disable PulseAudio.

Base volumes

The next topic we took up was base volumes, and whether they are useful to most end users. For those unfamiliar with the concept, we sometimes see sinks/sources where which support volume controls going to > 0dB (which is the no=attenuation point). We provide the maximum allowed gain in ALSA as the maximum volume, and suggest that UIs show a marker for the base volume.

It was felt that this concept was irrelevant, and probably confusing to most end users, and that we suggest that UIs do not show this information any more.

Relatedly, it was decided that having a per-port maximum volume configuration would be useful, so as to allow users to deal with hardware where the output might get too loud.

Devices with dynamic capabilities (HDMI)

Our next topic of discussion was finding a way to deal with devices such as those HDMI ports where the capabilities of the device could change at run time (for example, when you plug out a monitor and plug in a home theater receiver).

A few ideas to deal with this were discussed, and the best one seemed to be David’s proposal to always have a separate card for each HDMI device. The addition of dynamic profiles could then be exploited to only make profiles available when an actual device is plugged in (and conversely removed when the device is plugged out).

Splitting of configuration

It was suggested that we could split our current configuration files into three categories: core, policy and hardware adaptation. This was met with approval all-around, and the pre-existing ability to read configuration from subdirectories could be reused.

Another feature that was desired was the ability to ship multiple configurations for different hardware adaptations with a single package and have the correct one selected based on the hardware being run on. We did not know of a standard, architecture-independent way to determine hardware adaptation, so it was felt that the first step toward solving this problem would be to find or create such a mechanism. This could either then be used to set up configuration correctly in early boot, or by PulseAudio for do runtime configuration selection.

Relatedly, moving all distributed configuration to /usr/share/..., with overrides in /etc/pulse/... and $HOME were suggested.

Better drain/underrun reporting

David volunteered to implement a per-sink-input timer for accurately determining when drain was completed, rather than waiting for the period of the entire buffer as we currently do. Unsurprisingly, no objections were raised to this solution to the long-standing issue.

In a similar vein, redefining the underflow event to mean a real device underflow (rather than the client-side buffer running empty) was suggested. After some discussion, we agreed that a separate event for device underruns would likely be better.

Beer

We called it a day at this point and dispersed beer-wards.

Our valiant attendees after a day of plotting the future of PulseAudio

User experience

David very kindly invited us to spend a day after the conference hacking at his house in Lund, Sweden, just a short hop away from Copenhagen. We spent a short while in the morning talking about one last item on the agenda — helping to build a more seamless user experience. The idea was to figure out some tools to help users with problems quickly converge on what problem they might be facing (or help developers do the same). We looked at the Ubuntu apport audio debugging tool that David has written, and will try to adopt it for more general use across distributions.

Hacking

The rest of the day was spent in more discussions on topics from the previous day, poring over code for some specific problems, and rolling out the first release candidate for the upcoming 3.0 release.

And cut!

I am very happy that this conference happened, and am looking forward to being able to do it again next year. As you can see from the length of this post, there are lot of things happening in this part of the stack, and lots more yet to come. It was excellent meeting all the fellow PulseAudio hackers, and my thanks to all of them for making it.

Finally, I wouldn’t be sitting here writing this report without support from Collabora, who sponsored my travel to the conference, so it’s fitting that I end this with a shout-out to them. :)

Aaron Bockover: From Rdio to Xamarin

2012-10-24T18:40:15+00:00

Last week I gave a talk on Vernacular at MonkeySpace in Boston. As Wednesday concluded, so did my last day at Rdio after nearly two years. In that time I had the pleasure of working with very bright people on creating the best streaming music service available on Mobile, Web, and Desktop.

From the introduction of the Windows Phone app, to the launch of the new Android app, Rdio 2.0 for Web and Desktop, and finally the Android and iOS refresh betas, Rdio has been and will always be close to my heart. I wish everyone at Rdio the best -- keep on being excellent!

I originally intended to simply take a few weeks of vacation time after leaving Rdio to focus on Cat Overflow, beer drinking, and cycling (funemployment!), but after spending a long weekend in Florida for a wedding, I find myself headed back to Boston. A simple Monday and Tuesday at home in Raleigh is all I’ve afforded myself in terms of vacation, since today I start at Xamarin!

While I’m determined to actually take some real vacation in the near future, I was enticed to join Xamarin as soon as possible to begin bootstrapping work on an exciting new product. I'm looking forward to working with new and familiar faces!

Perhaps I’ll start blogging again as well, after nearly two years of silence. I certainly owe a follow-up post about Vernacular. For starters, here are slides from my talk last week, cats included. Video of the talk should be up at MonkeySpace soon as well.

Felipe Contreras: FelipeC

2012-10-14T01:30:27+00:00

We’ve all have noticed that when looking back at code you wrote years ago, you realize how much you have progressed–or how much you sucked in the past, depending on your perspective–so it’s always a fair assumption to say, that in the future you will look back at the code you are working now, and feel ashamed. Perhaps this becomes logarithmically less of an issue as time goes on, but at least I still experience this.

I want to share what I think is a good example of this. Ruby code is particularly special in this respect since there’s always tons of ways to write code that do exactly the same.

Back in 2007 when I started msn-pecan, I wanted a script to create a ChangeLog, not because I thought it was needed, but mainly to shut up the people that said the git log wasn’t good enough, and that you still needed to update the ChangeLog in every commit. Few of those people remain today, but my script is still here.

The objective was to output something like this:

== Release 0.0.2 ==

2007-11-30	Felipe Contreras 
	* Add readme.
	* Update copyright stuff.
	* Fix to allow aliases with special characters.

== Release 0.0.1 ==

2007-11-29	Felipe Contreras 
	* Fix warnings.
	* Add display name support.

2007-11-28	Felipe Contreras 
	* Add .gitignore file.
	* Update info.
	* Fix compilation.
	* Stuff required to compile.
	* Initial import.

For some reason I was forced to look at it:

#!/usr/bin/env ruby

require "stringio"

revs=`git rev-list master`
refs=`git for-each-ref --format="%(objectname) %(refname)" refs/tags`

commits = revs.map { |e| e.chomp }

tags = {}

list = {}
list_order = []
$tag = ""

refs.each do |l|
  tag, name = l.chomp.split(" ")
  name = name.slice(/refs\/tags\/v(.*)/, 1)
  tags[tag] = name
end

commits.each do |e|
  commit = StringIO.new(`git show --pretty="format:%an ::%ai::%s\n%b" --quiet #{e}`)
  author, date, subject = commit.readline.chomp.split("::")
  date = date.split(" ")[0]
  if tags[e]
    $tag = tags[e]
    list_order << {:id => $tag}
  end
  id = "#{date}\t#{author}"
  uid = "#{date}\t#{author}\t#{$tag}"
  if not list[uid]
    list[uid] = []
    list_order << {:id => id, :value => list[uid]}
  end
  list[uid] << subject
end

list_order.each do |i|
  id = i[:id]
  value = i[:value]

  if value
    puts "#{id}"
    puts value.map { |e| "\t* #{e}" }.join("\n")
    puts "\n"
  else
    puts "== Release #{id} ==\n\n"
  end
end

If you are familiar with Ruby, you would see that there’s tons of things that don’t make sense there, unnecessary complexity, simpler idioms, etc. So I decided to clean it up:

#!/usr/bin/env ruby

require 'date'

tags = {}
tag = nil
commits = []

class Commit
  attr_reader :author, :date, :tag

  def initialize(id, subject, author, date, tag)
    @id = id
    @author = author
    @date = Date.parse(date)
    @subject = subject
    @tag = tag[1..-1] if tag
  end

  def to_s
    return @subject
  end

end

open("|git for-each-ref --format='%(objectname) %(refname:short)' refs/tags").each do |l|
  commit, id = l.chomp.split(" ")
  tags[commit] = id
end

open("|git log --oneline --format='%H%x00%an %x00%ai%x00%s'").each do |l|
  commit, author, date, subject = l.chomp.split("")
  tag = tags[commit] if tags[commit]
  commits << Commit.new(commit, subject, author, date, tag)
end

commits.group_by(&:tag).each do |tag, commits|
    puts "== Release #{tag} ==\n\n"
    commits.group_by(&:date).sort.reverse.each do |date, commits|
      commits.group_by(&:author).each do |author, commits|
        puts "#{date}\t#{author}\n"
        puts commits.map { |e| "\t* #{e}" }.join("\n")
        puts "\n"
      end
    end
end

So we went from code that took more than 10s to complete, to one that barely took 100ms, it’s much easier to read, smaller, and it’s actually more correct (there was a bug in sorted dates). Not bad!

I was particularly surprised by the method group_by, which takes any Enumeratable and does what the name says:

[
  {:id => 0, :name => 'foo'},
  {:id => 0, :name => 'bar'},
  {:id => 1, :name => 'roo'} ].group_by { |e| e[:id] }
=> {
  0=>[{:id=>0, :name=>"foo"}, {:id=>0, :name=>"bar"}],
  1=>[{:id=>1, :name=>"roo"}] }

And then, in the functional tradition, you can sort, reverse, etc. the result.

Moreover, instead of getting the full output of a command with `git command`, it’s much better to pipe it with open("|git command"), something obvious, but a bit tedious to do in other languages.

The rest is git magick–I’ve been progressing on that front as well

Did I mention I love ruby?

Stefan Kost: 1 Oct 2012

2012-10-01T18:18:34+00:00

buzztard

After a discussion on IRC about tunings I looked into adding some of those to the ToneConversion class which so far only implemented the usual equal temperament. I have to say the wikipedia articles on this matter are awesome, detailed and plentiful. I added two 12tone tunings to the ToneConversion class now and implemented tuning in sidsyn and simsyn.

I also had a look at the TODO comments and planned items on the wiki. When I was writing the list model classes I did not wrote ones for the wavetable and wave-level-list. I did this now. The list models can also be tested separately from the UI (which is harder to test). I wrote basic tests for all model classes which bumps the test coverage in the ui by 10% (61% classes have tests now, 105 ui tests).

IRC was quite active this month. Another chat was on the wave-table page and wave-preview. The workflow was indeed needless complicated. Now preview is a toggle and if it is active, moving the selection in the file-browser will play sounds. Other discussion motivated me to implement audio-device selection in the settings. So far one could only select the protocol. Now for e.g. alsa or pulseaudio one can also choose to which device to play the audio. While working on the settings I started to make the interaction controller setting more useful. So far the only use of this settings-tab was to list the detected settings. I added a couple of messages to help the user to know what to do when e.g. the list of devices is empty. And I started to implement batch-learning of controls to the settings.

84 files changed, 4199 insertions(+), 1653 deletions(-)

Gustavo Orrillo: ac

2012-09-24T14:50:43+00:00

Processing 2.0 beta was finally released a couple of weeks ago, with the 2.0b3 version being the latest release in the beta series at this time. The new shader API described in the previous posts (1, 2, and 3) received some tweaking and cleaning-up, so the examples mentioned earlier might not work properly. Check below […]

Michael Sheldon: Firestarter now called Erudite and version 1.2 released

2012-09-12T15:58:08+00:00

Overview

Erudite makes it possible to use Amazon’s Cloud Reader service under MeeGo. You can either read books online, or download them for reading offline. Your progress in a book is kept synchronised between your N9 and other Kindle devices.

Name change

Unfortunately since Amazon own a wide range of “Fire” themed trademarks, and the application’s operation is heavily related to the same areas that these trademarks are used in, Amazon couldn’t really let me carry on using the name “Firestarter”. It was entirely my own foolishness that caused this hassle by going for a “clever” name without even considering the potential trademark implications, thankfully the lawyer who got in touch with me was very understanding and patiently helped me work through the various issues. I’m just glad that Amazon handled everything so amicably, rather than wading in with heavy-handed legal threats as some companies might.

So from now on the application is called “Erudite”, a name that’s clever in a different way.

New features

Added in 1.2:

The application name has now changed to Erudite.
A number of buttons and menu entries have been made larger and easier to press.
You can now switch in to and out of fullscreen mode by pressing a toggle button in the bottom right hand corner of the screen.

Added in 1.1:

The application can now start up completely offline. You still need to be online for the very first usage and to download books, but once you’ve downloaded some books you can now always access them offline.
A progress indicator shows when the application is loading.
Keyboard bug is fixed. Previously you needed to tap on text fields twice to bring up the virtual keyboard, now it pops up on the first tap.

Download

Available for the Nokia N9 and N950 in the Ovi Store: http://store.ovi.com/content/307241

Stefan Kost: 11 Sep 2012

2012-09-11T20:11:37+00:00

buzztard: presets, tests & live playback

I worked a bit on preset handling in gstreamer. Now machines with variable number of parameters with save their properties correctly. One example is the parametric equalizer. I needed this for sidsyn in buzztard too. I also updated the preset handling for buzzmachines. They now store presets in user-data-dir. This avoids that those get overwritten when the plugins get updated. The preset handling logic will read user-presets first and then merge in system wide presets. One shortcoming here is that we would need a flag on each preset whether it comes from a preset file in user-data-dir (and thus is writable) or comes from a system dir (and thus is read-only). This would allow the UI to require that users pick a new uniqe name when changing a system preset.

Songs in buzztard now also store their non-dynamic settings. This will fix fluidsynth forgetting the selected patch file and is also needed for e.g. setting the chip-type in sidsyn. I refactored the code a bit and now we don't show empty properties dialogs anymore and instead gray out the menu item that would invoke them.

The build system now can show the class-coverage of the tests. As this is just a few shell commands it quickly shows untested classes. It relies on naming conventions for tests. To make that work I also cleaned up the whole UI tests. Those are now more robust and actually verify a lot more things (we went from 54 to 92 ui tests).

77 % (24/31) lib/core/
40 % ( 4/10) lib/ic
100 % ( 1/ 1) ui/cmd
51 % (27/52) ui/edit

The last thing I worked was to progress on live machine playback. And hell yes, it works! One can bind e.g. midi keyboards to machines and play them live. This includes a round-robin voice management for polyphonic generators. It is a lot of fun to play with.

Tom made good progress on the pattern control source. Most of the tests now pass. We hope to be able to merge it soon and to remove the complicated 2step controller updating code.

151 files changed, 8453 insertions(+), 6451 deletions(-)

Mathieu Duponchelle: KSE 0.1 released !

Mathieu Duponchelle — 2012-09-03T15:07:50+00:00

I just finished fixing the blocker bugs for the open source assets' manager KSE, and I pushed a release there : https://github.com/MathieuDuponchelle/Kerious-Resource-Editor on the branch 0.1.

I recorded this screencast to make a demo of the features included in this release.

I want to say thank you to Simon Corsin for his work on the windows port and manual management.

I'd be glad to see pull-requests on github, or comments, critics and feature requests here.

Here is the text of the video for those with no audio:

Hi, my name is Mathieu Duponchelle and I am glad to present you the first release of kse, which is
a tool to simplify asset's management written in python.

This release covers spritesheet creation, sprite naming, automatic sprites detection in an existing spritesheet,
animation referencing and tons of other stuff.

Spritesheet creation.

Let's create a new spritesheet, and start importing resources.
First we create what we call an "atlas", specify its width and height and the location of the resulting file.
Then we can specify the size we wish the next sprite to have, and start adding sprites by double-clicking.
Next step is to start naming all these sprites, for now the naming part is a bit tedious, but it works.

Animation referencing.
Another cool feature of KSE is that we can reference animations easily, using multiple selection.
Let's say these sprites belong to the same animation.

We now save the project, and have a look at the xml that was produced.

We can see that a simple xml parser could easily extract coordinates for a given name, here X.

Automatic sprite detection.

Let's now say that we want to use an already existing spritesheet. I have this one, which will make for a good
demonstration.
A really cool feature is automatic sprite detection, which can be configured with this button.
Match-size will specify the desired width and height multiples for the resulting sprites.
Now let's try it out !
As you can see, KSE detected the sprites and automatically highlighted them. However, if the position or size
of any sprite does not match what we intend, we can easily resize and move them using the mouse or, if we want
a more fine-grained control the comboboxes that we have here.

Conclusion.

With some friends of mine, we already developed a space shooter using KSE.
This helped me finding out the features that were most needed.
It can greatly improve the collaboration between designers and developers, by reducing the number of back and forth
trial and errors, allowing the programmer to only parse the xml file looking for the sprite or the animation name,
and thus not having to hardcode coordinates, which is an obvious advantage, and allowing the designer to try things
without having to modify the code or asking the programmer to do so.

KSE now works on both linux and Windows, it has not been tested on OSX yet.

My thoughts for future features are listed in the poll on the right-hand side of the blog, don't hesitate to comment if you have other ideas/remarks/critics. Thanks for watching !

Thiago Sousa Santos: Cheese live from GStreamer Conference 2012

thiagoss — 2012-08-28T18:57:11+00:00

Greetings from San Diego!

Had a good time at the GStreamer Conference, seeing folks from Collabora, chatting, having a few beers, watching talks and some hacking. Good days :)

That's not what I want to talk about. I just want to announce this picture:

It is (likely) the first picture taken with Cheese using GStreamer 1.0 :)

Here's a fast wrap up on the changes:

Camerabin2 is now mostly ported, just a annoying bug left on video recording.
I believe most video effects were already ported some months ago (and they work, see the picture ^)
Very easy to port Cheese to 1.0. Seriously, application's API hasn't changed much.
There will be bugs and there are still some critical asserts being printed, but the hardest part is over now.

The ported version is in a branch at http://cgit.collabora.com/git/user/thiagoss/cheese.git/, I'll get it reviewed by Cheese developers before putting it into the official repository.

That's it, time for the last talk of the conference and then have some fun. o/

Andy Wingo: quasiconf 2012: lisp @ froscon

Andy Wingo — 2012-08-15T21:34:25+00:00

Are you a Lisper, in the big-tent sense of the term? You within a day's transport of St. Augustin in Germany? Well shucks, have I got a thing for you: Quasiconf, a lisp sub-conference of FrOSCon, held next weekend (25-26 August 2012).

The full program isn't quite out yet, but they invited yours truly to talk there, and talk I will. In fact I'll talk thrice: they gave me 90 minutes, so I'll do three little presentations.

In the first, I'll finally give that talk I've been threatening to do for years, about why delimited continuations are the bee's knees, how they can be implemented, and what kind of lovely code they enable. The example will be a small memcached server and client.

Next I'll talk about things that implementing JavaScript has taught me about Scheme implementations. They were very surprising lessons to me, so I hope they will entertain the attendees. No spoilers, though!

Finally, now that Guile 2.0 has been out for a year and a half or so, it's a good time to look back at what worked well and what didn't. If you are even remotely interested about what it takes to maintain and enhance a mature language implementation, this will be an interesting talk for you. I'll also take the opportunity to summarize the progress on the development branch, which will probably be Guile 2.2.

So that's the thing. It's somewhat of a late announcement, but hey, if you can go, it's pretty much free and should be an interesting get-together. See you there!

Andy Wingo: lakewards

Andy Wingo — 2012-08-15T21:10:07+00:00

Time passes! And it takes us along with it: now a lazy float, now the running rapids, now an eerie calm.

And now, for me, a rising white noise of waterfall. In a month my partner and I move to Geneva. It's exciting and terrifying and anxiety-producing, but the nice thing about time is that I know it will carry me over the stress of, you know, learning French and such things.

I have loved my time in Barcelona, since moving here in 2005. The city has been pretty good to me. While I am indeed ready to go and try new things, I won't be leaving it without regret.

As a practical matter, I'll be (quite happily) staying on with Igalia, in their compilers group. Practically speaking, my move doesn't change much, work-wise; I've always met with customers over the net or on-site, never in Barcelona itself.

There are loads of practicalities to sort out, but if you have any knowledge about the town, I'm all digital ears. Also, if you happen to need some responsible caretakers for your alpine villa, do let me know. It doesn't even have to have a hot tub. I'm not particular. Just sayin'.

Stefan Kost: 15 Aug 2012

2012-08-15T09:19:51+00:00

buzztard goes retro
I started the month with some preparational work for the 2nd GSoc task - to use GstEncodeBin for recording. The first change was to streamline the recording UI. Instead of two dialog - one for setting and one for the progress - we now just show one. This causes less noise on the screen and avoids us some data passing. I also added a bunch of unit tests for sink bin to provide a means to check for regressions when switching the implementation. This also uncovered some
unimplemented code path on the gstreamer side. Those are now fixed or handled on our side.

In the mid of the month I could merge the encodebin transition. It is a very decent code simplification (68 insertions(+), 113 deletions(-)). Adding new output formats is a piece of cake now. And as a bonus, the code is now platform independent - instead of specifying the explicit elements to use, we now specify the container and audio track format. This way encodebin would pick suitable elements depending on what is available.

After the audiosynth baseclass was added, I wanted to see how well it works and started with a long time todo - a c64 sid synthesizer plugin. This wraps the rsid library - an awesome cycle based sid emulation. The library API is quite simple; basically:

  init(); update(regs[29]); clock(*buffer, num_samples); release();

I was really happy to also see a lot of technical information about the sid chip available on the net. When designing the gstreamer element API one needs to have a little understanding how the chip is use to still provide a convenient access to the parameters, but now hide important modes. That needs a bit of experiments to decide whether individual bits need to be exposed or only certain combinations make sense. In addition I implemented a couple of the typical effects. The normal timing resolution of a tracker song is not too high and thus one needs support from the element to provide things like arpeggio, portamento and vibrato.

The sidsyn is the first more complex audio synth we have in buzztard. It is also the first one that uses more of the gst-buzztard API. This was a good opportunity to review and cleanup the APIs a bit. With those cleanups I also killed the help interface, as new enough gstreamer has the required feature on the element metadata side and the feature is not crucial any way.

This month I skip the git stats, as I took the indent git commit hook into use and reformatted the buzztard code base. Voilà: buzztard - now with whitespace!