Planet GStreamer

Arrived at the hotel/summit. Going to head to the venue with a couple of folks soon.

And a small screencast for the day. Testing out the new blur-cache meant for notify-osd:

(click to play back, ogg/theora, ~1.5 MBytes)

It’s easy on CPU. So light actually, that I was able to record this very screencast with recordmydesktop on a Dell Mini 9. This is finally also using the subtle text drop-shadow the design folks asked for. Color, font, size and all are just randomly picked by me, as this is a test-program to exercise the small interal APIs I created for implementing the blur-cache.

I finally managed some time to make another libmtag release. This is mostly tiding up the code, cleanups, new codestyle, building improvements and moved to github Also some handy features: strip tag: now you can remove say id3v1 tags while keeping id3v2 intact get specific tag: similarly, you can retrieve only id3v2 information get all: this function [...]

It’s that time of the year again I’m about to start my trip to the summit. Uff… 5:00 in the morning and a trip of roughly 14 hours before me. But can’t wait to see all you GNOME-heads again face to face!

Everyone’s written a JavaScript loop that just loops over all the {LIs, links, divs} on a page*, and it’s pretty standard. Something like

var lis = document.getElementsByTagName("li");
for (var i=0; i<lis.length; i++) { // yes this could be more efficient, don't care
  // do something here to lis[i]
};

or, if you’re using jQuery:

$("li").each(function() {
  // do something here to this
});

This is problematic if there are, say, 2000 LI elements on the page, and what you’re doing in the loop is semi-intensive (imagine you’re creating a couple of extra elements to append to each of those LIs, or something like that). The reason this is a problem is that JavaScript is single-threaded. A tight loop like this hangs the browser until it’s finished, you get the “this script has been running for a long time” dialog, and the user interface doesn’t update while you’re in this kind of loop. You might think: aha, this will take a long time, so I’ll have some sort of a progress monitor thing:

var lis = document.getElementsByTagName("li");
for (var i=0; i<lis.length; i++) { // yes this could be more efficient, don't care
  // do something here to lis[i]
  progressMonitor.innerHTML = "processing list item " + i; // fail
};

but that doesn’t work. What happens is that the browser freezes until the loop finishes. Annoying, but there it is.

One approach to getting around this is with timeouts rather than a for loop.

var lis = document.getElementsByTagName("li");
var counter = 0;
function doWork() {
  // do something here to lis[i]
  counter += 1;
  progressMonitor.innerHTML = "processing list item " + counter;
  if (counter < lis.length) {
    setTimeout(doWork, 1);
  }
};
setTimeout(doWork, 1);

so you move the bit of work you need to do into a function, and that function re-schedules itself repeatedly, using setTimeout. This time, your user interface will indeed update, and your progress monitor will show where you’re up to. There are a couple of caveats with this: it’ll take a bit longer, and you’re no longer guaranteed to have things processed in the order you expect, but they’re minor issues.

For doing this in jQuery, a tiny plugin:

jQuery.eachCallback = function(arr, process, callback) {
    var cnt = 0;
    function work() {
        var item = arr[cnt];
        process.apply(item);
        callback.apply(item, [cnt]);
        cnt += 1;
        if (cnt < arr.length) {
            setTimeout(work, 1);
        }
    }
    setTimeout(work, 1);
};
jQuery.fn.eachCallback = function(process, callback) {
    var cnt = 0;
    var jq = this;
    function work() {
        var item = jq.get(cnt);
        process.apply(item);
        callback.apply(item, [cnt]);
        cnt += 1;
        if (cnt < jq.length) {
            setTimeout(work, 1);
        }
    }
    setTimeout(work, 1);
};

and now you can do

$.eachCallback(someArray, function() {
  // "this" is the array item, just like $.each
}, function(loopcount) {
  // here you get to do some UI updating
  // loopcount is how far into the loop you are
});

$("li").eachCallback(function() {
  // do something to this
}, function(loopcount) {
  // update the UI
});

Not always a useful technique, but when you need it, you need it.

So far I neglected writing about this year's Google Summer of Code. This ends with this post. As last year, I'm working on GNOME DVB Daemon.

In the last couple of weeks I concentrated on the user experience, thus making setting up devices as easy as possible. I made a short screencast that shows the new assistant started by the Totem plugin.

If there's only one unconfigured device it's selected automatically. If you have multiple devices it's checked if there's already a device group of the same type and adds the device to the group, if possible. In addition, you don't have to care about channels.conf at all anymore. In expert mode, though, you still can create only a channels.conf file without actually setting up the devices.

The Totem plugin was improved, too. As you can see in the next screencast:

Everything that's available in gnome-dvb-control can be accessed from within Totem. You can browse EPG, manage recordings, schedule recordings and configure devices. The next step is to remove the existing DVB code from Totem and make the dvb-daemon plugin built-in.

Furthermore, I finally took care that live TV doesn't interfere with recordings. If a recording is coming up and you're watching a channel on a different transport stream, streaming is stopped so the recordings can start properly. That means you can still watch a different channel on the same transport stream (TS) or record multiple channels on the same TS simultaneously.

This are all unreleased features I'm talking about, but hopefully I can make a proper tarball release soon.

Now there are basically two items left on my GSoC todo list. Writing a ring buffer to provide a way to do time shifting, pause/rewind/fast-forward live TV and a plugin system for EPG aggregators.

After seeing that I possibly might have had some exploits run on my site again, I upgraded to wordpress 2.8

After reading up on hardening wordpress, the official site mentions AskApache, some plugin that helps hardening. I’m not too sure about it yet, because it wants to be writing .htaccess files in my directories and for that I have to open up more than I would want. But hey, let’s give it a go.

At some point it creates a username and password that you choose. I go on and configure stuff, not knowing very well which of its many modules I’m supposed to activate, or why.

I forget about it, and ten minutes later I check my mail. I have a mail from AskApache. With my login details. And the password in plaintext.

…

Is the WordPress security model just fundamentally broken ?

So yesterday was the 5th anniversary party for the Fluendo group. It only seems like yesterday that I joined the group (3 years 4 1/2 months ago) to work for Flumotion. It was really nice of the company to fly me in especially to Barcelona for the party, so thank you.

My level of Spanish being what it is, and my accent being what it is, my cab ride to Las Palmas cost me some €60 from the airport, and that's after the cabbie switched off the meter...

Apparently, the «Catalina Park» apartments booked by the nice people at the GNOME Foundation, have a namesake near Playa de Inglés.


From top to bottom: where I wanted to go, where I was, where I ended up (locations provided «by ear», do not try to replicate at home). Yippee!



FreeFA

In other news the FreeFA tournament is going to happen next Thursday, from 15:00 to 17:00. I'll put the details onto the Wiki when I can remember my password.

Yesterday was cause for celebration. We got together to celebrate five years of the Fluendo Group!

The picture quality is bad, and not everyone is in it, but I just took it on a whim after marveling how many people were there. I didn’t even know all of them – yes it’s gotten to that point. 67 months ago I arrived in Barcelona without the company even being created…

We celebrated with mountains of cheese and rivers of wine which in the first year would have lasted us a few weeks and now only lasted an hour.

As magical accidents sometimes happen, today is also the day Fluendo received the certification confirmation from Dolby for our DVD player. It didn’t take long to land in the webshop, so finally our DVD player is up for sale! So you know what to get us for our birthday – a shop checkout with the dvd player in your cart.

Good timing – that means that at this year’s GUADEC/Desktop Summit I know what the answer will be to one of the most asked questions I get.

This is the first GUADEC I’m going to with Kristien in tow, I hope she can manage. I’ll be there from Monday through Friday, because the week is bookended by two weddings. Looking forward to a GStreamer summit on Thursday discussing 1.0…

Today we celebrated 5 years of the Fluendo group around some French Wine & Cheese. Was quite impressive to make a speech in front of around 70 people explaining who we became in those 5 years, one of the biggest Internet traffic pusher in Spain with more than 22 Gbps of bandwidth, having GStreamer and our codecs spread in Linux devices all around us with famous customers like Dell, HP, Wyse, Dexxon, Guillemot, etc.. and having more than 6000 people a day installing our Moovida media center/player.

This day got even better when, by pure luck, Dolby gave us the green light to distribute the DVD player on this exact same day after one full year of certification and paperwork. So it’s my pleasure to announce that the Fluendo DVD player is finally available at http://www.fluendo.com/shop/product/fluendo-dvd-player/ .

A big thanks to everyone for being patient and supporting us along those years, it would not be the same without our beloved community. And now let’s see where we will be at in another 5 years !

So the BBC is making a new drama series about the battle between the ZX spectrum and the BBC Micro. Currently codenamed ‘Syntax Era’. As it turns out Clive Sinclair, the creator of the ZX Spectrum, had his offices very close to the current Collabora office here in Cambridge. And due to our own Edward Hervey knowing some of the people involved, the production team behind this new series came by our offices to do some location scouting some weeks ago. It is a little bit up in the air if they are going to use our offices or not in the end, but there is a chance they will, and if that happens there is also chance you might catch some familiar faces as extras in this new series

left work around 38 degrees C, got a haircut, went for some great tapas on my own reading Darkly Dreaming Dexter, went to a bar, met up with friends, an impromptu bbq plan was hatched, went to a lovely atico at Portal De L’Angel, barbecued in a soothing summer breeze, rode home on the back of a motorcycle hanging on for dear life. All in all a typical Barcelona summer Tuesday.

I updated gnome-bluetooth in Fedora 11, and that new version supports Secure Simple Pairing, an easier pairing mode for Bluetooth 2.1 devices.

The update currently lives in the updates-testing repository, but will be in the normal updates when we've had enough good feedback about it.

If you have Bluetooth devices in your possession that don't work as expected with your systems, and fancied a bit of playful testing, find me at GCDS, and we'll try and fix that.

If you're interested in joining in for some «Futbol» at GCDS, add your name to the list on the Wiki, and bring your shoes/shinpads/other bits of kit.

We don't have a date and time settled for it yet, so make sure you check the schedule when at the conference.

As every year, if you don't bring shinpads and you break your leg in two, you'll have little sympathy.

The GStreamer team is proud to announce a new release in the 0.10.x stable series of the GStreamer FFmpeg module.

Check out release notes here, or download tarballs from here.

UPnP used to be bound to local network. We're working around this with Coherence and Telepathy tubes to bridge 2 (or more) coherence instances over Jabber chatrooms. So the 2 screenshots below are a glimpse of what we will show at GDCS on the 7th of July :)

On Frank's desktop in Germany, 2 UPnP MediaServers showing up in his LAN. One is local (my media) and the other one is mine, in Spain. If you click on the image you'll see that the device UUID contains the tube string and that the media link is my IP in Spain.

Here is my desktop, showing my local MediaServer and Frank's server:

This is a early-stage prototype, there are still things to improve and new features to implement, like sharing a folder directly from Empathy with a single contact of the roster. For playback we can't yet do NAT traversal, so the Coherence IP address and port are currently hardcoded. We will implement ICE support later on. More to say during the talk :)

Jan and Arek entered this year’s ICFP programming contest. It’s a three day programming contest, so this morning they asked if they could swap their Friday project day for today to finish the contest. They seem to be in the top third at the moment.

Arek’s never been a fan of long meetings, but today’s standup meeting was particularly amusing with Arek urging everyone to keep focused and get out there quickly. They had less than two hours left on the clock.

Spot the seven differences

Amusingly, today they came to work with almost the same shirt on, by accident! I can only assume there is a big clothes factory in Poland where they have huge stock of the same fabric…

90 minutes left, knock them dead, guys!

I was pointed today to this blog, which shows the integration work being done in KDE with fprintd. Happy to see all that work on the daemon and the documentation is coming to good use.

It's been a while since the last post ;) People may wonder where I've gone during these days... Actually I spent some time on a graduation trip to Yun Nan province in China, and also the preparing for my thesis defense; the defense is in late July, wish me luck :)

My GSoC project on mpegtsmux is going on. Since the kicking off, I've been reading existing code and related specifications, also debugging for some use cases. As mpegtsmux is kind of mature already, my main task is to ensure robust support on common codecs, video or audio, and also add some useful features. I think this is not taking too much time, and after this my focus would turn to the new mpeg PS muxer.

I've proposed to add more features to the TS muxer in my GSoC application, but "more" seems too vague to implement. The first thing to do is to make this exact. After some research, I propose to add the following two major capabilities to the mpegtsmuxer:
1. Multiple program support
TS streams have the capability to handle multiple programs in a single stream by definition (ISO 13818-1), however mpegtsmux is currently not supporting this. (Neither do ffmpeg and VLC; If we have it then it's a unique feature! XD )
2. Constant bit rate (CBR) output
In some applications, like digital video broadcasting, the data sink needs a constant bit rate stream. This is usually archived by stuffing: if the current bit rate is less than the defined value, the muxer puts null packets in the stream to compensate for the stream. DVB people may need this feature.

Not sure if this would be interesting to other people. Any idea? I think the first feature is not hard to implement, as libtsmux has already provided the APIs for multiple program TS and we only need to define an interface and wire the APIs there. The second feature requires the functionality of bit rate detection, and can even be more complicated if we want some "auto bandwidth adjustment", that is, the muxer informing other components in the pipeline of bandwidth overflow, so that other elements like encoders can compress harder to fit the stream into the bandwidth. (Thanks Edmund Humenberger for rising this issue.) But after all I think such advanced features can be put aside for the time being.

Besides the planning, there is a note worthy finding for the case when mpegtsmux muxes an h264 + AAC combination like:

gst-launch \
filesrc location=v.264 ! h264parse ! mux. \
filesrc location=a.aac! aacparse ! mux. \
mpegtsmux name=mux ! filesink

The output stream of this pipeline has been reported to be problematic (see here). I used to suspect that the bug lies in PAT / PMT tables, but they turns out to be innocent. (damn the buggy analyzer I was using!) I was a bit worrying about the h264 handling code, until I noticed the interesting way the packets are arranged in this erroneous stream: all video packets takes the fore part, with the audio ones staying in the hind part. It's like a concatenation of two chunks of data, instead of interlaced according to their timestamps.

I thus look into h264parse and found that the parser is writing GST_CLOCK_TIME_NONE at the timestamp of every buffer. Then its' clear why the video packets goes entirely before the audio part, as the scheduling algorithm of mpegtsmux favours GST_CLOCK_TIME_NONE than other time stamps. This scheduling rule hopes to drain the GST_CLOCK_TIME_NONE stream to get more useful timestamps, but fails to detect the case when useful timestamps never come.

To verify that this is indeed the cause, I did a simple crazy test: if the scheduling algorithm is modified to interlace video and audio packets in a naive way, the stream plays on mplayer with both audio and video there (although not synchronized).

I'm going to look into h264parse and make it chopping the right timestamp this week. I'll file a bug and try to come up with a patch that fixes it. On the other hand, it might also be good if we enhance the scheduling algorithm of mpegtsmux to detect obvious timestamp error / bitrate overflow. Maybe this can be combined with the bit rate detection.

Finally I'm considering to implement different "service modes" for mpegtsmux. A service mode is a configuration of the muxer under which the output multiplex is compliant to some existing standards. What we have now can be called the basic service level, where we can mux video and audio streams together. We can also have DVB-T mode in which the stream is fully DVB-T compliant, or PS3 mode in which the stream is useful on PS3. (Thanks Edmund again for rising this requirement.) This is a more demanding task, and there is already some discussion on Bugzilla on this issue. The discussion there is really informative, and I need to consider a bit more before elaborating the goal. I'll probably write about this in later posts. Any suggestions are welcome.

I used to be a Linux-beta-tester of Realsoft3D long time ago. Even wrote a .r3i import-plugin for gimp once. I also got into RenderMan (RM), wanting to write a RM-binding (using RIB) for Realsoft3D. The RM-display-driver I got done and had RM (actually the free RM-implementation BMRT) render into Realsoft3D’s viewports. That was quite something! But as time went on, I watched blender accelerate in development and surpass numerous commercial 3D-suites in features… not to mention the user-community which formed behind it, after blender became OpenSource. It was obvious that Realsoft3D was a dead end for me, although I regard my CG/3D endeavours as a modest hobby at best.

Concepts for CG are the same, independent of the tools you use. Still, using fast and sophisticated tools makes your work/hobby more enjoyable. So I need to transfer my knowledge from Realsoft3D to blender. There are many disciplines in CG which you need to tackle, if you want to create good results. There’s the idea for an image or animation (which you need to cut into scenes to tell a captivating story), you need to build your characters/environment (modeling), then you need to “dress them up” (shading/texturing), the scene has to be properly lit (lighting) and you have to animate all of that (rigging/animation, sometimes involves physical simulation if you want to go the full stretch). All that is a huge pile of work, where I usually only enjoy the modeling-, shading- and lighting-parts.

Let’s get used to a new workflow then.

Thanks to Shawn Kirst’s normal-map plugin for Gimp 2.6.x, I created from this source color-map a derived bump-map and a derived specular-map. These put to use for a floor-material, together with a nice lighting setup in my scene, I get…

… as a result (it’s not about the monkey, but the lighting and floor-material).

Not photorealistic, but good enough to look decent. Such scene setups are commonly used to show off models you built in near life-like lighting-settings, without the need for hour long renderings. The above scene rendered in a couple of minutes on an intel Core 2 Duo (2.0 GHz) using the internal renderer of blender.

I also looked into luxrender, which is an unbiased renderer under the GPL, that comes with a blender-plugin. An unbiased renderer models its lighting calculations after the physical properties of light in the real world: wavelengths. It does not use a RGB-colormodel for its internal computations like many other renderers do. Furthermore the term “unbiased” means it does not make any assumptions or takes any shortcuts during the calculation. This has a huge advantage: true photorealism. The advantage comes at the cost of huge rendering times. An unbiased renderer is never done with an image-rendering. Instead it continues to improve it over time. After the first few seconds the image is very noisy and blocky. As it continues the noise gets finer and finer. You can let it go at an image until the noise resolution falls below the threshold of the images pixel-resolution. But that can take ages. What blender became for the 3D-suites in the OpenSource domain, luxrender will for the field of unbiased renderers I believe. I would not be surprised, to see it prosper as well as blender does. But luxrender has some tough commercial competition to beat Maxwell and indigo among a few others. Unbiased renderers are still new territory for me and I don’t wield this tool well yet, thus I don’t feel confident enough to show something in public right now.

mach allows you to set up clean roots from scratch for any distribution or distribution variation supported.

This release of mach contains fixes for Python 2.6, and adds Fedora 10 and 11, while fixing the archived Fedora locations.

Get it from the mach project page.

Next step on this weekend’s yakshave: a first implementation of moap vcs bisect!

The interface is lifted from git, obviously, since that’s where most people will know the feature from.

I implemented it first with CVS, so I could fix this pychecker bug which was blocking Fedora from bumping the pychecker version from 0.8.17 (3 years old) to 0.8.18. And sure enough, it picked out the commit I broke.

While implementing and while dealing with CVS’s idea of how it stores CVS revisions and dates and so on, I googled and was amused to find this first hit on google for the words cvs and bisect. Clever Andy! And he cleverly sidestepped the problem I wrestled with by making the user specify two dates at the start instead of trying to figure it out from the checkout. And all in lisp too!

Then, to test that my VCS interface was sane, I implemented it for Subversion as well. That took about 15 minutes, since Subversion is much more sane than CVS. I tried the following command on a flumotion checkout:

moap vcs bisect reset; moap vcs bisect start; svn up; moap vcs bisect good; svn up -r 3000; moap vcs bisect bad; MOAP_DEBUG=4 moap vcs bisect run ./test.sh

With test.sh containing
test -e flumotion/component/consumers/gdp/gdp.py

(In other words, look for the commit that added this file.)

Sure enough, it picked out this commit:


[moap-trunk] [thomas@ana flumotion]$ moap vcs bisect diff
Index: /home/thomas/tmp/flumotion/configure.ac
===================================================================
--- /home/thomas/tmp/flumotion/configure.ac (revision 6909)
+++ /home/thomas/tmp/flumotion/configure.ac (revision 6908)
@@ -212,7 +212,6 @@
flumotion/component/combiners/switch/Makefile
flumotion/component/consumers/Makefile
flumotion/component/consumers/disker/Makefile
-flumotion/component/consumers/gdp/Makefile
flumotion/component/consumers/httpstreamer/Makefile
flumotion/component/consumers/preview/Makefile
flumotion/component/consumers/shout2/Makefile
Index: /home/thomas/tmp/flumotion/flumotion/component/consumers/Makefile.am
===================================================================
--- /home/thomas/tmp/flumotion/flumotion/component/consumers/Makefile.am (revision 6909)
+++ /home/thomas/tmp/flumotion/flumotion/component/consumers/Makefile.am (revision 6908)
@@ -11,7 +11,6 @@

SUBDIRS = \
disker \
- gdp \
httpstreamer \
preview \
shout2
Index: /home/thomas/tmp/flumotion/ChangeLog
===================================================================
--- /home/thomas/tmp/flumotion/ChangeLog (revision 6909)
+++ /home/thomas/tmp/flumotion/ChangeLog (revision 6908)
@@ -1,16 +1,5 @@
2008-06-20 Thomas Vander Stichele <thomas at apestaart dot org>
</thomas>

- * configure.ac:
- * flumotion/component/consumers/Makefile.am:
- * flumotion/component/consumers/gdp (added):
- * flumotion/component/consumers/gdp/gdp.py (added):
- * flumotion/component/consumers/gdp/__init__.py (added):
- * flumotion/component/consumers/gdp/Makefile.am (added):
- * flumotion/component/consumers/gdp/gdp.xml (added):
- Add a GDP consumer.
-
-2008-06-20 Thomas Vander Stichele <thomas at apestaart dot org>
-
* flumotion/component/producers/gdp/gdp.py:
Add error for http://bugzilla.gnome.org/show_bug.cgi?id=532364
</thomas>

So, the feature is ready for testing. It could use some more documenting, and some additional goodies like accepting arguments to moap vcs bisect start for example.

Feedback appreciated!

Still on the yak shave expedition.

I’ve written some simple scripts and files to set up and build python 2.3, 2.4, and 2.5 in separate prefixes to be able to test my software against these versions.

If you’re interested, in theory it should be really simple:

• Install jhbuild
• Check out my python jhbuild directory: svn co https://thomas.apestaart.org/thomas/svn/jhbuild/python
• run ‘make’

As the README says, this should go on to build all versions of python, and install some scripts.

After that, you just run py-2.3 to go into a shell with Python 2.3 on your path.

Don’t say I never did anything for you.

As part of this weekend’s yakshave, I’m trying to implement a handler for STORE_MAP in pychecker. STORE_MAP is a new opcode in Python 2.6, which speeds up dict building.

So, for the first time I went under the hood of Python and figured out just enough to understand this problem. It was a lot less scary than I thought it was going to be!

It seems that using dis.dis(), one can easily dissassemble any python function into its opcodes. This shows clearly where the behaviour is different between python 2.5 and python 2.6.

Given the following function:
f = lambda: {'a': 1, 'b': 2}

Python 2.6 gives:
1 0 BUILD_MAP 2
3 LOAD_CONST 0 (1)
6 LOAD_CONST 1 ('a')
9 STORE_MAP
10 LOAD_CONST 2 (2)
13 LOAD_CONST 3 ('b')
16 STORE_MAP
17 RETURN_VALUE

I couldn’t find a good description of the output of dis.dis, but in my naiveness I am guessing the following:

• The first 1 maps to the line number in the code object where the function is found.
• The second column is the offset of the opcode and its arguments
• The third column is the opcode name
• The next column is the arguments for the opcode; in the case of CONST, it shows the index as well as the const object indexed

I am assuming each opcode takes one address location, and each argument takes two; that maps with the address pointers in front of the opcodes.

The opcodes are all documented.

So, in human terms:

• we start with BUILD_MAP, saying that we’ll create a new dictionary on the stack, with 2 entries.
• we load the constant with index 0 onto the stack (which happens to be the integer object ‘1′, the value of the ‘a’ key)
• we load the constant with index 1 onto the stack (which happens to be the string object ‘a’, the key for the ‘1′ value)
• STORE_MAP pops the key and the value off the stack, storing them in the dict. Note that the key was indeed loaded on the stack after the value. Now only the dictionary is left on the stack.
• Repeat LOAD_CONST, LOAD_CONST and STORE_MAP for the next set
• RETURN_VALUE returns the current value on the stack to the caller

Pretty simple, when you look at it twice.

For the same code, python 2.5 gives:
>>> dis.dis(f)
1 0 BUILD_MAP 0
3 DUP_TOP
4 LOAD_CONST 1 ('a')
7 LOAD_CONST 2 (1)
10 ROT_THREE
11 STORE_SUBSCR
12 DUP_TOP
13 LOAD_CONST 3 ('b')
16 LOAD_CONST 4 (2)
19 ROT_THREE
20 STORE_SUBSCR
21 RETURN_VALUE

This code is slightly longer and more complicated. Basically, LOAD_CONST, LOAD_CONST, STORE_MAP was implemented with DUP_TOP, LOAD_CONST, LOAD_CONST, ROT_THREE, STORE_SUBSCR

It looks like DUP_TOP was needed because STORE_SUBSCR consumes the dictionary off the stack, and ROT_THREE is needed because the arguments are pushed on the stack in the wrong order.

Seems like a nice and obvious improvement once you understand it. An exercise for the reader is to profile whether this change actually makes things faster in practice.

So, where does this leave me for pychecker ? It now looks deceptively simple. STORE_MAP simply pops off two items of the stack. There is nothing to check for, since we’re in a dictionary context. So all my implementation needs to do is to pop 2 items off the stack, and that’s it.

And thus it was commited to pychecker CVS. Popping one item off the yak stack!

So in preparation for heading of to Gran Canaria and GUADEC on Thursday I pushed a new Transmageddon release today. 0.11 is actually the first release I posted to sites such as Gnomefiles and Freshmeat so in some sense I guess I feel more confident about this version that earlier ones. A lot of new features included, like multipass encoding, videoflipping (so if your video is 90 degrees tilted you can correct that during transcoding) and better profiles. Still kinda rough at the edges though, and the very latest GStreamer releases are needed for everything to work. Still expecting quite a few bugs to be reported still.

In some sense development have stood still for a while as I have focused on testing various devices and GStreamer features. Remuxing is still on my todo list, but discovered that for instance an ac3parser is still needed in GStreamer to
do nice DVD conversions.

Was kinda nice to successfully use Transmageddon for a business related need at work, we had gotten a request at Collabora Multimedia which meant I needed to transcode a video into quicktime+h264+aac from a .vob file. Worked like a charm I mean while I have of course done hundred of transcodes as part of development, it was nice to do one for something ‘real’

The yak shave started yesterday evening. The yak stack is actually a forked one this time, both of the forks involving pychecker.

I might not remember everything in order, but in a nutshell the stack is something like this:

• The original goal for this weekend at the beginning of the week was to release my cd ripper, morituri
• Something in its pychecker run did not run with pychecker 0.8.17 (the version still in Fedora, from 2006), and worked with pychecker 0.8.18 (my own build). Fork point 1.
• While releasing moap this week, I realized that Freshmeat changed their remote API, which I should fix before I do another release of anything. Fork point 2.

Fork point 1 continues here:

• I mailed Fedora’s pychecker maintainer, offering my help, and sending a patch to update to 0.8.18, which I built and installed locally. He mailed back informing me about this pychecker bug with anaconda which was blocking the upgrade. Looking at that bug, it looked suspiciously like a bug triggered by code I added to pychecker last year.
• However, I’d like to confirm so in the easiest way. git’s got this great feature, git bisect, and wouldn’it be nice if I could do that on pychecker now ? Hey, why not add bisection to moap ?
• CVS is actually not a very manageable VCS if you want to do fancy stuff. It costs me a few hours to figure out how I should get a more-or-less usable date from a CVS checkout. The final solution is similar to the reply to my stackoverflow question
• moap vcs bisect run is now implemented and finds the commit that broke anaconda’s pychecking. STACK POINTER IS CURRENTLY HERE.

Fork point 2 continues here:

• moap’s make check didn’t work because pychecker complained about the following code:
  ef func():
d = { 'a': 1, 'b': 2}
print d.keys()

which triggers, in python 2.6, the following warning:
Object (d) has no attribute (keys)

I can’t let myself change code in moap without a working make check, so on to figuring out what’s wrong in pychecker

After lots of debugging and print statements, I figure out that pychecker dispatches Python opcodes, and it silently drops the ones it doesn’t know about. Python 2.6 added a new opcode, STORE_MAP, and so pychecker doesn’t properly handle the stack since it ignores the opcode. I should error out on those opcodes. STACK POINTER IS CURRENTLY HERE

Before I can fix that though, I decide I should make pychecker’s test suite error out on unknown opcodes.

Of course, this will error out differently for different python versions, so I need different python versions on this machine.

I could do that by hand, but I’d also like Twisted’s trial to run the testsuite, which also needs zope-interface in recent versions, which needs setuptools. So hey, why not set up jhbuild stuff to build all these python versions ?

Python 2.3 on my 64 bit machine doesn’t work with setuptools, so the newest Twisted without it is 1.3.0, and it takes a while to figure out why trial doesn’t run my testcases (it ends up being because 1.3.0 trial expects subclasses of twisted.trial.unittest.TestCase, not unittest.TestCase)

I’ll blog about the useful products of my yak shave separately, for those who don’t enjoy descriptions of yak shavings, only outcomes.

In general, I actually enjoy yak shaves. They’re massive treasure hunts, you learn a lot, and you end up fixing a nice bunch of things all over the stack if you persevere. But it’s probably more a mentality thing than anything else, and I really only indulge myself in these in my spare time.

The obvious idea that pops into everyone’s head, including mine, when talking about having a running CouchDB that’s specific to me is: why use TCP for it? Why not just use a unix domain socket? Then you don’t have to worry about other people on the same machine trying to access it. Everyone thinks this, and on balance it’s not the way to go. This is why.

1. You can’t browse to a unix domain socket in your web browser. This, by itself, is enough to kill the idea for me. I genuinely love the idea that applications store their data and then I can see that data in my browser using Futon, the CouchDB web UI. I can edit that data. That’s fantastic. Using unix sockets would break that.
2. One other nice thing about CouchDB is that you can do replication between two different databases; I like this because I can have the same data on my laptop and my netbook if I want. That doesn’t work if you use domain sockets, because the two CouchDBs can’t see one another.
3. As far as I can tell, Mochiweb, the underlying Erlang HTTP library that Couch uses, doesn’t do domain sockets anyway. (Obviously fixing this is only a Small Matter Of Programming.)

This has been a public service broadcast on behalf of the Write These Reasons Down So I Have Them To Hand Next Time Someone Suggests Unix Domain Sockets party.

I tried to beat the wash cycle on my washing machine at doing something useful[1].

Tried to add Introspection support to gnome-bluetooth.

The result nearly works, as it seems that there's no bindings for DBusGProxy in gobject-introspection...

** WARNING **: Entry 'DBusGProxy' not found

If somebody knows...

[1]: I played football twice today, and needed to wash my kit again, as I'll be playing tomorrow, though I hope we'll play better than we did this evening, shrug.