June 21, 2022

Andy Wingoan optimistic evacuation of my wordhoard

(Andy Wingo)

Good morning, mallocators. Last time we talked about how to split available memory between a block-structured main space and a large object space. Given a fixed heap size, making a new large object allocation will steal available pages from the block-structured space by finding empty blocks and temporarily returning them to the operating system.

Today I'd like to talk more about nothing, or rather, why might you want nothing rather than something. Given an Immix heap, why would you want it organized in such a way that live data is packed into some blocks, leaving other blocks completely free? How bad would it be if instead the live data were spread all over the heap? When might it be a good idea to try to compact the heap? Ideally we'd like to be able to translate the answers to these questions into heuristics that can inform the GC when compaction/evacuation would be a good idea.

lospace and the void

Let's start with one of the more obvious points: large object allocation. With a fixed-size heap, you can't allocate new large objects if you don't have empty blocks in your paged space (the Immix space, for example) that you can return to the OS. To obtain these free blocks, you have four options.

  1. You can continue lazy sweeping of recycled blocks, to see if you find an empty block. This is a bit time-consuming, though.

  2. Otherwise, you can trigger a regular non-moving GC, which might free up blocks in the Immix space but which is also likely to free up large objects, which would result in fresh empty blocks.

  3. You can trigger a compacting or evacuating collection. Immix can't actually compact the heap all in one go, so you would preferentially select evacuation-candidate blocks by choosing the blocks with the least live data (as measured at the last GC), hoping that little data will need to be evacuated.

  4. Finally, for environments in which the heap is growable, you could just grow the heap instead. In this case you would configure the system to target a heap size multiplier rather than a heap size, which would scale the heap to be e.g. twice the size of the live data, as measured at the last collection.

If you have a growable heap, I think you will rarely choose to compact rather than grow the heap: you will either collect or grow. Under constant allocation rate, the rate of empty blocks being reclaimed from freed lospace objects will be equal to the rate at which they are needed, so if collection doesn't produce any, then that means your live data set is increasing and so growing is a good option. Anyway let's put growable heaps aside, as heap-growth heuristics are a separate gnarly problem.

The question becomes, when should large object allocation force a compaction? Absent growable heaps, the answer is clear: when allocating a large object fails because there are no empty pages, but the statistics show that there is actually ample free memory. Good! We have one heuristic, and one with an optimum: you could compact in other situations but from the point of view of lospace, waiting until allocation failure is the most efficient.

shrinkage

Moving on, another use of empty blocks is when shrinking the heap. The collector might decide that it's a good idea to return some memory to the operating system. For example, I enjoyed this recent paper on heuristics for optimum heap size, that advocates that you size the heap in proportion to the square root of the allocation rate, and that as a consequence, when/if the application reaches a dormant state, it should promptly return memory to the OS.

Here, we have a similar heuristic for when to evacuate: when we would like to release memory to the OS but we have no empty blocks, we should compact. We use the same evacuation candidate selection approach as before, also, aiming for maximum empty block yield.

fragmentation

What if you go to allocate a medium object, say 4kB, but there is no hole that's 4kB or larger? In that case, your heap is fragmented. The smaller your heap size, the more likely this is to happen. We should compact the heap to make the maximum hole size larger.

side note: compaction via partial evacuation

The evacuation strategy of Immix is... optimistic. A mark-compact collector will compact the whole heap, but Immix will only be able to evacuate a fraction of it.

It's worth dwelling on this a bit. As described in the paper, Immix reserves around 2-3% of overall space for evacuation overhead. Let's say you decide to evacuate: you start with 2-3% of blocks being empty (the target blocks), and choose a corresponding set of candidate blocks for evacuation (the source blocks). Since Immix is a one-pass collector, it doesn't know how much data is live when it starts collecting. It may not know that the blocks that it is evacuating will fit into the target space. As specified in the original paper, if the target space fills up, Immix will mark in place instead of evacuating; an evacuation candidate block with marked-in-place objects would then be non-empty at the end of collection.

In fact if you choose a set of evacuation candidates hoping to maximize your empty block yield, based on an estimate of live data instead of limiting to only the number of target blocks, I think it's possible to actually fill the targets before the source blocks empty, leaving you with no empty blocks at the end! (This can happen due to inaccurate live data estimations, or via internal fragmentation with the block size.) The only way to avoid this is to never select more evacuation candidate blocks than you have in target blocks. If you are lucky, you won't have to use all of the target blocks, and so at the end you will end up with more free blocks than not, so a subsequent evacuation will be more effective. The defragmentation result in that case would still be pretty good, but the yield in free blocks is not great.

In a production garbage collector I would still be tempted to be optimistic and select more evacuation candidate blocks than available empty target blocks, because it will require fewer rounds to compact the whole heap, if that's what you wanted to do. It would be a relatively rare occurrence to start an evacuation cycle. If you ran out of space while evacuating, in a production GC I would just temporarily commission some overhead blocks for evacuation and release them promptly after evacuation is complete. If you have a small heap multiplier in your Immix space, occasional partial evacuation in a long-running process would probably reach a steady state with blocks being either full or empty. Fragmented blocks would represent newer objects and evacuation would periodically sediment these into longer-lived dense blocks.

mutator throughput

Finally, the shape of the heap has its inverse in the shape of the holes into which the mutator can allocate. It's most efficient for the mutator if the heap has as few holes as possible: ideally just one large hole per block, which is the limit case of an empty block.

The opposite extreme would be having every other "line" (in Immix terms) be used, so that free space is spread across the heap in a vast spray of one-line holes. Even if fragmentation is not a problem, perhaps because the application only allocates objects that pack neatly into lines, having to stutter all the time to look for holes is overhead for the mutator. Also, the result is that contemporaneous allocations are more likely to be placed farther apart in memory, leading to more cache misses when accessing data. Together, allocator overhead and access overhead lead to lower mutator throughput.

When would this situation get so bad as to trigger compaction? Here I have no idea. There is no clear maximum. If compaction were free, we would compact all the time. But it's not; there's a tradeoff between the cost of compaction and mutator throughput.

I think here I would punt. If the heap is being actively resized based on allocation rate, we'll hit the other heuristics first, and so we won't need to trigger evacuation/compaction based on mutator overhead. You could measure this, though, in terms of average or median hole size, or average or maximum number of holes per block. Since evacuation is partial, all you need to do is to identify some "bad" blocks and then perhaps evacuation becomes attractive.

gc pause

Welp, that's some thoughts on when to trigger evacuation in Immix. Next time, we'll talk about some engineering aspects of evacuation. Until then, happy consing!

by Andy Wingo at June 21, 2022 12:21 PM

June 20, 2022

Andy Wingoblocks and pages and large objects

(Andy Wingo)

Good day! In a recent dispatch we talked about the fundamental garbage collection algorithms, also introducing the Immix mark-region collector. Immix mostly leaves objects in place but can move objects if it thinks it would be profitable. But when would it decide that this is a good idea? Are there cases in which it is necessary?

I promised to answer those questions in a followup article, but I didn't say which followup :) Before I get there, I want to talk about paged spaces.

enter the multispace

We mentioned that Immix divides the heap into blocks (32kB or so), and that no object can span multiple blocks. "Large" objects -- defined by Immix to be more than 8kB -- go to a separate "large object space", or "lospace" for short.

Though the implementation of a large object space is relatively simple, I found that it has some points that are quite subtle. Probably the most important of these points relates to heap size. Consider that if you just had one space, implemented using mark-compact maybe, then the procedure to allocate a 16 kB object would go:

  1. Try to bump the allocation pointer by 16kB. Is it still within range? If so we are done.

  2. Otherwise, collect garbage and try again. If after GC there isn't enough space, the allocation fails.

In step (2), collecting garbage could decide to grow or shrink the heap. However when evaluating collector algorithms, you generally want to avoid dynamically-sized heaps.

cheatery

Here is where I need to make an embarrassing admission. In my role as co-maintainer of the Guile programming language implementation, I have long noodled around with benchmarks, comparing Guile to Chez, Chicken, and other implementations. It's good fun. However, I only realized recently that I had a magic knob that I could turn to win more benchmarks: simply make the heap bigger. Make it start bigger, make it grow faster, whatever it takes. For a program that does its work in some fixed amount of total allocation, a bigger heap will require fewer collections, and therefore generally take less time. (Some amount of collection may be good for performance as it improves locality, but this is a marginal factor.)

Of course I didn't really go wild with this knob but it now makes me doubt all benchmarks I have ever seen: are we really using benchmarks to select for fast implementations, or are we in fact selecting for implementations with cheeky heap size heuristics? Consider even any of the common allocation-heavy JavaScript benchmarks, DeltaBlue or Earley or the like; to win these benchmarks, web browsers are incentivised to have large heaps. In the real world, though, a more parsimonious policy might be more appreciated by users.

Java people have known this for quite some time, and are therefore used to fixing the heap size while running benchmarks. For example, people will measure the minimum amount of memory that can allow a benchmark to run, and then configure the heap to be a constant multiplier of this minimum size. The MMTK garbage collector toolkit can't even grow the heap at all currently: it's an important feature for production garbage collectors, but as they are just now migrating out of the research phase, heap growth (and shrinking) hasn't yet been a priority.

lospace

So now consider a garbage collector that has two spaces: an Immix space for allocations of 8kB and below, and a large object space for, well, larger objects. How do you divide the available memory between the two spaces? Could the balance between immix and lospace change at run-time? If you never had large objects, would you be wasting space at all? Conversely is there a strategy that can also work for only large objects?

Perhaps the answer is obvious to you, but it wasn't to me. After much reading of the MMTK source code and pondering, here is what I understand the state of the art to be.

  1. Arrange for your main space -- Immix, mark-sweep, whatever -- to be block-structured, and able to dynamically decomission or recommission blocks, perhaps via MADV_DONTNEED. This works if the blocks are even multiples of the underlying OS page size.

  2. Keep a counter of however many bytes the lospace currently has.

  3. When you go to allocate a large object, increment the lospace byte counter, and then round up to number of blocks to decommission from the main paged space. If this is more than are currently decommissioned, find some empty blocks and decommission them.

  4. If no empty blocks were found, collect, and try again. If the second try doesn't work, then the allocation fails.

  5. Now that the paged space has shrunk, lospace can allocate. You can use the system malloc, but probably better to use mmap, so that if these objects are collected, you can just MADV_DONTNEED them and keep them around for later re-use.

  6. After GC runs, explicitly return the memory for any object in lospace that wasn't visited when the object graph was traversed. Decrement the lospace byte counter and possibly return some empty blocks to the paged space.

There are some interesting aspects about this strategy. One is, the memory that you return to the OS doesn't need to be contiguous. When allocating a 50 MB object, you don't have to find 50 MB of contiguous free space, because any set of blocks that adds up to 50 MB will do.

Another aspect is that this adaptive strategy can work for any ratio of large to non-large objects. The user doesn't have to manually set the sizes of the various spaces.

This strategy does assume that address space is larger than heap size, but only by a factor of 2 (modulo fragmentation for the large object space). Therefore our risk of running afoul of user resource limits and kernel overcommit heuristics is low.

The one underspecified part of this algorithm is... did you see it? "Find some empty blocks". If the main paged space does lazy sweeping -- only scanning a block for holes right before the block will be used for allocation -- then after a collection we don't actually know very much about the heap, and notably, we don't know what blocks are empty. (We could know it, of course, but it would take time; you could traverse the line mark arrays for all blocks while the world is stopped, but this increases pause time. The original Immix collector does this, however.) In the system I've been working on, instead I have it so that if a mutator finds an empty block, it puts it on a separate list, and then takes another block, only allocating into empty blocks once all blocks are swept. If the lospace needs blocks, it sweeps eagerly until it finds enough empty blocks, throwing away any nonempty blocks. This causes the next collection to happen sooner, but that's not a terrible thing; this only occurs when rebalancing lospace versus paged-space size, because if you have a constant allocation rate on the lospace side, you will also have a complementary rate of production of empty blocks by GC, as they are recommissioned when lospace objects are reclaimed.

What if your main paged space has ample space for allocating a large object, but there are no empty blocks, because live objects are equally peppered around all blocks? In that case, often the application would be best served by growing the heap, but maybe not. In any case in a strict-heap-size environment, we need a solution.

But for that... let's pick up another day. Until then, happy hacking!

by Andy Wingo at June 20, 2022 02:59 PM

June 15, 2022

GStreamerGStreamer 1.20.3 stable bug fix release

(GStreamer)

The GStreamer team is pleased to announce the third bug fix release in the stable 1.20 release series of your favourite cross-platform multimedia framework!

This release only contains important security fixes. It should be safe to update from 1.20.x and we recommend you upgrade at your earliest convenience.

Highlighted bugfixes:

  • Security fixes in Matroska, MP4 and AVI demuxers
  • Fix scrambled video playback with hardware-accelerated VA-API decoders on certain Intel hardware
  • playbin3/decodebin3 regression fix for unhandled streams
  • Fragmented MP4 playback fixes
  • Android H.265 encoder mapping
  • Playback of MXF files produced by older FFmpeg versions
  • Fix rtmp2sink crashes on 32-bit platforms
  • WebRTC improvements
  • D3D11 video decoder and screen recorder fixes
  • Performance improvements
  • Support for building against OpenCV 4.6 and other build fixes
  • Miscellaneous bug fixes, memory leak fixes, and other stability and reliability improvements

See the GStreamer 1.20.3 release notes for more details.

Binaries for Android, iOS, macOS and Windows will be available shortly.

Release tarballs can be downloaded directly here:

June 15, 2022 11:00 PM

Andy Wingodefragmentation

(Andy Wingo)

Good morning, hackers! Been a while. It used to be that I had long blocks of uninterrupted time to think and work on projects. Now I have two kids; the longest such time-blocks are on trains (too infrequent, but it happens) and in a less effective but more frequent fashion, after the kids are sleeping. As I start writing this, I'm in an airport waiting for a delayed flight -- my first since the pandemic -- so we can consider this to be the former case.

It is perhaps out of mechanical sympathy that I have been using my reclaimed time to noodle on a garbage collector. Managing space and managing time have similar concerns: how to do much with little, efficiently packing different-sized allocations into a finite resource.

I have been itching to write a GC for years, but the proximate event that pushed me over the edge was reading about the Immix collection algorithm a few months ago.

on fundamentals

Immix is a "mark-region" collection algorithm. I say "algorithm" rather than "collector" because it's more like a strategy or something that you have to put into practice by making a concrete collector, the other fundamental algorithms being copying/evacuation, mark-sweep, and mark-compact.

To build a collector, you might combine a number of spaces that use different strategies. A common choice would be to have a semi-space copying young generation, a mark-sweep old space, and maybe a treadmill large object space (a kind of copying collector, logically; more on that later). Then you have heuristics that determine what object goes where, when.

On the engineering side, there's quite a number of choices to make there too: probably you make some parts of your collector to be parallel, maybe the collector and the mutator (the user program) can run concurrently, and so on. Things get complicated, but the fundamental algorithms are relatively simple, and present interesting fundamental tradeoffs.


figure 1 from the immix paper

For example, mark-compact is most parsimonious regarding space usage -- for a given program, a garbage collector using a mark-compact algorithm will require less memory than one that uses mark-sweep. However, mark-compact algorithms all require at least two passes over the heap: one to identify live objects (mark), and at least one to relocate them (compact). This makes them less efficient in terms of overall program throughput and can also increase latency (GC pause times).

Copying or evacuating spaces can be more CPU-efficient than mark-compact spaces, as reclaiming memory avoids traversing the heap twice; a copying space copies objects as it traverses the live object graph instead of after the traversal (mark phase) is complete. However, a copying space's minimum heap size is quite high, and it only reaches competitive efficiencies at large heap sizes. For example, if your program needs 100 MB of space for its live data, a semi-space copying collector will need at least 200 MB of space in the heap (a 2x multiplier, we say), and will only run efficiently at something more like 4-5x. It's a reasonable tradeoff to make for small spaces such as nurseries, but as a mature space, it's so memory-hungry that users will be unhappy if you make it responsible for a large portion of your memory.

Finally, mark-sweep is quite efficient in terms of program throughput, because like copying it traverses the heap in just one pass, and because it leaves objects in place instead of moving them. But! Unlike the other two fundamental algorithms, mark-sweep leaves the heap in a fragmented state: instead of having all live objects packed into a contiguous block, memory is interspersed with live objects and free space. So the collector can run quickly but the allocator stops and stutters as it accesses disparate regions of memory.

allocators

Collectors are paired with allocators. For mark-compact and copying/evacuation, the allocator consists of a pointer to free space and a limit. Objects are allocated by bumping the allocation pointer, a fast operation that also preserves locality between contemporaneous allocations, improving overall program throughput. But for mark-sweep, we run into a problem: say you go to allocate a 1 kilobyte byte array, do you actually have space for that?

Generally speaking, mark-sweep allocators solve this problem via freelist allocation: the allocator has an array of lists of free objects, one for each "size class" (say 2 words, 3 words, and so on up to 16 words, then more sparsely up to the largest allocatable size maybe), and services allocations from their appropriate size class's freelist. This prevents the 1 kB free space that we need from being "used up" by a 16-byte allocation that could just have well gone elsewhere. However, freelists prevent objects allocated around the same time from being deterministically placed in nearby memory locations. This increases variance and decreases overall throughput for both the allocation operations but also for pointer-chasing in the course of the program's execution.

Also, in a mark-sweep collector, we can still reach a situation where there is enough space on the heap for an allocation, but that free space broken up into too many pieces: the heap is fragmented. For this reason, many systems that perform mark-sweep collection can choose to compact, if heuristics show it might be profitable. Because the usual strategy is mark-sweep, though, they still use freelist allocation.

on immix and mark-region

Mark-region collectors are like mark-sweep collectors, except that they do bump-pointer allocation into the holes between survivor objects.

Sounds simple, right? To my mind, though the fundamental challenge in implementing a mark-region collector is how to handle fragmentation. Let's take a look at how Immix solves this problem.


part of figure 2 from the immix paper

Firstly, Immix partitions the heap into blocks, which might be 32 kB in size or so. No object can span a block. Block size should be chosen to be a nice power-of-two multiple of the system page size, not so small that common object allocations wouldn't fit. Allocating "large" objects -- greater than 8 kB, for Immix -- go to a separate space that is managed in a different way.

Within a block, Immix divides space into lines -- maybe 128 bytes long. Objects can span lines. Any line that does not contain (a part of) an object that survived the previous collection is part of a hole. A hole is a contiguous span of free lines in a block.

On the allocation side, Immix does bump-pointer allocation into holes. If a mutator doesn't have a hole currently, it scans the current block (obtaining one if needed) for the next hole, via a side-table of per-line mark bits: one bit per line. Lines without the mark are in holes. Scanning for holes is fairly cheap, because the line size is not too small. Note, there are also per-object mark bits as well; just because you've marked a line doesn't mean that you've traced all objects on that line.

Allocating into a hole has good expected performance as well, as it's bump-pointer, and the minimum size isn't tiny. In the worst case of a hole consisting of a single line, you have 128 bytes to work with. This size is large enough for the majority of objects, given that most objects are small.

mitigating fragmentation

Immix still has some challenges regarding fragmentation. There is some loss in which a single (piece of an) object can keep a line marked, wasting any free space on that line. Also, when an object can't fit into a hole, any space left in that hole is lost, at least until the next collection. This loss could also occur for the next hole, and the next and the next and so on until Immix finds a hole that's big enough. In a mark-sweep collector with lazy sweeping, these free extents could instead be placed on freelists and used when needed, but in Immix there is no such facility (by design).

One mitigation for fragmentation risks is "overflow allocation": when allocating an object larger than a line (a medium object), and Immix can't find a hole before the end of the block, Immix allocates into a completely free block. So actually mutator threads allocate into two blocks at a time: one for small objects and medium objects if possible, and the other for medium objects when necessary.

Another mitigation is that large objects are allocated into their own space, so an Immix space will never be used for blocks larger than, say, 8kB.

The other mitigation is that Immix can choose to evacuate instead of mark. How does this work? Is it worth it?

stw

This question about the practical tradeoffs involving evacuation is the one I wanted to pose when I started this article; I have gotten to the point of implementing this part of Immix and I have some doubts. But, this article is long enough, and my plane is about to land, so let's revisit this on my return flight. Until then, see you later, allocators!

by Andy Wingo at June 15, 2022 12:47 PM

June 11, 2022

Sebastian PölsterlUsing VS Code and Podman to Develop SYCL Applications With DPC++'s CUDA Backend

I recently wanted to create a development container for VS Code to develop applications using SYCL based on the CUDA backend of the oneAPI DPC++ (Data Parallel C++) compiler. As I’m running Fedora, it seemed natural to use Podman’s rootless containers instead of Docker for this. This turned out to be more challenging than expected, so I’m going to summarize my setup in this post. I’m using Fedora Linux 36 with Podman version 4.1.0.

Prerequisites

Since the DPC++ is going to use CUDA behind the scene, you will need an NVIDIA GPU and the corresponding Kernel driver for it. I’ve been using the NVIDIA GPU driver from RPM Fusion. Note that you do not have to install CUDA, it is part of the development container alongside the DPC++ compiler.

Next, you require Podman, which on Fedora can be installed by executing

sudo dnf install -y podman

Finally, you require VS Code and the Remote - Containers extension. Just follow the instructions behind those links.

Installing and Configuring the NVIDIA Container Toolkit

The default configuration of the NVIDIA Container Toolkit does not work with Podman, so it needs to be adjusted. Most steps are based on this guide by Red Hat, which I will repeat below.

  1. Add the repository:

    curl -sL https://nvidia.github.io/nvidia-docker/rhel9.0/nvidia-docker.repo | sudo tee /etc/yum.repos.d/nvidia-docker.repo
    

    If you aren’t using Fedora 36, you might have to replace rhel9.0 with your distribution, see the instructions.

  2. Next, install the nvidia-container-toolkit package.

    sudo dnf install -y nvidia-container-toolkit
    
  3. Red Hat’s guide mentioned to configure two settings in /etc/nvidia-container-runtime/config.toml. But when using Podman with --userns=keep-id to map the UID of the user running the container to the user running inside the container, you have to change a third setting. So open /etc/nvidia-container-runtime/config.toml with

    sudo -e /etc/nvidia-container-runtime/config.toml
    

    and change the following three lines:

    #no-cgroups = false
    no-cgroups = true
    
    #ldconfig = "@/sbin/ldconfig"
    ldconfig = "/sbin/ldconfig"
    
    #debug = "/var/log/nvidia-container-runtime.log"
    debug = "~/.local/nvidia-container-runtime.log"
    
  4. Next, you have to create a new SELinux policy to enable GPU access within the container:

    curl -sLO https://raw.githubusercontent.com/NVIDIA/dgx-selinux/master/bin/RHEL7/nvidia-container.pp
    sudo semodule -i nvidia-container.pp
    sudo nvidia-container-cli -k list | sudo restorecon -v -f -
    sudo restorecon -Rv /dev
    
  5. Finally, tell VS Code to use Podman instead of Docker by going to User SettingsExtensionsRemote - Containers, and change Remote › Containers: Docker Path to podman, as in the image below.

Docker Path setting

Replace docker with podman.

Using the Development Container

I created an example project that is based on a container that provides

You can install additional tools by editing the project’s Dockerfile.

To use the example project, clone it:

git clone https://github.com/sebp/vscode-sycl-dpcpp-cuda.git

Next, open the vscode-sycl-dpcpp-cuda directory with VS Code. At this point VS Code should recognize that the project contains a development container and suggest reopening the project in the container

Reopen project in container

Click Reopen in Container.

Initially, this step will take some time because the container’s image is downloaded and VS Code will install additional tools inside the container. Subsequently, VS Code will reuse this container, and opening the project in the container will be quick.

Once the project has been opened within the container, you can open the example SYCL application in the file src/sycl-example.cpp. The project is configured to use the DPC++ compiler with the CUDA backend by default. Therefore, you just have to press Ctrl+Shift+B to compile the example file. Using the terminal, you can now execute the compiled program, which should print the GPU it is using and the numbers 0 to 31.

Alternatively, you can compile and directly run the program by executing the Test task by opening the Command Palette (F1, Ctrl+Shift+P) and searching for Run Test Task.

Conclusion

While the journey to use a rootless Podman container with access to the GPU with VS Code was rather cumbersome, I hope this guide will make it less painful for others. The example project should provide a good reference for a devcontainer.json to use rootless Podman containers with GPU access. If you aren’t interested in SYCL or DPC++, you can replace the exising Dockerfile. There are two steps that are essential for this to work:

  1. Create a vscode user inside the container.
  2. Make sure you create certain directories that VS Code (inside the container) will require access to.

Otherwise, you will encounter various permission denied errors.

June 11, 2022 12:33 PM

June 10, 2022

Christian SchallerHow to get your application to show up in GNOME Software

(Christian Schaller)

Adding Applications to the GNOME Software Center

Written by Richard Hughes and Christian F.K. Schaller

This blog post is based on a white paper style writeup Richard and I did a few years ago, since I noticed this week there wasn’t any other comprehensive writeup online on the topic of how to add the required metadata to get an application to appear in GNOME Software (or any other major open source appstore) online I decided to turn the writeup into a blog post, hopefully useful to the wider community. I tried to clean it up a bit as I converted it from the old white paper, so hopefully all information in here is valid as of this posting.

Abstract

Traditionally we have had little information about Linux applications before they have been installed. With the creation of a software center we require access to rich set of metadata about an application before it is deployed so it it can be displayed to the user and easily installed. This document is meant to be a guide for developers who wish to get their software appearing in the Software stores in Fedora Workstation and other distributions. Without the metadata described in this document your application is likely to go undiscovered by many or most linux users, but by reading this document you should be able to relatively quickly prepare you application.

Introduction

GNOME Software

Installing applications on Linux has traditionally involved copying binary and data files into a directory and just writing a single desktop file into a per-user or per-system directory so that it shows up in the desktop environment. In this document we refer to applications as graphical programs, rather than other system add-on components like drivers and codecs. This document will explain why the extra metadata is required and what is required for an application to be visible in the software center. We will try to document how to do this regardless of if you choose to package your application as a rpm package or as a flatpak bundle. The current rules is a combination of various standards that have evolved over the years and will will try to summarize and explain them here, going from bottom to top.

System Architecture

Linux File Hierarchy

Traditionally applications on Linux are expected to install binary files to /usr/bin, the install architecture independent data files to /usr/share/ and configuration files to /etc. If you want to package your application as a flatpak the prefix used will be /app so it is critical for applications to respect the prefix setting. Small temporary files can be stored in /tmp and much larger files in /var/tmp. Per-user configuration is either stored in the users home directory (in ~/.config) or stored in a binary settings store such as dconf. As an application developer never hardcode these paths, but set them following the XDG standard so that they relocate correctly inside a Flatpak.

Desktop files

Desktop files have been around for a long while now and are used by almost all Linux desktops to provide the basic description of a desktop application that your desktop environment will display. Like a human readable name and an icon.

So the creation of a desktop file on Linux allows a program to be visible to the graphical environment, e.g. KDE or GNOME Shell. If applications do not have a desktop file they must be manually launched using a terminal emulator. Desktop files must adhere to the Desktop File Specification and provide metadata in an ini-style format such as:

  • Binary type, typically ‘Application’
  • Program name (optionally localized)
  • Icon to use in the desktop shell
  • Program binary name to use for launching
  • Any mime types that can be opened by the applications (optional)
  • The standard categories the application should be included in (optional)
  • Keywords (optional, and optionally localized)
  • Short one-line summary (optional, and optionally localized)

The desktop file should be installed into /usr/share/applications for applications that are installed system wide. An example desktop file provided below:


[Desktop Entry]
Type=Application
Name=OpenSCAD
Icon=openscad
Exec=openscad %f
MimeType=application/x-openscad;
Categories=Graphics;3DGraphics;Engineering;
Keywords=3d;solid;geometry;csg;model;stl;

The desktop files are used when creating the software center metadata, and so you should verify that you ship a .desktop file for each built application, and that these keys exist: Name, Comment, Icon, Categories, Keywords and Exec and that desktop-file-validate correctly validates the file. There should also be only one desktop file for each application.

The application icon should be in the PNG format with a transparent background and installed in
/usr/share/icons,/usr/share/icons/hicolor//apps/, or /usr/share/${app_name}/icons/*. The icon should be at least 128×128 in size (as this is the minimum size required by Flathub).

The file name of the desktop file is also very important, as this is the assigned ‘application ID’. New applications typically use a reverse-DNS style, e.g. org.gnome.Nautilus would be the app-id. And the .desktop entry file should thus be name org.gnome.Nautilus.desktop, but older programs may just use a short name, e.g. gimp.desktop. It is important to note that the file extension is also included as part of the desktop ID.

You can verify your desktop file using the command ‘desktop-file-validate’. You just run it like this:


desktop-file-validate myapp.desktop

This tools is available through the desktop-file-utils package, which you can install on Fedora Workstation using this command


dnf install desktop-file-utils

You also need what is called a metainfo file (previously known as AppData file= file with the suffix .metainfo.xml (some applications still use the older .appdata.xml name) file should be installed into /usr/share/metainfo with a name that matches the name of the .desktop file, e.g. gimp.desktop & gimp.metainfo.xml or org.gnome.Nautilus.desktop & org.gnome.Nautilus.metainfo.xml.

In the metainfo file you should include several 16:9 aspect screenshots along with a compelling translated description made up of multiple paragraphs.

In order to make it easier for you to do screenshots in 16:9 format we created a small GNOME Shell extension called ‘Screenshot Window Sizer’. You can install it from the GNOME Extensions site.

Once it is installed you can resize the window of your application to 16:9 format by focusing it and pressing ‘ctrl+alt+s’ (you can press the key combo multiple times to get the correct size). It should resize your application window to a perfect 16:9 aspect ratio and let you screenshot it.

Make sure you follow the style guide, which can be tested using the appstreamcli command line tool. appstreamcli is part of the ‘appstream’ package in Fedora Workstation.:


appstreamcli validate foo.metainfo.xml

If you don’t already have the appstreamcli installed it can be installed using this command on Fedora Workstation:

dnf install appstream

What is allowed in an metainfo file is defined in the AppStream specification but common items typical applications add is:

  • License of the upstream project in SPDX identifier format [6], or ‘Proprietary’
  • A translated name and short description to show in the software center search results
  • A translated long description, consisting of multiple paragraphs, itemized and ordered lists.
  • A number of screenshots, with localized captions, typically in 16:9 aspect ratio
  • An optional list of releases with the update details and release information.
  • An optional list of kudos which tells the software center about the integration level of the
    application
  • A set of URLs that allow the software center to provide links to help or bug information
  • Content ratings and hardware compatibility
  • An optional gettext or QT translation domain which allows the AppStream generator to collect statistics on shipped application translations.

A typical (albeit somewhat truncated) metainfo file is shown below:

<?xml version="1.0" encoding="UTF-8"?>
<component type="desktop-application">
<id>org.gnome.Terminal</id>
<metadata_license>GPL-3.0+ or GFDL-1.3-only</metadata_license>
<project_license>GPL-3.0+</project_license>
<name>Terminal</name>
<name xml:lang="ar">الطرفية</name>
<name xml:lang="an">Terminal</name>
<summary>Use the command line</summary>
<summary xml:lang="ar">استعمل سطر الأوامر</summary>
<summary xml:lang="an">Emplega la linia de comandos</summary>
<description>
<p>GNOME Terminal is a terminal emulator application for accessing a UNIX shell environment which can be used to run programs available on your system.</p>
<p xml:lang="ar">يدعم تشكيلات مختلفة، و الألسنة و العديد من اختصارات لوحة المفاتيح.</p>
<p xml:lang="an">Suporta quantos perfils, quantas pestanyas y implementa quantos alcorces de teclau.</p>
</description>
<recommends>
<control>console</control>
<control>keyboard</control>
<control>pointing</control>
</recommends>
<screenshots>
<screenshot type="default">https://help.gnome.org/users/gnome-terminal/stable/figures/gnome-terminal.png</screenshot>
</screenshots>
<kudos>
<kudo>HiDpiIcon</kudo>
<kudo>HighContrast</kudo>
<kudo>ModernToolkit</kudo>
<kudo>SearchProvider</kudo>
<kudo>UserDocs</kudo>
</kudos>
<content_rating type="oars-1.1"/>
<url type="homepage">https://wiki.gnome.org/Apps/Terminal</url>
<project_group>GNOME</project_group>
<update_contact>https://wiki.gnome.org/Apps/Terminal/ReportingBugs</update_contact>
</component>

Some Appstrean background

The Appstream specification is an mature and evolving standard that allows upstream applications to provide metadata such as localized descriptions, screenshots, extra keywords and content ratings for parental control. This intoduction just touches on the surface what it provides so I recommend reading the specification through once you understood the basics. The core concept is that the upstream project ships one extra metainfo XML file which is used to build a global application catalog called a metainfo file. Thousands of open source projects now include metainfo files, and the software center shipped in Fedora, Ubuntu and OpenSuse is now an easy to use application filled with useful application metadata. Applications without metainfo files are no longer shown which provides quite some incentive to upstream projects wanting visibility in popular desktop environments. AppStream was first introduced in 2008 and since then many people have contributed to the specification. It is being used primarily for application metadata but also now is used for drivers, firmware, input methods and fonts. There are multiple projects producing AppStream metadata and also a number of projects consuming the final XML metadata.

When applications are being built as packages by a distribution then the AppStream generation is done automatically, and you do not need to do anything other than installing a .desktop file and an metainfo.xml file in the upstream tarball or zip file. If the application is being built on your own machines or cloud instance then the distributor will need to generate the AppStream metadata manually. This would for example be the case when internal-only or closed source software is being either used or produced. This document assumes you are currently building RPM packages and exporting yum-style repository metadata for Fedora or RHEL although the concepts are the same for rpm-on-OpenSuse or deb-on-Ubuntu.

NOTE: If you are building packages, make sure that there are not two applications installed with one single package. If this is currently the case split up the package so that there are multiple subpackages or mark one of the .desktop files as NoDisplay=true. Make sure the application-subpackages depend on any -common subpackage and deal with upgrades (perhaps using a metapackage) if you’ve shipped the application before.

Summary of Package building

So the steps outlined above explains the extra metadata you need to have your application show up in GNOME Software. This tutorial does not cover how to set up your build system to build these, but both for Meson and autotools you should be able to find a long range of examples online. And there are also major resources available to explain how to create a Fedora RPM or how to build a Flatpak. You probably also want to tie both the Desktop file and the metainfo file into your i18n system so the metadata in them can be translated. It is worth nothing here that while this document explains how you can do everything yourself we do generally recommend relying on existing community infrastructure for hosting source code and packages if you can (for instance if your application is open source), as they will save you work and effort over time. For instance putting your source code into the GNOME git will give you free access to the translator community in GNOME and thus increase the chance your application is internationalized significantly. And by building your package in Fedora you can get peer review of your package and free hosting of the resulting package. Or by putting your package up on Flathub you get wide cross distribution availability.

Setting up hosting infrastructure for your package

We will here explain how you set up a Yum repository for RPM packages that provides the needed metadata. If you are making a Flatpak we recommend skipping ahead to the Flatpak section a bit further down.

Yum hosting and Metadata:

When GNOME Software checks for updates it downloads various metadata files from the server describing the packages available in the repository. GNOME Software can also download AppStream metadata at the same time, allowing add-on repositories to include applications that are visible in the the software center. In most cases distributors are already building binary RPMS and then building metadata as an additional step by running something like this to generate the repomd files on a directory of packages. The tool for creating the repository metadata is called createrepo_c and is part of the package createrepo_c in Fedora. You can install it by running the command:


dnf install createrepo_c.

Once the tool is installed you can run these commands to generate your metadata:


$ createrepo_c --no-database --simple-md-filenames SRPMS/
$ createrepo_c --no-database --simple-md-filenames x86_64/

This creates the primary and filelist metadata required for updating on the command line. Next to build the metadata required for the software center we we need to actually generate the AppStream XML. The tool you need for this is called appstream-builder. This works by decompressing .rpm files and merging together the .desktop file, the .metainfo.xml file and preprocessing the icons. Remember, only applications installing AppData files will be included in the metadata.

You can install appstream builder in Fedora Workstation by using this command:

dnf install libappstream-glib-builder

Once it is installed you can run it by using the following syntax:

$ appstream-builder \
   --origin=yourcompanyname \
   --basename=appstream \
   --cache-dir=/tmp/asb-cache \
   --enable-hidpi \
   --max-threads=1 \
   --min-icon-size=32 \
   --output-dir=/tmp/asb-md \
   --packages-dir=x86_64/ \
   --temp-dir=/tmp/asb-icons

This takes a few minutes and generates some files to the output directory. Your output should look something like this:


Scanning packages...
Processing packages...
Merging applications...
Writing /tmp/asb-md/appstream.xml.gz...
Writing /tmp/asb-md/appstream-icons.tar.gz...
Writing /tmp/asb-md/appstream-screenshots.tar...Done!

The actual build output will depend on your compose server configuration. At this point you can also verify the application is visible in the yourcompanyname.xml.gz file.
We then have to take the generated XML and the tarball of icons and add it to the repomd.xml master document so that GNOME Software automatically downloads the content for searching.
This is as simple as doing:

modifyrepo_c \
    --no-compress \
    --simple-md-filenames \
    /tmp/asb-md/appstream.xml.gz \
    x86_64/repodata/
modifyrepo_c \
    --no-compress \
    --simple-md-filenames \
    /tmp/asb-md/appstream-icons.tar.gz \
    x86_64/repodata/

 

Deploying this metadata will allow GNOME Software to add the application metadata the next time the repository is refreshed, typically, once per day. Hosting your Yum repository on Github Github isn’t really set up for hosting Yum repositories, but here is a method that currently works. So once you created a local copy of your repository create a new project on github. Then use the follow commands to import your repository into github.


cd ~/src/myrepository
git init
git add -A
git commit -a -m "first commit"
git remote add origin git@github.com:yourgitaccount/myrepo.git
git push -u origin master

Once everything is important go into the github web interface and drill down in the file tree until you find the file called ‘repomd.xml’ and click on it. You should now see a button the github interface called ‘Raw’. Once you click that you get the raw version of the XML file and in the URL bar of your browser you should see a URL looking something like this:
https://raw.githubusercontent.com/cschalle/hubyum/master/noarch/repodata/repomd.xml
Copy that URL as you will need the information from it to create your .repo file which is what distributions and users want in order to reach you new repository. To create your .repo file copy this example and edit it to match your data:


[remarkable]
name=Remarkable Markdown editor software and updates
baseurl=https://raw.githubusercontent.com/cschalle/hubyum/master/noarch
gpgcheck=0
enabled=1
enabled_metadata=1

So on top is your Repo shortname inside the brackets, then a name field with a more extensive name. For the baseurl paste the URL you copied earlier and remove the last bits until you are left with either the ‘norach’ directory or your platform directory for instance x86_64. Once you have that file completed put it into /etc/yum.repos.d on your computer and load up GNOME Software. Click on the ‘Updates’ button in GNOME Software and then on the refresh button in the top left corner to ensure your database is up to date. If everything works as expected you should then be able to do a search in GNOME software and find your new application showing up.

Example of self hosted RPM

Flapak hosting and Metadata

The flatpak-builder binary generates AppStream metadata automatically when building applications if the appstream-compose tool is installed on the flatpak build machine. Flatpak remotes are exported with a separate ‘appstream’ branch which is automatically downloaded by GNOME Software and no addition work if required when building your application or updating the remote. Adding the remote is enough to add the application to the software center, on the assumption the AppData file is valid.

Conclusions

AppStream files allow us to build a modern software center experience either using distro packages with yum-style metadata or with the new flatpak application deployment framework. By including a desktop file and AppData file for your Linux binary build your application can be easily found and installed by end users greatly expanding its userbase.

by uraeus at June 10, 2022 02:12 PM

May 11, 2022

Christian SchallerWhy the open source driver release from NVIDIA is so important for Linux?

(Christian Schaller)

Background
Today NVIDIA announced that they are releasing an open source kernel driver for their GPUs, so I want to share with you some background information and how I think this will impact Linux graphics and compute going forward.

One thing many people are not aware of is that Red Hat is the only Linux OS company who has a strong presence in the Linux compute and graphics engineering space. There are of course a lot of other people working in the space too, like engineers working for Intel, AMD and NVIDIA or people working for consultancy companies like Collabora or individual community members, but Red Hat as an OS integration company has been very active on trying to ensure we have a maintainable and shared upstream open source stack. This engineering presence is also what has allowed us to move important technologies forward, like getting hiDPI support for Linux some years ago, or working with NVIDIA to get glvnd implemented to remove a pain point for our users since the original OpenGL design only allowed for one OpenGl implementation to be installed at a time. We see ourselves as the open source community’s partner here, fighting to keep the linux graphics stack coherent and maintainable and as a partner for the hardware OEMs to work with when they need help pushing major new initiatives around GPUs for Linux forward. And as the only linux vendor with a significant engineering footprint in GPUs we have been working closely with NVIDIA. People like Kevin Martin, the manager for our GPU technologies team, Ben Skeggs the maintainer of Nouveau and Dave Airlie, the upstream kernel maintainer for the graphics subsystem, Nouveau contributor Karol Herbst and our accelerator lead Tom Rix have all taken part in meetings, code reviews and discussions with NVIDIA. So let me talk a little about what this release means (and also what it doesn’t mean) and what we hope to see come out of this long term.

First of all, what is in this new driver?
What has been released is an out of tree source code kernel driver which has been tested to support CUDA usecases on datacenter GPUs. There is code in there to support display, but it is not complete or fully tested yet. Also this is only the kernel part, a big part of a modern graphics driver are to be found in the firmware and userspace components and those are still closed source. But it does mean we have a NVIDIA kernel driver now that will start being able to consume the GPL-only APIs in the linux kernel, although this initial release doesn’t consume any APIs the old driver wasn’t already using. The driver also only supports NVIDIA Turing chip GPUs and newer, which means it is not targeting GPUs from before 2018. So for the average Linux desktop user, while this is a great first step and hopefully a sign of what is to come, it is not something you are going to start using tomorrow.

What does it mean for the NVidia binary driver?
Not too much immediately. This binary kernel driver will continue to be needed for older pre-Turing NVIDIA GPUs and until the open source kernel module is full tested and extended for display usecases you are likely to continue using it for your system even if you are on Turing or newer. Also as mentioned above regarding firmware and userspace bits and the binary driver is going to continue to be around even once the open source kernel driver is fully capable.

What does it mean for Nouveau?
Let me start with the obvious, this is actually great news for the Nouveau community and the Nouveau driver and NVIDIA has done a great favour to the open source graphics community with this release. And for those unfamiliar with Nouveau, Nouveau is the in-kernel graphics driver for NVIDIA GPUs today which was originally developed as a reverse engineered driver, but which over recent years actually have had active support from NVIDIA. It is fully functional, but is severely hampered by not having had the ability to for instance re-clock the NVIDIA card, meaning that it can’t give you full performance like the binary driver can. This was something we were working with NVIDIA trying to remedy, but this new release provides us with a better path forward. So what does this new driver mean for Nouveau? Less initially, but a lot in the long run. To give a little background first. The linux kernel does not allow multiple drivers for the same hardware, so in order for a new NVIDIA kernel driver to go in the current one will have to go out or at least be limited to a different set of hardware. The current one is Nouveau. And just like the binary driver a big chunk of Nouveau is not in the kernel, but are the userspace pieces found in Mesa and the Nouveau specific firmware that NVIDIA currently kindly makes available. So regardless of the long term effort to create a new open source in-tree kernel driver based on this new open source driver for NVIDIA hardware, Nouveau will very likely be staying around to support pre-turing hardware just like the NVIDIA binary kernel driver will.

The plan we are working towards from our side, but which is likely to take a few years to come to full fruition, is to come up with a way for the NVIDIA binary driver and Mesa to share a kernel driver. The details of how we will do that is something we are still working on and discussing with our friends at NVIDIA to address both the needs of the NVIDIA userspace and the needs of the Mesa userspace. Along with that evolution we hope to work with NVIDIA engineers to refactor the userspace bits of Mesa that are now targeting just Nouveau to be able to interact with this new kernel driver and also work so that the binary driver and Nouveau can share the same firmware. This has clear advantages for both the open source community and the NVIDIA. For the open source community it means that we will now have a kernel driver and firmware that allows things like changing the clocking of the GPU to provide the kind of performance people expect from the NVIDIA graphics card and it means that we will have an open source driver that will have access to the firmware and kernel updates from day one for new generations of NVIDIA hardware. For the ‘binary’ driver, and I put that in ” signs because it will now be less binary :), it means as stated above that it can start taking advantage of the GPL-only APIs in the kernel, distros can ship it and enable secure boot, and it gets an open source consumer of its kernel driver allowing it to go upstream.
If this new shared kernel driver will be known as Nouveau or something completely different is still an open question, and of course it happening at all depends on if we and the rest of the open source community and NVIDIA are able to find a path together to make it happen, but so far everyone seems to be of good will.

What does this release mean for linux distributions like Fedora and RHEL?

Over time it provides a pathway to radically simplify supporting NVIDIA hardware due to the opportunities discussed elsewhere in this document. Long term we will hope be able to get a better user experience with NVIDIA hardware in terms out of box functionality. Which means day 1 support for new chipsets, a high performance open source Mesa driver for NVIDIA and it will allow us to sign the NVIDIA driver alongside the rest of the kernel to enable things like secureboot support. Since this first release is targeting compute one can expect that these options will first be available for compute users and then graphics at a later time.

What are the next steps
Well there is a lot of work to do here. NVIDIA need to continue the effort to make this new driver feature complete for both Compute and Graphics Display usecases, we’d like to work together to come up with a plan for what the future unified kernel driver can look like and a model around it that works for both the community and NVIDIA, we need to add things like a Mesa Vulkan driver. We at Red Hat will be playing an active part in this work as the only Linux vendor with the capacity to do so and we will also work to ensure that the wider open source community has a chance to participate fully like we do for all open source efforts we are part of.

If you want to hear more about this I did talk with Chris Fisher and Linux Action News about this topic. Note: I did state some timelines in that interview which I didn’t make clear was my guesstimates and not in any form official NVIDIA timelines, so apologize for the confusion.

by uraeus at May 11, 2022 07:52 PM

May 09, 2022

Robert McQueenEvolving a strategy for 2022 and beyond

(Robert McQueen)

As a board, we have been working on several initiatives to make the Foundation a better asset for the GNOME Project. We’re working on a number of threads in parallel, so I wanted to explain the “big picture” a bit more to try and connect together things like the new ED search and the bylaw changes.

We’re all here to see free and open source software succeed and thrive, so that people can be be truly empowered with agency over their technology, rather than being passive consumers. We want to bring GNOME to as many people as possible so that they have computing devices that they can inspect, trust, share and learn from.

In previous years we’ve tried to boost the relevance of GNOME (or technologies such as GTK) or solicit donations from businesses and individuals with existing engagement in FOSS ideology and technology. The problem with this approach is that we’re mostly addressing people and organisations who are already supporting or contributing FOSS in some way. To truly scale our impact, we need to look to the outside world, build better awareness of GNOME outside of our current user base, and find opportunities to secure funding to invest back into the GNOME project.

The Foundation supports the GNOME project with infrastructure, arranging conferences, sponsoring hackfests and travel, design work, legal support, managing sponsorships, advisory board, being the fiscal sponsor of GNOME, GTK, Flathub… and we will keep doing all of these things. What we’re talking about here are additional ways for the Foundation to support the GNOME project – we want to go beyond these activities, and invest into GNOME to grow its adoption amongst people who need it. This has a cost, and that means in parallel with these initiatives, we need to find partners to fund this work.

Neil has previously talked about themes such as education, advocacy, privacy, but we’ve not previously translated these into clear specific initiatives that we would establish in addition to the Foundation’s existing work. This is all a work in progress and we welcome any feedback from the community about refining these ideas, but here are the current strategic initiatives the board is working on. We’ve been thinking about growing our community by encouraging and retaining diverse contributors, and addressing evolving computing needs which aren’t currently well served on the desktop.

Initiative 1. Welcoming newcomers. The community is already spending a lot of time welcoming newcomers and teaching them the best practices. Those activities are as time consuming as they are important, but currently a handful of individuals are running initiatives such as GSoC, Outreachy and outreach to Universities. These activities help bring diverse individuals and perspectives into the community, and helps them develop skills and experience of collaborating to create Open Source projects. We want to make those efforts more sustainable by finding sponsors for these activities. With funding, we can hire people to dedicate their time to operating these programs, including paid mentors and creating materials to support newcomers in future, such as developer documentation, examples and tutorials. This is the initiative that needs to be refined the most before we can turn it into something real.

Initiative 2: Diverse and sustainable Linux app ecosystem. I spoke at the Linux App Summit about the work that GNOME and Endless has been supporting in Flathub, but this is an example of something which has a great overlap between commercial, technical and mission-based advantages. The key goal here is to improve the financial sustainability of participating in our community, which in turn has an impact on the diversity of who we can expect to afford to enter and remain in our community. We believe the existence of this is critically important for individual developers and contributors to unlock earning potential from our ecosystem, through donations or app sales. In turn, a healthy app ecosystem also improves the usefulness of the Linux desktop as a whole for potential users. We believe that we can build a case for commercial vendors in the space to join an advisory board alongside with GNOME, KDE, etc to input into the governance and contribute to the costs of growing Flathub.

Initiative 3: Local-first applications for the GNOME desktop. This is what Thib has been starting to discuss on Discourse, in this thread. There are many different threats to free access to computing and information in today’s world. The GNOME desktop and apps need to give users convenient and reliable access to technology which works similarly to the tools they already use everyday, but keeps them and their data safe from surveillance, censorship, filtering or just being completely cut off from the Internet. We believe that we can seek both philanthropic and grant funding for this work. It will make GNOME a more appealing and comprehensive offering for the many people who want to protect their privacy.

The idea is that these initiatives all sit on the boundary between the GNOME community and the outside world. If the Foundation can grow and deliver these kinds of projects, we are reaching to new people, new contributors and new funding. These contributions and investments back into GNOME represent a true “win-win” for the newcomers and our existing community.

(Originally posted to GNOME Discourse, please feel free to join the discussion there.)

by ramcq at May 09, 2022 02:01 PM

May 02, 2022

GStreamerGStreamer 1.20.2 stable bug fix release

(GStreamer)

The GStreamer team is pleased to announce the second bug fix release in the stable 1.20 release series of your favourite cross-platform multimedia framework!

This release only contains bugfixes and it should be safe to update from 1.20.x.

Highlighted bugfixes:

  • avviddec: Remove vc1/wmv3 override and fix crashes on WMV files with FFMPEG 5.0+
  • macOS: fix plugin discovery for GStreamer installed via brew and fix loading of Rust plugins
  • rtpbasepayload: various header extension handling fixes
  • rtpopusdepay: fix regression in stereo input handling if sprop-stereo is not advertised
  • rtspclientsink: fix possible shutdown deadlock
  • mpegts: gracefully handle "empty" program maps and fix AC-4 detection
  • mxfdemux: Handle empty VANC packets and fix EOS handling
  • playbin3: various playbin3, uridecodebin3, and playsink fixes
  • ptpclock: fix initial sync-up with certain devices
  • gltransformation: let graphene alloc its structures memory aligned
  • webrtcbin fixes and webrtc sendrecv example improvements
  • video4linux2: various fixes including some fixes for Raspberry Pi users
  • videorate segment handling fixes and other fixes
  • nvh264dec, nvh265dec: Fix broken key-unit trick modes and reverse playback
  • wpe: Reintroduce persistent WebContext
  • cerbero: Make it easier to consume 1.20.1 macOS GStreamer .pkgs
  • build fixes and gobject annotation fixes
  • bug fixes, security fixes, memory leak fixes, and other stability and reliability improvements

See the GStreamer 1.20.2 release notes for more details.

Binaries for Android, iOS, macOS and Windows will be available shortly.

Release tarballs can be downloaded directly here:

May 02, 2022 11:30 PM

Sebastian DrögeInstantaneous RTP synchronization & retrieval of absolute sender clock times with GStreamer

(Sebastian Dröge)

Over the last few weeks, GStreamer’s RTP stack got a couple of new and quite useful features. As it is difficult to configure, mostly because there being so many different possible configurations, I decided to write about this a bit with some example code.

The features are RFC 6051-style rapid synchronization of RTP streams, which can be used for inter-stream (e.g. audio/video) synchronization as well as inter-device (i.e. network) synchronization, and the ability to easily retrieve absolute sender clock times per packet on the receiver side.

Note that each of this was already possible before with GStreamer via different mechanisms with different trade-offs. Obviously, not being able to have working audio/video synchronization would be simply not acceptable and I previously talked about how to do inter-device synchronization with GStreamer before, for example at the GStreamer Conference 2015 in Düsseldorf.

The example code below will make use of the GStreamer RTSP Server library but can be applied to any kind of RTP workflow, including WebRTC, and are written in Rust but the same can also be achieved in any other language. The full code can be found in this repository.

And for reference, the merge requests to enable all this are [1], [2] and [3]. You probably don’t want to backport those to an older version of GStreamer though as there are dependencies on various other changes elsewhere. All of the following needs at least GStreamer from the git main branch as of today, or the upcoming 1.22 release.

Baseline Sender / Receiver Code

The starting point of the example code can be found here in the baseline branch. All the important steps are commented so it should be relatively self-explanatory.

Sender

The sender is starting an RTSP server on the local machine on port 8554 and provides a media with H264 video and Opus audio on the mount point /test. It can be started with

$ cargo run -p rtp-rapid-sync-example-send

After starting the server it can be accessed via GStreamer with e.g. gst-play-1.0 rtsp://127.0.0.1:8554/test or similarly via VLC or any other software that supports RTSP.

This does not do anything special yet but lays the foundation for the following steps. It creates an RTSP server instance with a custom RTSP media factory, which in turn creates custom RTSP media instances. All this is not needed at this point yet but will allow for the necessary customization later.

One important aspect here is that the base time of the media’s pipeline is set to zero

pipeline.set_base_time(gst::ClockTime::ZERO);
pipeline.set_start_time(gst::ClockTime::NONE);

This allows the timeoverlay element that is placed in the video part of the pipeline to render the clock time over the video frames. We’re going to use this later to confirm on the receiver that the clock time on the sender and the one retrieved on the receiver are the same.

let video_overlay = gst::ElementFactory::make("timeoverlay", None)
    .context("Creating timeoverlay")?;
[...]
video_overlay.set_property_from_str("time-mode", "running-time");

It actually only supports rendering the running time of each buffer, but in a live pipeline with the base time set to zero the running time and pipeline clock time are the same. See the documentation for some more details about the time concepts in GStreamer.

Overall this creates the following RTSP stream producer bin, which will be used also in all the following steps:

Receiver

The receiver is a simple playbin pipeline that plays an RTSP URI given via command-line parameters and runs until the stream is finished or an error has happened.

It can be run with the following once the sender is started

$ cargo run -p rtp-rapid-sync-example-send -- "rtsp://192.168.1.101:8554/test"

Please don’t forget to replace the IP with the IP of the machine that is actually running the server.

All the code should be familiar to anyone who ever wrote a GStreamer application in Rust, except for one part that might need a bit more explanation

pipeline.connect_closure(
    "source-setup",
    false,
    glib::closure!(|_playbin: &gst::Pipeline, source: &gst::Element| {
        source.set_property("latency", 40u32);
    }),
);

playbin is going to create an rtspsrc, and at that point it will emit the source-setup signal so that the application can do any additional configuration of the source element. Here we’re connecting a signal handler to that signal to do exactly that.

By default rtspsrc introduces a latency of 2 seconds of latency, which is a lot more than what is usually needed. For live, non-VOD RTSP streams this value should be around the network jitter and here we’re configuring that to 40 milliseconds.

Retrieval of absolute sender clock times

Now as the first step we’re going to retrieve the absolute sender clock times for each video frame on the receiver. They will be rendered by the receiver at the bottom of each video frame and will also be printed to stdout. The changes between the previous version of the code and this version can be seen here and the final code here in the sender-clock-time-retrieval branch.

When running the sender and receiver as before, the video from the receiver should look similar to the following

The upper time that is rendered on the video frames is rendered by the sender, the bottom time is rendered by the receiver and both should always be the same unless something is broken here. Both times are the pipeline clock time when the sender created/captured the video frame.

In this configuration the absolute clock times of the sender are provided to the receiver via the NTP / RTP timestamp mapping provided by the RTCP Sender Reports. That’s also the reason why it takes about 5s for the receiver to know the sender’s clock time as RTCP packets are not scheduled very often and only after about 5s by default. The RTCP interval can be configured on rtpbin together with many other things.

Sender

On the sender-side the configuration changes are rather small and not even absolutely necessary.

rtpbin.set_property_from_str("ntp-time-source", "clock-time");

By default the RTP NTP time used in the RTCP packets is based on the local machine’s walltime clock converted to the NTP epoch. While this works fine, this is not the clock that is used for synchronizing the media and as such there will be drift between the RTP timestamps of the media and the NTP time from the RTCP packets, which will be reset every time the receiver receives a new RTCP Sender Report from the sender.

Instead, we configure rtpbin here to use the pipeline clock as the source for the NTP timestamps used in the RTCP Sender Reports. This doesn’t give us (by default at least, see later) an actual NTP timestamp but it doesn’t have the drift problem mentioned before. Without further configuration, in this pipeline the used clock is the monotonic system clock.

rtpbin.set_property("rtcp-sync-send-time", false);

rtpbin normally uses the time when a packet is sent out for the NTP / RTP timestamp mapping in the RTCP Sender Reports. This is changed with this property to instead use the time when the video frame / audio sample was captured, i.e. it does not include all the latency introduced by encoding and other processing in the sender pipeline.

This doesn’t make any big difference in this scenario but usually one would be interested in the capture clock times and not the send clock times.

Receiver

On the receiver-side there are a few more changes. First of all we have to opt-in to rtpjitterbuffer putting a reference timestamp metadata on every received packet with the sender’s absolute clock time.

pipeline.connect_closure(
    "source-setup",
    false,
    glib::closure!(|_playbin: &gst::Pipeline, source: &gst::Element| {
        source.set_property("latency", 40u32);
        source.set_property("add-reference-timestamp-meta", true);
    }),
);

rtpjitterbuffer will start putting the metadata on packets once it knows the NTP / RTP timestamp mapping, i.e. after the first RTCP Sender Report is received in this case. Between the Sender Reports it is going to interpolate the clock times. The normal timestamps (PTS) on each packet are not affected by this and are still based on whatever clock is used locally by the receiver for synchronization.

To actually make use of the reference timestamp metadata we add a timeoverlay element as video-filter on the receiver:

let timeoverlay =
    gst::ElementFactory::make("timeoverlay", None).context("Creating timeoverlay")?;

timeoverlay.set_property_from_str("time-mode", "reference-timestamp");
timeoverlay.set_property_from_str("valignment", "bottom");

pipeline.set_property("video-filter", &timeoverlay);

This will then render the sender’s absolute clock times at the bottom of each video frame, as seen in the screenshot above.

And last we also add a pad probe on the sink pad of the timeoverlay element to retrieve the reference timestamp metadata of each video frame and then printing the sender’s clock time to stdout:

let sinkpad = timeoverlay
    .static_pad("video_sink")
    .expect("Failed to get timeoverlay sinkpad");
sinkpad
    .add_probe(gst::PadProbeType::BUFFER, |_pad, info| {
        if let Some(gst::PadProbeData::Buffer(ref buffer)) = info.data {
            if let Some(meta) = buffer.meta::<gst::ReferenceTimestampMeta>() {
                println!("Have sender clock time {}", meta.timestamp());
            } else {
                println!("Have no sender clock time");
            }
        }

        gst::PadProbeReturn::Ok
    })
    .expect("Failed to add pad probe");

Rapid synchronization via RTP header extensions

The main problem with the previous code is that the sender’s clock times are only known once the first RTCP Sender Report is received by the receiver. There are many ways to configure rtpbin to make this happen faster (e.g. by reducing the RTCP interval or by switching to the AVPF RTP profile) but in any case the information would be transmitted outside the actual media data flow and it can’t be guaranteed that it is actually known on the receiver from the very first received packet onwards. This is of course not a problem in every use-case, but for the cases where it is there is a solution for this problem.

RFC 6051 defines an RTP header extension that allows to transmit the NTP timestamp that corresponds an RTP packet directly together with this very packet. And that’s what the next changes to the code are making use of.

The changes between the previous version of the code and this version can be seen here and the final code here in the rapid-synchronization branch.

Sender

To add the header extension on the sender-side it is only necessary to add an instance of the corresponding header extension implementation to the payloaders.

let hdr_ext = gst_rtp::RTPHeaderExtension::create_from_uri(
    "urn:ietf:params:rtp-hdrext:ntp-64",
    )
    .context("Creating NTP 64-bit RTP header extension")?;
hdr_ext.set_id(1);
video_pay.emit_by_name::<()>("add-extension", &[&hdr_ext]);

This first instantiates the header extension based on the uniquely defined URI for it, then sets its ID to 1 (see RFC 5285) and then adds it to the video payloader. The same is then done for the audio payloader.

By default this will add the header extension to every RTP packet that has a different RTP timestamp than the previous one. In other words: on the first packet that corresponds to an audio or video frame. Via properties on the header extension this can be configured but generally the default should be sufficient.

Receiver

On the receiver-side no changes would actually be necessary. The use of the header extension is signaled via the SDP (see RFC 5285) and it will be automatically made use of inside rtpbin as another source of NTP / RTP timestamp mappings in addition to the RTCP Sender Reports.

However, we configure one additional property on rtpbin

source.connect_closure(
    "new-manager",
    false,
    glib::closure!(|_rtspsrc: &gst::Element, rtpbin: &gst::Element| {
        rtpbin.set_property("min-ts-offset", gst::ClockTime::from_mseconds(1));
    }),
);

Inter-stream audio/video synchronization

The reason for configuring the min-ts-offset property on the rtpbin is that the NTP / RTP timestamp mapping is not only used for providing the reference timestamp metadata but it is also used for inter-stream synchronization by default. That is, for getting correct audio / video synchronization.

With RTP alone there is no mechanism to synchronize multiple streams against each other as the packet’s RTP timestamps of different streams have no correlation to each other. This is not too much of a problem as usually the packets for audio and video are received approximately at the same time but there’s still some inaccuracy in there.

One approach to fix this is to use the NTP / RTP timestamp mapping for each stream, either from the RTCP Sender Reports or from the RTP header extension, and that’s what is made use of here. And because the mapping is provided very often via the RTP header extension but the RTP timestamps are only accurate up to clock rate (1/90000s for video and 1/48000s) for audio in this case, we configure a threshold of 1ms for adjusting the inter-stream synchronization. Without this it would be adjusted almost continuously by a very small amount back and forth.

Other approaches for inter-stream synchronization are provided by RTSP itself before streaming starts (via the RTP-Info header), but due to a bug this is currently not made use of by GStreamer.

Yet another approach would be via the clock information provided by RFC 7273, about which I already wrote previously and which is also supported by GStreamer. This also allows inter-device, network synchronization and used for that purpose as part of e.g. AES67, Ravenna, SMPTE 2022 / 2110 and many other protocols.

Inter-device network synchronization

Now for the last part, we’re going to add actual inter-device synchronization to this example. The changes between the previous version of the code and this version can be seen here and the final code here in the network-sync branch. This does not use the clock information provided via RFC 7273 (which would be another option) but uses the same NTP / RTP timestamp mapping that was discussed above.

When starting the receiver multiple times on different (or the same) machines, each of them should play back the media synchronized to each other and exactly 2 seconds after the corresponding audio / video frames are produced on the sender.

For this, both sender and all receivers are using an NTP clock (pool.ntp.org in this case) instead of the local monotonic system clock for media synchronization (i.e. as the pipeline clock). Instead of an NTP clock it would also be possible to any other mechanism for network clock synchronization, e.g. PTP or the GStreamer netclock.

println!("Syncing to NTP clock");
clock
    .wait_for_sync(gst::ClockTime::from_seconds(5))
    .context("Syncing NTP clock")?;
println!("Synced to NTP clock");

This code instantiates a GStreamer NTP clock and then synchronously waits up to 5 seconds for it to synchronize. If that fails then the application simply exits with an error.

Sender

On the sender side all that is needed is to configure the RTSP media factory, and as such the pipeline used inside it, to use the NTP clock

factory.set_clock(Some(&clock));

This causes all media inside the sender’s pipeline to be synchronized according to this NTP clock and to also use it for the NTP timestamps in the RTCP Sender Reports and the RTP header extension.

Receiver

On the receiver side the same has to happen

pipeline.use_clock(Some(&clock));

In addition a couple more settings have to be configured on the receiver though. First of all we configure a static latency of 2 seconds on the receiver’s pipeline.

pipeline.set_latency(gst::ClockTime::from_seconds(2));

This is necessary as GStreamer can’t know the latency of every receiver (e.g. different decoders might be used), and also because the sender latency can’t be automatically known. Each audio / video frame will be timestamped on the receiver with the NTP timestamp when it was captured / created, but since then all the latency of the sender, the network and the receiver pipeline has passed and for this some compensation must happen.

Which value to use here depends a lot on the overall setup, but 2 seconds is a (very) safe guess in this case. The value only has to be larger than the sum of sender, network and receiver latency and in the end has the effect that the receiver is showing the media exactly that much later than the sender has produced it.

And last we also have to tell rtpbin that

  1. sender and receiver clock are synchronized to each other, i.e. in this case both are using exactly the same NTP clock, and that no translation to the pipeline’s clock is necessary, and
  2. that the outgoing timestamps on the receiver should be exactly the sender timestamps and that this conversion should happen based on the NTP / RTP timestamp mapping

source.set_property_from_str("buffer-mode", "synced");
source.set_property("ntp-sync", true);

And that’s it.

A careful reader will also have noticed that all of the above would also work without the RTP header extension, but then the receivers would only be synchronized once the first RTCP Sender Report is received. That’s what the test-netclock.c / test-netclock-client.c example from the GStreamer RTSP server is doing.

As usual with RTP, the above is by far not the only way of doing this and GStreamer also supports various other synchronization mechanisms. Which one is the correct one for a specific use-case depends on a lot of factors.

by slomo at May 02, 2022 01:00 PM

Víctor JáquezFrom gst-build to local-projects

Two years ago I wrote a blog post about using gst-build inside of WebKit SDK flatpak. Well, all that has changed. That’s the true upstream spirit.

There were two main reason for the change:

  1. Since the switch to GStreamer mono repository, gst-build has been deprecated. The mechanism in WebKit were added, basically, to allow GStreamer upstream, so keeping gst-build directory just polluted the conceptual framework.
  2. By using gst-build one could override almost any other package in WebKit SDK. For example, for developing gamepad handling in WPE I added libmanette as a GStreamer subproject, to link a modified version of the library rather than the one in flatpak. But that approach added an unneeded conceptual depth in tree.

In order to simplify these operations, by taking advantage of Meson’s subproject support directly, gst-build handling were removed and new mechanism was set in place: Local Dependencies. With local dependencies, you can add or override almost any dependency, while flatting the tree layout, by placing at the same level GStreamer and any other library. Of course, in order add dependencies, they must be built with meson.

For example, to override libsoup and GStreamer, just clone both repositories below of Tools/flatpak/local-projects/subprojects, and declare them in WEBKIT_LOCAL_DEPS environment variable:


$ export WEBKIT_SDK_LOCAL_DEPS=libsoup,gstreamer-full
$ export WEBKIT_SDK_LOCAL_DEPS_OPTIONS="-Dgstreamer-full:introspection=disabled -Dgst-plugins-good:soup=disabled"
$ build-webkit --wpe

by vjaquez at May 02, 2022 11:11 AM

April 25, 2022

Sebastian Pölsterlscikit-survival 0.17.2 released

I’m pleased to announce the release of scikit-survival 0.17.2. This release fixes several small issues with packaging scikit-survival and the documentation. For a full list of changes in scikit-survival 0.17.2, please see the release notes.

Most notably, binary wheels are now available for Linux, Windows, and macOS (Intel). This has been possible thanks to the cibuildwheel build tool, which makes it incredible easy to use GitHub Actions for building those wheels for multiple versions of Python. Therefore, you can now use pip without building everything from source by simply running

pip install scikit-survival

As before, pre-built conda packages are available too, by running

 conda install -c sebp scikit-survival

Support for Apple Silicon M1

Currently, there are no pre-built packages for Mac with Apple Silicon M1 hardware (also known as macos/arm64). There are two main reasons for that. The biggest problem is the lack of CI servers that run on Apple Silicon M1. This makes it difficult to systematically test scikit-survival on such hardware. Second, some of scikit-survival’s dependencies do not offer wheels for macos/arm64 yet, namely ecos and osqp.

However, conda-forge figured out a way to cross-compile packages, and do offer scikit-survival for macos/arm64. I tried to adapt their conda recipe, but failed to achieve a working cross-compilation process so far. Cross-compiling with cibuildwheel would be an alternative, but currently doesn’t make much sense if run-time dependencies ecos and osqp do not provide wheels for macos/arm64. I hope these issues can be resolved in the near future.

April 25, 2022 06:14 PM

April 20, 2022

Thomas Vander SticheleRunning Anthos inside Google

(Thomas Vander Stichele)

"With everyone and their dog shifting to containers, and away from virtual machines (VMs), we realized that running vendor-provided software on VMs at Google was slowing us down. So we moved."

Bikram co-authored this blog post last year about DASInfra's experience moving workloads from Corp to Anthos. The group I run at work is going down a similar path by migrating VMs to Anthos on bare metal for on-prem.

Taken from The Playlist - a curated perspective on the intersection of form and content (subscribe, discuss)

flattr this!

by Thomas at April 20, 2022 08:40 PM

April 18, 2022

Thomas Vander SticheleWhat’s the next action?

(Thomas Vander Stichele)

"Without a next action, there remains a potentially infinite gap between current reality and what you need to do."

David Allen's Getting Things Done is the non-fiction book I've reread the most in my life. I reread it every couple of years and still pick up on new ideas that I missed before, or parts that resonate better now and I'm excited to implement. Before Google, I used to give this book to new employees as a welcome gift.

The book got an update in 2015, and I haven't read the new version yet, so I'm planning an extended GTD book club at work in Q2, spreading the book out over multiple sessions. (In fact, I did just that for the young adult version of the book with my 16 year old godson back home in Belgium) If you've run a GTD book club, drop me a line!

Find out more at Getting Things Done® - David Allen's GTD® Methodology

"Too many meetings end with a vague feeling among the players that something ought to happen, and the hope that it’s not their personal job to make it so. [...] ask “So what’s the next action on this?” at the end of each discussion point in your next staff meeting"

Taken from The Playlist - a curated perspective on the intersection of form and content (subscribe, discuss)

flattr this!

by Thomas at April 18, 2022 04:39 PM

April 16, 2022

Thomas Vander SticheleRebecca Solnit – Men Explain Things to Me

(Thomas Vander Stichele)

"Most women fight wars on two fronts, one for whatever the putative topic is and one simply for the right to speak, to have ideas, to be acknowledged to be in possession of facts and truths, to have value, to be a human being."

In honor of International Women's Day 2022 (this past March 8th), some quotes from the 2008 article that inspired the term "mansplaining": to comment on or explain something to a woman in a condescending, overconfident, and often inaccurate or oversimplified manner.

I've certainly been (and probably still am) guilty of this behavior, and this is a standing invitation to let me know when I'm doing it to you.

Read the original article with a new introduction at Men Explain Things to Me – Guernica

"None was more astonishing than the one from the Indianapolis man who wrote in to tell me that he had “never personally or professionally shortchanged a woman” and went on to berate me for not hanging out with “more regular guys or at least do a little homework first,” gave me some advice about how to run my life, and then commented on my “feelings of inferiority.” He thought that being patronized was an experience a woman chooses to, or could choose not to have–and so the fault was all mine. Life is short; I didn’t write back."

Taken from The Playlist - a curated perspective on the intersection of form and content (subscribe, discuss)

flattr this!

by Thomas at April 16, 2022 02:37 PM

April 14, 2022

Thomas Vander SticheleDraft emails from Google Docs

(Thomas Vander Stichele)

In the ever more vertical company that Google is becoming, it is even more important to collaborate on some of your communication - more people want to contribute to the message and get it right, and more thought needs to be given to the ever wider audience you're sending mails to.

A while back I copied over AppScript code from an internal Google project to send meeting notes to make a different tool which makes it easy to go from Google Docs draft to a mail in GMail and avoid embarrassing copy/paste errors. I'm happy to be able to retire that little side project in favor of a recently released built-in feature of Google Docs: Draft emails from Google Docs - Docs Editors Help

Taken from The Playlist - a curated perspective on the intersection of form and content (subscribe, discuss)

flattr this!

by Thomas at April 14, 2022 03:33 PM

April 07, 2022

Christian SchallerWhy is Kopper and Zink important? AKA the future of OpenGL

(Christian Schaller)

Since Kopper got merged today upstream I wanted to write a little about it as I think the value it brings can be unclear for the uninitiated.

Adam Jackson in our graphics team has been working for the last Months together with other community members like Mike Blumenkrantz implementing Kopper. For those unaware Zink is an OpenGL implementation running on top of Vulkan and Kopper is the layer that allows you to translate OpenGL and GLX window handling to Vulkan WSI handling. This means that you can get full OpenGL support even if your GPU only has a Vulkan driver available and it also means you can for instance run GNOME on top of this stack thanks to the addition of Kopper to Zink.

During the lifecycle of the soon to be released Fedora Workstation 36 we expect to allow you to turn on the doing OpenGL using Kopper and Zink as an experimental feature once we update Fedora 36 to Mesa 22.1.

So you might ask why would I care about this as an end user? Well initially you probably will not care much, but over time it is likely that GPU makers will eventually stop developing native OpenGL drivers and just focus on their Vulkan drivers. At that point Zink and Kopper provides you with a backwards compatibility solution for your OpenGL applications. And for Linux distributions it will also at some point help reduce the amount of code we need to ship and maintain significantly as we can just rely on Zink and Kopper everywhere which of course reduces the workload for maintainers.

This is not going to be an overnight transition though, Zink and Kopper will need some time to stabilize and further improve performance. At the moment performance is generally a bit slower than the native drivers, but we have seen some examples of games which actually got better performance with specific driver combinations, but over time we expect to see the negative performance delta shrink. The delta is unlikely to ever fully go away due to the cost of translating between the two APIs, but on the other side we are going to be in a situation in a few years where all current/new applications use Vulkan natively (or through Proton) and thus the stuff that relies on OpenGL will be older software, so combined with faster GPUs you should still get more than good enough performance. And at that point Zink will be a lifesaver for your old OpenGL based applications and games.

by uraeus at April 07, 2022 06:14 PM

March 31, 2022

Thomas Vander SticheleThe COVID Cocoon

(Thomas Vander Stichele)

"The global COVID-19 pandemic has had countless impacts on society. One interesting effect is that it has created an environment in which many people have been able to explore their gender identity and, in many cases, undergo a gender transition. As organizations return to in-person work, be it full-time or hybrid, there is a greater chance of “out” transgender, non-binary, or gender non-conforming employees in the workforce." (From the "5 Ally Actions Newsletter - Mar 25, 2022")

March 31 is the Transgender Day of Visibility. The COVID Cocoon is a nickname given for the phenomenon of people discovering their gender diversity during the pandemic environment.

The full report is an interesting read; one recommendation that we can all contribute to is on Culture and Communication: Proactively communicating that gender diversity is accepted, asking staff for their input, and being open and ready to listen helps create a culture where employees can feel safe, welcome, and valued.

Taken from The Playlist - a curated perspective on the intersection of form and content (subscribe, discuss)

flattr this!

by Thomas at March 31, 2022 11:46 PM

March 28, 2022

Thomas Vander SticheleBuilding a Second Brain

(Thomas Vander Stichele)

"Your Second Brain is for preserving raw information over time until it's ready to be used, because information is perishable. Your Second Brain is the brain that doesn't forget." - Tiago Forte

Personal Knowledge Management is going through a wave of innovation with new tools like Roam, Logseq, Obsidian, Notion, RemNote, and others gaining traction over Evernote, OneNote and the like. It's a great time to get curious or reacquaint yourself with the tools and processes that strengthen learning, processing, and expressing your knowledge work.

The expression "Second Brain" has been popularized by Tiago Forte, who's been running an online cohort-based class called Building a Second Brain I took the class last year and found it a powerful distillation of an approach to PKM and note-taking. If you want to learn more, they just wrapped up the Second Brain Summit and posted all videos online: Second Brain Summit 2022 - Full Session Recordings - YouTube

The next class cohort is open for enrollment until March 30th midnight ET, at Building a Second Brain: Live 5-Week Online Course, and runs from April 12th to May 10th, 2022.

"Taking notes is the closest thing we have to time travel." - Kendrick Lamar

Taken from The Playlist - a curated perspective on the intersection of form and content (subscribe, discuss)

flattr this!

by Thomas at March 28, 2022 08:44 PM

March 14, 2022

GStreamerGStreamer 1.20.1 stable bug fix release

(GStreamer)

The GStreamer team is pleased to announce the first bug fix release in the stable 1.20 release series of your favourite cross-platform multimedia framework!

This release only contains bugfixes and it should be safe to update from 1.20.0.

Highlighted bugfixes:

  • deinterlace: various bug fixes for yadif, greedy and scalerbob methods
  • gtk video sink: Fix rotation not being applied when paused
  • gst-play-1.0: Fix trick-mode handling in keyboard shortcut
  • jpegdec: fix RGB conversion handling
  • matroskademux: improved ProRes video handling
  • matroskamux: Handle multiview-mode/flags/pixel-aspect-ratio caps fields correctly when checking caps equality on input caps changes
  • videoaggregator fixes (negative rate handling, current position rounding)
  • soup http plugin: Lookup libsoup dylib files on Apple platforms; fix Cerbero static build on Android and iOS
  • Support build against libfreeaptx in openaptx plugin
  • Fix linking issues on Illumos distros
  • GstPlay: Fix new error + warning parsing API (was unusuable before)
  • mpegtsmux: VBR muxing fixes
  • nvdecoder: Various fixes for 4:4:4 and high-bitdepth decoding
  • Support build against libfreeaptx in openaptx plugin
  • webrtc: Various fixes to the webrtc-sendrecv python example
  • macOS: support a relocatable `GStreamer.framework` on macOS
  • macOS: fix applemedia plugin failing to load on ARM64 macOS
  • windows: ship wavpack library
  • gst-python: Fix build with Python 3.11
  • various bug fixes, memory leak fixes, and other stability and reliability improvements

See the GStreamer 1.20.1 release notes for more details.

Binaries for Android, iOS, Mac OS X and Windows will be available shortly.

Release tarballs can be downloaded directly here:

March 14, 2022 12:00 PM

March 09, 2022

Jean-François Fortin TamPlease adapt Mozilla’s code so that PDF readers on Linux can handle XFA forms!

Y’know, all those horrible government forms?

No, I mean the digital ones, meant to be opened specifically with Adobe Reader?

Well, in Q4 2021, Mozilla’s PDF.js landed support for XFA PDF forms, so Firefox is now able to deal with them, which is huge deal, as we have been increasingly encountering such documents over the years, and still will be for a long time, especially given how slow-moving governments can be when it comes to their digital practices.

It would be fantastic to see these code insights put to use in Poppler, the library that Evince, Okular and other applications use… so if someone feels like fixing one of the few biggest issues with reading/filling PDFs under Linux, please use this code (see also: all the XFA-related pull requests) as inspiration to contribute a fix to this and that issue in Poppler!

Of course, there are remaining issues related to forms in PDF.js, but it’s still better than nothing; and perhaps your efforts in replicating this functionality into Poppler can lead to interesting cross-project findings that can also benefit the PDF.js project?

by Jeff at March 09, 2022 10:55 AM

March 03, 2022

Jean-François Fortin TamGetting Things GNOME 0.6 released

Yes, ladies, gentlemen, and seemingly-dead plants, it’s happening: after over 10 months of incremental work from the community, we are now releasing version 0.6 of our favorite personal productivity app, Getting Things GNOME. This release comes with some new features, lots of code improvements, many bugfixes and UX refinements (I am told that the “Better procrastination button”, presented below, deserves a place in the Museum of Modern Art).

Save the children, and the parents… tasks.

GTG 0.6 includes fixes for a long-standing higgs-bugson crasher, that would happen under some unpredictable conditions (such as issue 595 and issue 724) and was therefore hard to reproduce for a long time… until I hit the point, in my chaotic 2021 year, where I had accumulated over 2500 tasks as I forgot to clean my closed tasks for a few months… when your data file is that big, the bug becomes much easier to trigger.

We also fixed this mandelbug that would make GTG show inconsistent results in the list of tasks, under some circumstances. Neui was able to deduce the cause for the problem by looking at the tracebacks, and provided a fix in liblarch. GTG 0.6 will therefore require an updated version of liblarch.

Those two deeply nested bugs are the reason why I’m officially codenaming this release… “Shin Kidō Senki GTG Wing: Endless Recursion”.

(Insert a Zimmer-style “bwaaah” brass sound effect here. An Auralnaut is fine too.)

“Where’s my cal, Dave?”

Hey, we have a new synchronization backend now! If you have a CalDAV server (for example, something with OwnCloud or YUNOHOST), you can use this to synchronize GTG across your multiple computers.

It’s been a long time in the works. I would like to thank Mildred for doing the initial research and coding, then François Schmidts for doing another attempt at coding this feature, and for being very patient with us until we could finally merge this, after a lot of other architectural work landed. I know it can sometimes be tough for new contributors to wait for their code to land in an established open-source project, and for that project to also release the code in a stable release.

(With apologies to Mildred, François, and Padmé.)

Check out the built-in user manual pages to learn how you can use the CalDAV sync feature. There’s an online copy of the user manual, too. So far nobody reported catastrophic failures, so this sync backend seems to be Enterprise Ready™, but if you do encounter issues related to the CalDAV backend, kindly report them (even better if you can help with fixes and refinements!)

Gamification plugin

Sometimes, a little extra playful motivation can help you go through your day. Part of the brainstorming for gamification and virtual assistant features, Mohieddine Drissi created an initial plugin to add some game-like elements to GTG:

Here too, apologies to Mohieddine and Padmé!

Please try out this new feature, and look at the ticket linked above. Do you see additional features that would be good to add? Should they be part of this plugin, or a separate plugin? Let us know.

Modernized tag editor

This is what it used to look like:

This is what it looks like now:

Better procrastination button

One of the most important buttons in my GTG workflow is the “Do it tomorrow” button, and its associated menubutton that lets you reschedule a task’s start date a couple of days into the future. I call that feature the “procrastination” button and this might sound a bit silly, but it really is just an essential way to manage a frequently-changing set of priorities, schedule and obligations, and a way to manage your energy levels.

This release improves this feature with some additional attention to detail:

  • Inspired by some initial work by Laurent Combe, First, “Neui” made some pretty impressive hackery to make this an adaptive button that would automatically change its label depending on the available window/screen width, which now makes GTG capable of fitting within a split-screen on smaller screen resolutions. He did this only with Python code within GTG, so in case you’re wondering, GTG is not dependent on libadwaita for this to function.
  • In issue #550, I detailed the various inconsistencies between this menubutton and the contextual (right-click) menus for deferring tasks. As we were nearing the 0.6 release, I had a “How Hard Can It Be, Really?™” moment and went through a late night coding session to scratch at least part of my design itch, as it had been annoying me for a year by now and I didn’t want to stare at it for another release cycle. So this pull request of mine solves at least one half of the problem, which is probably better than nothing. Anyone is welcome to finish the 2nd half; you might have to harmonize that code with dates.py. Until then, here’s how it looks like now:

Errors will be noticed easily again

GTG used to have a fantastically avant-garde technological feature where it would automatically catch and display Python errors (tracebacks) in the graphical user interface. This feature got lost in what led up to the 0.4 release, but it is now making a comeback in GTG 0.6, thanks to Neui’s fine engineering work. It not only catches tracebacks, but also determines whether they are critical or if the application can possibly continue running. I did some UI & UX refinements on top of Neui’s version, and we now have this honest but still reasonably reassuring dialog:

If you click the expander, you get the traceback, along with some additional information, all neatly MarkDown-formatted so that it can be readily pasted alongside your bug report on modern bug trackers:

Of course, our software is perfect and you should never actually encounter such errors/uncaught exceptions, but… you never know. In the rare cases where this might happen, as a user, it’s better to be made aware if such a problem occurs—and the possibility of the application’s internal state being inconsistent—right when you trigger the issue, so that you can know what sequence of events led to a problem in the code, and report it in our issue tracker.

If you need to test this dialog (ex.: to test translations) but can’t find bugs in our flawless code, you can use GTG’s built-in “Developer Console” to trigger a traceback that will make that dialog appear (see this tip in our documentation).

Tough times, strong community

It’s been a hell of a year, but it’s still been less than a year since 0.5. Ideally we would be on a faster cycle, but we kept merging new and interesting things, and 2021 was pretty intense for many of us, so we slipped a bit.

Diego had to take time off to take care of his family and personal health, and I, certainly like many readers here, had a pretty busy year in my day-to-day work and beyond.

I deeply appreciate the GTG community’s patience in contributing, getting involved and sticking together to improve this project. Personally, I was not able to keep up very closely with all the activity going on; I received somewhere around 1900 emails related to GTG tickets and merge requests during the 0.6 cycle (from April 6th 2021 to this day), so it is clear that the community’s involvement is really strong here, and that’s awesome. If you’re not already part of this, consider joining the fun!

Let me take a minute to dish out some praise here for the fabulous and tireless work of our most frequent contributors. Thank you,

  • “Neui” for contributing a ton of patches, code reviews, advice, that helped the project’s technical architecture progress steadily throughout the year (also, thank you for investigating some of my craziest bugs and providing solutions for them!) ;
  • Diego, for plowing through architectural code refactoring, bugfixing, code reviews, during what I know was a difficult time;
  • Mildred and François, for making the CalDAV sync backend possible.
  • Mohieddine Drissi for creating the gamification plugin
  • Danielle Vansia, who not only updated and expanded the user manual so that you could know how the hell to use that CalDAV feature, but also kindly took Diego’s “changes since 0.5” braindump, expanded and polished it into the release notes you can find further below.
  • …and many other contributors who have made this release possible by providing bug fixes, code quality improvements, etc. They are listed in the about dialog’s credit section for this release, too 😉

Releasing because I can’t stand 0.5 anymore

You know, my gtg_data.xml file is pretty heavy:

No, I mean really heavy:

(Insert Hans Zimmer brass sounds)

As I write this, my data file contains over 2700 tasks (~1800 open, ~700 done, ~ 200 dismissed). The problem is, when you reach that kind of heaviness in your data file, in 0.5 you will encounter not only occasional crashes “when adding a parent/child task or when marking a recurrent task as done” (as mentioned at the beginning of this blog post), but also when trying to delete a bunch of tasks at once… which meant my “purge tasks” feature was not working anymore, which meant I kept piling on hundreds of “closed” tasks every month that I couldn’t easily remove from my tasks data file, which meant performance kept getting worse and crashes were getting more likely…

These are the total number of tasks in my XML file over time (including open, done and dismissed tasks). Each dip is when I tell GTG to purge closed tasks from the file. I know, those numbers keep going back up, higher and higher; I really need to find people to take care of some of my home and infrastructure bullshit, but that’s besides the point 😉

With those issues exacerbated by my “abnormally heavy” data file growing heavier every day, you can imagine that version 0.5 had become unbearable for me in my day-to-day use…

…therefore we have a perfect excuse to release version 0.6, which solves those issues! 😇
“Release when you can’t stand the previous release anymore!”

So here we are, GTG 0.6 comes right on time for income tax season! (and if it does help you make it through that period alive, maybe you can deduct some massive donations to Diego via LiberaPay or via GumRoad 😉 especially considering the metric crapton of refactoring work that will need to be accomplished for 0.7


Release notes

Thanks to Danielle, Diego and Neui for taking the time to research & detail the noteworthy changes to create the NEWS file, which you can read below.


New Features:

  • A new CalDAV backend is available, and the backends dialog is available again. CalDAV is a calendaring protocol that allows a client to access items from a server. GTG now provides support for this standard, and with this new sync service, you can manage all your tasks in one place.
  • The new “Gamify” plugin adds a game aspect to your GTG workflow, such as the ability to set task targets and task completion streaks.
  • The Tag Editor was completely redesigned.
  • Added support for undo/redo actions in the Task Editor.
  • Added the ability to collapse and expand all tasks in the main menu.
  • Added the F10 shortcut to open the main menu.
  • ESC now closes the calendar picker window.
  • Added the CTRL+B shortcut to set focus on the sidebar.
  • Added an option to set the due date to “today” in the context menu.
  • The “Mark as not done” and “Undismiss” actions were replaced by a unified “Reopen” action for closed tasks in the right-click context menu.
  • The “Start Tomorrow” button is now adaptive, so that the main window can be resized to smaller widths.
  • The task deferral menubutton (next to “Start Tomorrow”) now shows both weekday names and the offset number of days from today. It also allows easily deferring to 7 days instead of only 6 days, and the ordering of this menubutton matches better with other menus.

Backend, Code Quality, and Performance Improvements:

  • Reintroduced the global exception/traceback catcher, with a dialog showing up when an error occurs.
  • Made an improvement to avoid infinite loops when entering invalid dates in the Quick Add entry.
  • Made an update to prevent errors when no task is selected.
  • Removed some deprecation warnings.
  • Added StartupWMClass to make pinning on KDE work.
  • Used application ID as the window icon (so KDE shows the correct icon).
  • Made several changes in preparation for Gtk 4.0, as well as updated a number of deprecated GTK-related items.
  • Made a ton of PEP8 and style fixes.
  • Refactored the date class.
  • Updated the ability to render tag icons better on HiDPI.
  • Updated the anonymize script.
  • Added gtg://TASK-ID to the command-line help.
  • Added the -p parameter for profiling in debug.sh.
  • We migrated from “Nose” to “PyTest” for the test suite, as Nose is unmaintained.

Bug Fixes:

  • Fixed possible crash when trying to create parent task, and similar operations, when the “Open” tab has not been opened yet (maximum recursion depth error).
  • Made a change to save a task before creating a parent. This prevents an error that appeared when a task title would be reset after adding a parent.
  • Fixed an issue where tags were being duplicated.
  • Fixed an issue where tags and saved searches with the same name were being considered duplicates.
  • Fixed a bug where every editor window would come back if GTG wasn’t closed cleanly (i.e., shut down).
  • Made several fixes for scripts.
  • Fixed certain main menu entries not being selectable via the keyboard.
  • Fixed the cut-off when you expand the columns too much.
  • Fixed a regression where symbols in tags (e.g., dashes and dots) were not recognized by the Task Editor. Also, added support for a few more.
  • Fixed a bug where tags’ icon, parent, and color were not being removed in the XML file.
  • Fixed a bug that occured when GTG just starts up with no tasks selected, and the user tried to use the “Add Parent” hotkey.
  • Avoid blurry tag color rounded rectangles on non-HiDPI screens.

Documentation Updates:

  • Added documentation to the Contributor Docs for contributing to the User Manual (i.e., for writing help files).
  • Added information about using flamegraph for profiling GTG for performance documentation (in the Contributor Docs).
  • Updated the user manual for this release.

For more details on all of these new features, improvements, and fixes, see the 0.6 release’s milestone.


Time to "flatpak update", folks:

by Jeff at March 03, 2022 05:00 PM

March 01, 2022

Jean-François Fortin TamYear MMXXI in 8 minutes

Near the end of 2020, I put a lot of thought into reevaluating my business’ value proposition, strategy, and processes. It’s a good thing I did that back then, because 2021 was quite different from 2020; I had much less time to “deepthink”, and I spent a majority of 2021 on an intense work treadmill, which led to me micro-burning out three times in the process. Also, guilt about feeling like I’m not contributing to open-source enough.

In that year, I also noticed a lot of “shortened fuses” around me—a pervasive edginess throughout society—along with bully-type abuse in various places. It’s an interesting story to analyze from an experiential and psychological standpoint, but I’ll cover that in a separate blog post sometime, to keep this one simple (so it takes roughly 8 minutes to read).

For now, I’ll focus on the usual short summary of my work + personal + open-source discoveries and activities from 2021, without too much emphasis on the deeper chaos going on in that year.


Professional life year overview

  • My then accountants suddenly told me, in January of that year, that they would no longer be serving small businesses for income tax reports, and thus “We’re dropping you, good luck!” 🤯 I then spent weeks exhaustively searching and evaluating accounting firms to find a new partner, and eventually found a top-notch accounting duo to help me with income tax reports in the long term. I asked them hardball questions on their career objectives and whether/when they would drop me like a hot potato, and I told them I couldn’t go through another taxheartbreak. Now that I’ve found you, you better stay with me, nya?
  • Worked for a surprising number of clients throughout the year (including a couple NGOs, as you can see in this case study about an in-depth technical & financial feasibility study and that case study about a naming/branding consulting project). I’ll keep it at that for now, again to keep this blog post simple.
  • Throughout the year, I (re)built a lot of websites for myself, and realized again just how much time & effort this requires. Since I timed myself, I now have some quantitative insights to share with all of you wondering, “How Hard Can It Be?™”; see my “How long does it take to create a website? (and why your FLOSS project doesn’t need one)” article.
    • Publicly unveiled Regento, after having finally created its website, at the same time as I was doing some pretty intensive consulting work in April (y’know, when things happen “all at the same time”…), hence the 2nd micro-burnout of the year.
    • Reworked the idéemarque branding agency website contents entirely during the summer. Did not blog about it until now. The design isn’t especially fancy/flashy, but it’ll do, I guess.
    • Quietly created Atypica’s photo portfolio in the fall season. Did not blog about it until 2022.

Personal life year overview

  • Summer 2021, with the arrival of vaccines and nice outdoors weather in the North, felt exactly like this Tokyo-III scene:
Yes, I know, this is the scene where everything feels “normal” before shit hits the fan again. The parallels are uncanny.
  • You might remember that I tried a kratky hydroponic indoors garden in 2020, and had no luck with tomatoes. In the winter/spring of 2021 I thought, “Perhaps I need to try my luck again with a regular self-watering soil planter, and lots of artificial light”. Well, I did: since September 2021, I can confirm that even with that kind of setup and a 4′ vertical fluorescent light to supplement the winter’s natural light, the tomato plant does not flower (and thus does not fruit), and barely survives better than the previous kratky hydroponic experiment; possibly its slightly better mildew resistance this time around is due to the artificial light, but I am not sure.
  • Made my Brother multifunction printer’s scanner work again with Linux, by setting up a dedicated machine with lots of ugly hackery to make it work. Now it works, let’s not ever touch it again for the next five years.
  • German cockroaches got in (via a package I brought inside) during the summer. It creeped me the hell out. I went through some very extensive research on how to get rid of them effectively and without unintended side-effects. Long story short, I killed them all without using chemicals, but that messed up my quality of life for some months, as it definitely didn’t help my sleep quality. Spraying pesticides or using tape is harmful and pointless; the real trick is to use a very thinly sprayed fine layer of diatomaceous earth (so that it is invisible), lay a couple of boron-infused gel baits (Advion™ is impossible to find up here) in every corner of every baseboard near the nest, all while starving them from any food or water, then… waiting and having a lot of patience and trust in that method. Also, always having a vacuum cleaner on hand (I can confirm that they do not survive a cyclonic canister) and manoeuvering quickly to catch them late at night to reduce their numbers. It took two months (or 4 months if you take into account the last two months of “zero encounters”), and it was nerve-racking. I learned a lot in the process, and they are certainly never going to be an issue again.
  • I probably spent 30-60% of the year away from home because of the constant hammering most days. Did you know that a set of one 8-stories-high building and one 20-stories-high building spanning a whole street requires about 2000 piles for their foundations? It took them roughly 12 months to drive all those piles into the ground (it didn’t help that the GC was using only one or two pile drivers at a time).
Pro Tip™: this song really helps to cover it all up.

A most welcome visitor

My longtime friend Étienne suddenly came back to visit Canada again for the first time in two years. He was originally too shy to ask for my hospitality, so I insisted to have him as my honored guest and to host him for the whole 3 weeks of his stay in Montréal. Being able to spend so much time with him is a privilege I have not had since, I think, fifteen years (the previous time was probably 2006). The timing was perfect; after a whole year of stress and work that had left me eager to have some sort of change of pace in December, this was my mood in anticipation of spending time with him again:

We had fun fixing lots of stuff, and simply being care-free geeks on a vacation:

  • I helped him livestream a wedding, with some of my studio equipment and the wonder of OBS Studio. I also took a couple hundred photos of the event.
  • He brought me a bunch of VoIP deskphones for my office, originally taken from an office in Tokyo. I’m now ready to open up a call center or something.
  • He brought me a replacement Nintendo Gamecube controller, again from Japan. Now we can honestly play Super Smash Brothers Melee again!.. if only having multiple guests was still a thing nowadays.
  • He completely tore down, cleaned and reassembled my projector (that I had föked up a year prior by accidentally blowing some dust onto the projector’s DMD chip, and that I did not have the time nor courage to fix by myself until that point). I’d like to think I helped with this deep-cleaning process as an assistant, but frankly he did all the difficult work, as you can see:
Étienne starting disassembly Optical block (sensor and lens assembly) taken out Lens taken out Separating the DMD chip from its PCB
  • He contaminated me with the wonder that is the “future funk” music genre, particularly Moe Shop‘s music.
  • He helped me make my other Brother printer work over the network.
  • I had never had the time and energy to mess with my router in the past year after its initial setup, so he flashed a new version of OpenWRT and finally figured out how to make multi-frequencies multi-SSD networks work seamlessly.
  • More importantly, it gave me the opportunity to have lengthy discussions with him and see what projects he’s been working on, and see how he’s been doing mentally, physically, and in terms of life trajectory. Being oceans apart for years meant that it was more difficult to check on friends’ well-being, so I welcomed the opportunity to catch up.

Open-source findings

  • Did you know that the “Gnome” icon theme still exists, and is not the same as the “Adwaita” icon theme?
  • I proposed an epic wallpaper concept for Fedora, but it went nowhere, much to my surprise.
  • I found the sudden disappearance of the GNOME Swedish Conspiracy to be an unacceptable loss to our shared cultural heritage; therefore I filed this formal inquiry into the matter, issue number 1.
  • Successfully lobbied for Planet GNOME to properly support Unicode blog posts, including emojis. You’re welcome 😇
  • Filed a couple of Jitsi UX issues from my observations, during the pandemic, of computer-illiterate users’ difficulties with this open-source WebRTC videoconferencing app. I’m happy to see now that many of them have been fixed.
  • Was pleased to see XFA PDF forms now being supported in Firefox. Look forward to my blog post on March 9th to learn about the possible implications.
  • Discovered that you can actually export addressbooks to .CSV in Evolution, but that capability is not presented in the GUI and you really need to know it exists at all.
  • Want to contribute an easy UX refinement to Evolution’s mail composer GUI? How about modernizing the hyperlinking dialog using GtkPopover? Check out the other newcomers-friendly tasks too.
  • Heads up all Evolution die-hards who would like to escape Google Calendar but still can’t because of visual info density: I’ve now re-filed my old bug report on the matter: “Upcoming weeks” / “Next 3 weeks” future events view mode. Give it a 👍 or consider contributing a patch if you’re more competent than me (as you most certainly are).
  • Suggested my amazing, revolutionary™ timeout-based search architecture for GTG. The results, as seen in GTG 0.5 and newer (such as 0.6, planned for release in the coming days) is a dramatic improvement in performance; search takes 1-3 seconds instead of 17+ seconds in heavy data sets.
    • I made the same recommendation in Simple Tab Groups for Firefox and the results were dramatic, as enthusiastically tweeted. You’re welcome.
    • As tweeted: I made the same live search method recommendation for Nautilus, the GNOME files manager. See this ticket (and please contribute a patch to the project, if you can).
    • As tweeted: I also filed a ticket to make the same suggestion—use a timeout-based search-as-you-type approach—for GTK’s FileChooser widget. I’m just wildly guessing, but this might be a reasonably easy-enough bug for a newcomer to contribute a patch for, but at the same time, this is GtkFileChooser we’re talking about, you’re certainly bound for technical surprises (but a great learning opportunity).

Miscellaneous findings

  • Realized part of why I’ve accumulated such a huge backlog of subjects to blog about here over the years, with so few actually getting published: this heat matrix of mine shows that the technical but low-impact topics are endless! High technicality = high accuracy and documentation requirements, a tremendous investment of time (each of my blog posts typically takes multiple days of work); if there is no clear impact/reward, then there is little incentive when there are so many other imperatives in life.
  • Realized that 78% of the USA’s cash (of all time) appeared out of thin air within the month of May 2021, and I still don’t understand how this stock market bubble that has been going on since 2011 keeps going without popping, with all that’s been going on in 2020-2021. Markets sure can stay irrational longer than we can stay solvent or sane.
  • Invented a method to reinforce cheap leatherette keychains with zipties.
  • Found some rusted, abandoned cast iron cookware in my building’s recycling room, and embarked on an adventure to learn all about cast iron and how to properly season and maintain it. With my eight back-to-back grapeseed oil seasoning passes in the oven, I think I did a pretty good job at seasoning them. So I guess I’m part of the Cast Iron Cult now. Considering that properly seasoned cast iron is naturally non-stick, who needs Teflon?! Besides, you can never have too many mêlée weapons. Even the Government of Canada basically says that cast iron and stainless steel are probably the only things that you can really trust (from a health/safety perspective)…
  • Discovered that you can sanitize (and endlessly reuse) N95 and surgical face masks, by using an electric pressure cooker (such as an Instant Pot) as a dry heat autoclave.
  • For all you nerds whose glasses keep slipping off your nose: binder clips! (no, not really… turns out there are silicone ear retainers out there)

by Jeff at March 01, 2022 09:45 PM

February 27, 2022

Thomas Vander SticheleTime in Meetings

(Thomas Vander Stichele)

article #productivity

Meetings are both necessary and useful, but they fragment your week, your opportunity for flow, and you need non-meeting time for your output as a knowledge worker.

"Those of us on the maker's schedule are willing to compromise. We know we have to have some number of meetings. All we ask from those on the manager's schedule is that they understand the cost." - Paul Graham, Maker's Schedule, Manager's Schedule

flattr this!

by Thomas at February 27, 2022 10:25 AM

February 25, 2022

Thomas Vander Sticheleapenwarr@ – The Gift of It’s Your Problem Now

(Thomas Vander Stichele)

#article #log4j

"Sometimes the gift interprets JNDI strings in my log messages and executes random code from my LDAP server. This is the nature of gifts."

An interesting musing on the nature of gifts, big companies and startups, and free software, from apenwarr@

Taken from The Playlist - a curated perspective on the intersection of form and content (subscribe, discuss)

flattr this!

by Thomas at February 25, 2022 06:25 PM

February 23, 2022

Jean-François Fortin TamHow long does it take to create a website? (and why your FLOSS project doesn’t need one)

The 20192020 period was a long R&D cycle for me, with a whole herd of yaks to shave, however it did give me new tools and abilities, such as the capacity to rapidly develop modern-looking websites without hand-coding them nor spending hours fruitlessly searching for—and being disappointed by—”suitable” themes.

One might wonder why then, with those upgraded skills, I didn’t seize the opportunity to make a new website for GTG or other open-source projects that could benefit from fancier marketing.

The short answer is, even if my tooling and technique have improved, it’s still a metric crapton of work. It might seem easy (“Use Hugo!”, “Just slap a couple of pages together!”, etc.) but there’s much more to building a website than just the technical design and coding (or integration) aspects. This article provides you with an idea of the amount of time it takes to plan, write, design and build a “reasonably simple” website with a dozen pages or so.

As a case in point, the website of my Montreal branding agency has been out there for many years, but right up until 2021, it was all hand-coded—and therefore, not really maintainable nor scalable to a team. One of the things I did in 2021 was to completely revamp it, by:

  • Switching it to WordPress instead of my own static HTML + CSS + PHP glue system I had perfected since the mid-2000’s (if you find that blasphemous, have a look at why I chose WordPress over other solutions for my business websites).
  • Rethinking all the contents.
  • Designing a basic layout to hold it all together, because frankly (and somewhat ironically) we didn’t have time to be obsessing about the visuals here.

Overhauling (or creating) a website is never a simple affair when you want to do things right for your project or business. Planning, writing & designing two of my new websites in 2021 required an investment of well over 230 hours of work. Among those were roughly 112 hours of work (at least, those I remembered to track) spent on the idéemarque website specifically, broken down roughly as follows:

  • 22 hours preparing a strategic positioning & structure document
  • 20 hours writing the new website’s “core pages” text, in two languages
  • 20 hours integrating the text into the website and laying out the pages’ overall design
  • 40 hours writing and laying out the 4 initial case studies, in two languages
  • ?? hours setting up and integrating an email notifications (newsletter) system, for those who want to know when a new article comes out
  • 9 hours (so far) sketching and drawing team members, and researching possibilities (a story for another day)

112+ hours, for a 18-pages website, is actually pretty damned fast: business websites typically tend to take much more time than that. This was made possible by circumstances rarely found with client work: I knew exactly what I was doing (from a technical standpoint), where I was going (from a marketing standpoint), and what I wanted to put in there (from a copywriting standpoint). Your mileage will almost certainly vary, especially if working with a large corporate team or committee where you can’t just directly pour all the content straight from your mind.

Those numbers do not include the R&D (and infrastructure maintenance) I did in previous years to actually get to the point where I could efficiently build such websites.

And, y’know, once you have a website out there, you need to feed it with new content, and to announce it. I spent an unknown amount of hours (probably well over 20-40; forgot to track them) planning and working on draft email announcements that kept getting outdated as the months went by—because I kept having to deal with emergencies, and because I kept moving the goalposts of what constituted a “good enough” website to write home about (nothing is more dangerous than “oh, one more thing…”). Me, a perfectionist? Nooooo.

So yeah… this is why I’m no longer spending that kind of time and energy building websites for my open-source projects, especially as FLOSS projects have contents that need to change much more frequently than most small-to-medium business websites. In the case of GTG, while I hate GNOME’s MoinMoin wiki, it Does the Job™ and it’s Good Enough to stay as it is; it’s still easier to maintain than a website. So y’know what? Your open-source project doesn’t need a website, unless it’s also a business or charitable organisation.

by Jeff at February 23, 2022 03:56 PM

February 22, 2022

Thomas Vander SticheleIsabel Wilkerson – Caste: The Origins of Our Discontents

(Thomas Vander Stichele)

Isabel Wilkerson - Caste: The Origins of Our Discontents

#book #dei

"Like other old houses, [...] has an unseen skeleton, a caste system that is as central to its operation as are the studs and joists that we cannot see in the physical buildings we call home. Caste is the infrastructure of our divisions. It is the architecture of human hierarchy, the subconscious code of instructions for maintaining [...] a [...] social order."

Caste has taken the lead in my library as the most highlighted book, and is a deep exploration of Caste as the lens through which to see discrimination, drawing parallels between Europe, the United States, and India, providing a universal framing.

“Young people,” he said, “I would like to present to you a fellow untouchable from the United States of America.” King was floored. He had not expected that term to be applied to him. He was, in fact, put off by it at first.

Taken from The Playlist - a curated perspective on the intersection of form and content (subscribe, discuss)

flattr this!

by Thomas at February 22, 2022 05:35 PM

February 21, 2022

Thomas Vander Stichele9 Ways We Self Sabotage

(Thomas Vander Stichele)

9 Ways We Self Sabotage

#survey #productivity

"Saboteurs are the voices in your head that generate negative emotions in the way you handle life’s everyday challenges. They represent automated patterns in your mind for how to think, feel, and respond. They cause all of your stress, anxiety, self-doubt, frustration, restlessness, and unhappiness. They sabotage your performance, wellbeing, and relationships."

Positive Intelligence is a mental fitness framework and, among other concepts, taught me helpful practical ways to deal with stress, both professionally and personally.

Take the test or read more on How we self-sabotage

Taken from The Playlist - a curated perspective on the intersection of form and content (subscribe, discuss)

flattr this!

by Thomas at February 21, 2022 07:21 PM

February 20, 2022

Jean-François Fortin TamGTG 0.6 release candidate

Today we are publishing a “release candidate” version of Getting Things GNOME 0.6. You can either try it out directly from the git master version (by running launch.sh; see the general instructions), or from the testing package available on Flathub’s “beta” repository, separately from the standard stable flathub/flatpak release you may already be running. To run it as a flatpak, simply run these two commands:

flatpak remote-add flathub-beta https://flathub.org/beta-repo/flathub-beta.flatpakrepo
flatpak install org.gnome.GTG

…and choose the flathub-beta as the repository source to install GTG. Note that for some technical reasons, it will still say it’s version 0.5 if you look in the About dialog, but it really is 0.6 behind the scenes.

This release candidate is supposed to be stable and regressions-free, and is intended to:

  • Give translators an opportunity to update their favorite language(s) before the final release. You have roughly a week where you can voice your interest and provide us with an updated translation, if desired. The translation template (.pot file) has already been updated in the git version.
  • Give the opportunity for users to feel confident in trying out the current git version (or flatpak package above) and to easily check that everything is OK (and if not, report bugs).

This is the last call! If you haven’t tested out the git version recently, now would be a great time to test the release candidate and ensure there are no last minute issues we might have overlooked. If no showstoppers/regressions are found, we expect to release GTG 0.6 around the end of the month (roughly in a week).

by Jeff at February 20, 2022 04:20 PM

February 19, 2022

Nirbheek ChauhanBuilding GStreamer on Windows the Correct Way

For the past 4 years, Tim and I have spent thousands of hours on better Windows support for GStreamer. Starting in May 2016 when I first wrote about this and then with the first draft of the work before it was revised, updated, and upstreamed.

Since then, we've worked tirelessly to improve Windows support in GStreamer  with patches to many projects such as the Meson build system, GStreamer's Cerbero meta-build system, and writing build files for several non-GStreamer projects such as x264, openh264, ffmpeg, zlib, bzip2, libffi, glib, fontconfig, freetype, fribidi, harfbuzz, cairo, pango, gtk, libsrtp, opus, and many more that I've forgotten.

More recently, Seungha has also been working on new GStreamer elements for Windows such as d3d11, mediafoundation, wasapi2, etc. Sometimes we're able to find someone to sponsor all this work, but most of the time it's on our own dime.

Most of this has been happening in the background; noticed only by people who follow GStreamer development. I think more people should know about the work that's been happening upstream, and the official and supported ways to build GStreamer on Windows. Searching for this on Google can be a very confusing experience with the top results being outdated links or just plain clickbait.

So here's an overview of your options when you want to use GStreamer on Windows:

Installing GStreamer on Windows

 
GStreamer has released MinGW binary installers for Windows since the early 1.0 days using the Cerbero meta-build system which was created by Andoni for the non-upstream "GStreamer SDK" project, which was based on GStreamer 0.10.
 
Today it supports building GStreamer with both MinGW and Visual Studio, and even supports outputting UWP packages. So you can actually go and download all of those from the download page:


This is the easiest way to get started with GStreamer on Windows.
 

Building GStreamer yourself for Deployment

 
If you need to build GStreamer with a custom configuration for deployment, the easiest option is to use Cerbero, which is a meta-build system. It will download all the dependencies for you (including most of the build-tools), build them with Autotools, CMake, or Meson (as appropriate), and output a neat little MSI installer.
 
The README contains all the information you need, including screenshots for how to set things up:


As of a few days ago, after months of work the native Cerbero Windows builds have also been integrated into our Continuous Integration pipeline that runs on every merge request, which further improves the quality of our Windows support. We already had native Windows CI using gst-build, but this increases our coverage.

Contributing to GStreamer on Windows

 
If you want to contribute to GStreamer from Windows, the best option is to clone the gstreamer monorepo (derived from gst-build which was created by Thibault), which is basically a meson 'wrapper' project that has all the gstreamer repositories aggregated as subprojects. Once again, the README file is pretty easy to follow and has screenshots for how to set things up:


This is also the method used by all GStreamer developers to hack on gstreamer on all platforms, so it should work pretty well out of the box, and it's tested on the CI. If it doesn't work, come poke us on #gstreamer on OFTC IRC (or the same channel via Matrix) or on the gstreamer mailing list.
 

It's All Upstream.

 
You don't need any special steps, and you don't need to read complicated blog posts to build GStreamer on Windows. Everything is upstream.

This post previously contained examples of such articles and posts that are spreading misinformation, but I have removed those paragraphs after discussion with the people who were responsible for them, and to keep this post simple. All I can hope is that it doesn't happen again.

by Nirbheek (noreply@blogger.com) at February 19, 2022 01:37 AM