Screencasting in the AI era

During my tenure as InfoWorld’s blogger-in-chief I became fascinated with narrated videos of software in action. Tools existed to do that kind of storytelling but the explanatory power of what came to be called screencasting had not yet been widely recognized. Although the medium became a standard way to demonstrate software, and I went on to produce and edit many screencasts over the years, the available tools never worked the way I wanted them to: easy and precise frame-accurate previewing and execution of annotations, cuts, pauses, and narration. When people asked for screencasts to demonstrate how to use Bram, I tried using conventional tools, gave up in frustration, and on a whim decided to see if I could use Bram to create the editor I’d always wanted.

In a couple of days I had the first version of Bram Studio; a few more days of iteration yielded Bram’s debut screencast. The tool runs from a local clone of Bram Studio’s GitHub repo, where you launch Bram and then open its so-called target app.

Bram, in the bram-studio repo, ready to open Bram Studio. 

Foundry

The Bram Studio projects page.

Foundry

Nudging a box annotation to be one frame longer.

Foundry

It works well for me, but the setup felt a bit weird. Who would want to clone a repo, and then install Bram, and then use Bram to launch Bram Studio? Why not just package Bram Studio as an easy-to-download-and-install single binary like Bram itself?

Bram Studio is a user innovation toolkit

My North Star for product development is best described in Eric von Hippel’s Democratizing Innovation. Don’t just deliver a canned product to meet an anticipated need, he argues. Instead make the product malleable by lead users who can shape it to their actual needs. The book’s examples include but are not limited to software products. My favorite example is snowboards. Companies shipped boards that customers could modify, watched the mods happen, rolled the most important and general mods into their shipping products, then iterated that virtuous cycle.

As I developed Bram Studio I realized that the agents I was using Bram to direct — mostly Claude Code, sometimes Codex — were wielding FFmpeg in powerful ways. In a post on the protean accomplishments of its creator, Fabrice Bellard, John Carmack wrote:

Today FFmpeg is the invisible engine of the internet. YouTube uses it. Netflix uses it. VLC uses it. Chrome and Firefox use parts of it. Every Android phone, every iPhone, every smart TV, every video editing tool you have ever touched runs FFmpeg somewhere underneath. If you have watched a video on a screen in the last 20 years, Fabrice’s code processed it.

Here is Claude’s explanation of one of the ways it uses FFmpeg in Bram Studio.

You talk over part of a scene, and your voice replaces the scene’s sound for exactly that stretch. The picture is left untouched. It takes four ffmpeg passes:

Clean up the voice. Cut the recording to what was said, from the moment playback started, then denoise, high-pass and compress it: anlmdn → highpass=f=80 → acompressor (NARRATE_AF).

Measure it. Run loudnorm=print_format=json in a measure-only pass. It only reports the integrated loudness; it doesn’t change the audio. If the result is below –45 LUFS, the take is refused with “no speech was heard”. Otherwise the gain needed to reach –16 LUFS is applied as a single volume=…dB, then an alimiter.

Save the audio being replaced (atrim) in media/.narrated/, so a narration can be removed later even after other edits.

Splice it in with one filter_complex: the original audio up to the start (atrim, then a 10 ms afade out), the new voice padded or trimmed to fill the stretch exactly (apad=whole_dur, atrim=duration), then the original from the end on (a 10 ms afade in), joined with concat=v=0:a=1.

These uses of FFmpeg are baked into Bram Studio, in a Python harness, and could be delivered as part of a stand-alone binary. If I went that route I’d ditch Python, which is a deployment headache, in favor of a language like Go or Rust that lends itself to single-binary deployment. Something like that happened with Bram itself. It originally used Python for the hooks that run before and after agent tool calls. That complicated life for Windows users who might not have Python installed. How hard would it be, I wondered, to port those hooks to Rust, the primary language used by Bram (because it’s based on the Tauri framework)? In a couple of hours it was done and Bram was freed of its Python dependency. Programming languages are fungible now; LLMs see into the deep structure of code; they easily and reliably translate among surface representations.

But a single-binary version of Bram Studio would not be a user innovation toolkit; it would be a canned app with baked-in assumptions about how to use FFmpeg. When you use Bram to run Bram Studio from its local repo you can

Read the original article: