City Lifeline

The short version

City Lifeline is a radio station whose music is played by a street. A computer-vision service watches a Transport for London traffic camera, turns what it sees into a set of numbers between zero and one, and those numbers drive a headless C++ synthesizer. The audio goes out as a live HLS stream.

Two of my interests meet in it. A camera pointed at a junction is a data source that never stops, and a synthesizer is an instrument that will happily play forever. Wiring one into the other was the whole idea.

It ran unattended for 245 hours across two weeks in August and September 2026, and every bar it played is in the archive. The harmony changed 3,323 times over that run. 243 of those changes came from the street. The other 3,080 came from a timer that fires when 96 bars have passed with nothing happening. The street moved the music 7.3 per cent of the time.

The caveat, up front. This is one junction, seen at 352 by 288 pixels, over two weeks. A different camera or a busier crossing would give a different number, and I have not tested one. The station has never been deployed publicly; it runs on localhost and it is currently dormant. There is no audio from the soak either: HLS deletes its segments as the window rolls, so two weeks of running left telemetry and music decisions but no sound.

To show what it sounds like, I ran the station again on 14 September 2026. The camera posts a new 11-second clip every few minutes, and I kept the three it posted between 9:11 and 9:21 that morning. The station watched them and played along. The boxes and labels are what it saw, and the numbers along the bottom are what it passed to the synth. The lead is a newer FM voice with mallet, kalimba and piano sounds, which the two-week run did not have.

Camera footage: Transport for London JamCam, via TfL Open Data.

Python and OpenCV for the vision, YOLO for detection, a YAML patchbay, a C++ and JUCE synth built on my Pariki harness, ffmpeg and HLS for the broadcast, Docker to hold it together.

How it works

The camera is a Transport for London JamCam pointed at Trafalgar Square. It gives me 352 by 288 pixels, and the vision service analyses every second frame of a 25 fps stream: 12.5 analysed frames a second. That resolution limits everything downstream and explains most of what follows. A pedestrian in that frame is around nine pixels tall.

A Python service reads the frames, running optical flow for movement and two YOLO passes for objects, one for people, one for vehicles. Typed zones sit on top, drawn by hand for each station. A road declares which way its traffic should run. A crossing names the two footpaths it connects. Each zone instance reports estimated people, heading, speed and its own checks.

Some parts of the picture have to be ignored. The camera burns a ticking clock into the top left and the TfL logo into the top right, and the changing digits read as movement. Two patches of wall on the left do the same. I masked all four out, and the vision service skips any movement inside them. The sky is drawn as its own zone too, but nothing in the music listens to it yet.

Those reports become musical events. Twenty normalised sources leave the vision service, each a number between zero and one: vehicle count, footpath presence, luma, crowd energy and the rest. A 90-line patchbay file wires each source onto a synth voice parameter. Vehicle count drives the drums’ density. Luma drives their tone. Two events fire triggers instead of dials: a vehicle crossing a lane, and a completed pedestrian journey.

The music has six voices: a low drone, a bass line, drums, pad chords, a lead melody, and a sparkle voice that answers people on foot. Every voice has the same small set of controls. Volume is how loud it is. Tone is how bright it sounds. Activity is how much it plays. Variance is how far it wanders from the obvious note or beat. Each measurement is scaled between zero and one against the last five minutes of the same camera, which keeps a quiet night and a busy morning both in range. This is what each one drives on the Trafalgar Square station.

What the station measures	What it changes in the music
Vehicles on screen	How busy and how loud the drums are, and how often the bass plays. Busier traffic also speeds the tempo up, and a lasting change in traffic can move the chords on.
How much of the picture is moving	The bass gets louder and the chords change faster. The drone does the opposite: it swells as the street empties.
Brightness	How bright the drums and the lead sound. On the sparkle voice it works in reverse, and darker scenes ring for longer.
Colour warmth	How bright the drone, the bass and the pad chords sound.
Contrast	How much the drone’s two notes waver against each other.
Movement on the footpaths	How loud the lead is.
People on the footpaths	How often the sparkle voice plays and how loud it is.
Cars and people in balance	Whether the lead plays at all. It plays most when both share the street, and goes quiet when one side has it or the street is empty.
Who has the street	When people have it, the lead and the sparkle voice wander more in pitch. When cars have it, the drums add fills and the lead drifts off the beat.
A vehicle crosses a lane line	One drum hit. The lane picks the drum, and buses and trucks land on the kick.
A person walks from one footpath across a crossing to another	A short run of notes on the sparkle voice that moves left or right with them.
Time of day	The centre of the tempo, and which group of mallet and piano sounds the lead picks from.
Rain, from a weather feed	The pad chords get louder, and the sparkle voice plays more.

The synth is a headless C++ binary with six voices: drone, bass, drum, pad, lead and decorator. It is built on the harness from Pariki, my drum synth in the Moog DFAM tradition, elsewhere on this site. It writes raw PCM to stdout, ffmpeg segments that into HLS, and a page plays the stream.

The detector needed a benchmark before I could tune it. I built one, adjudicating every crop by hand: 667 proposals across five London scenes, resolved into 234 true boxes, 309 don’t-care regions and 124 rejections. Dropping the confidence floor moved overall F1 from 0.55 to 0.67, at 0.29 to 0.38 seconds a frame. Every geometric idea I tried cost 1.5 to 5 times the time for no gain: 1536 and 1920 sweeps, 4 by 4 tiles, half-overlapping tiles, yolo11s, yolo11m, and a causal union of three frames.

Every row also reports precision with the 309 don’t-care regions counted as errors, not forgiven. Quoting only the forgiving number would quote my ground truth’s blind spot. Under that reading the tuned recipe still wins, and the vehicle gain survives.

Two bugs had the station blind before any of that mattered. ultralytics keeps the previous predict() call’s keyword arguments on the model object. My people pass narrowed to classes=[0,1]. My vehicle pass said nothing about classes and silently inherited that narrowing. From the first people pass onward the detector returned pedestrians only. The second is the more embarrassing. broadcast/pipeline.py launched the vision service without the --yolo flag at all, leaving drums, bass and lead riding a source pinned at zero on the live camera.

Over 245 hours the harmony turned 3,323 times, and 243 of those turns came from the street. The other 3,080 came from a timer that fires after 96 bars with no change. A turn counts as city-driven only when a 6-bar and an 80-bar average of normalised vehicle count diverge by 0.22, and real traffic at one junction is smoother than that. The decorator voice never made a sound in the whole archive: its trigger needs a complete tracked walk from footpath to crossing to footpath, and only 33 of those resolved against roughly 43,459 vehicle crossings. A radio station driven by a city mostly plays itself.

The long version

This section is being written.

The raw material for it already exists in the project’s research notes: a survey of the prior art in camera-driven and city-driven generative music, and a record of what the shipped code does not yet contain: an arrangement layer, a key clock, a second station in Shibuya, and the fix for the decorator that stayed silent for the whole soak.