intake — design

Goal

Rebuild a lost music collection from a staging directory of albums downloaded from Soulseek. The output is two trees: lossless masters in archive/, and a streamable library in library/ transcoded from those masters. The archive is the thing that has to last, so everything is verified before it lands.

This is a personal tool written for that one job, not a general-purpose one. See "How this was built" in the README for what that means in practice.

Decisions

Archive	<root>/archive/<artist>/<album>/<disc>-<track>_<title>.flac plus cover.jpg
Library	<root>/library/<artist>/<album>/ — Vorbis at -q:a 9, optimised cover
Tag whitelist	TRACKNUMBER, TRACKTOTAL, TOTALTRACKS, DISCNUMBER, DISCTOTAL, TOTALDISCS, TITLE, ALBUM, ARTIST, ALBUMARTIST, DATE, nothing else
Tags are computed	every field is recomputed by intake, never copied from the source
Compilations	various-artists/<album>
Cover sources	embedded art, folder image, Cover Art Archive, Deezer, iTunes. No Spotify
Placeholder art	always cover.jpg, flagged in the journal so library verify can list it
Artist memory	confirmed decisions go in <root>/artists.toml, keyed by the raw tag credit
Writes	staged in a temp dir inside archive/, verified, then promoted atomically
VCS	jj, unlicensed (public domain)

The totals were added against the original seven-field list. They describe the shape of the release rather than offering an opinion about it, and intake counts them from the files it is writing, so they cannot contradict the album. Both spellings are written because players disagree about which they read.

External facts

Verified on this machine, not assumed. Each one cost time to find.

ffmpeg 9.0.1 cannot tag Opus

• -c:a libopus -metadata TITLE=... writes an OpusTags header containing only the vendor string. The tags are silently dropped.
• ffprobe cannot read Opus tags either.
• ffmpeg cannot attach cover art to Opus or Ogg Vorbis at all: Unsupported codec id in stream 1.
• vorbiscomment refuses Opus outright.

The container itself is fine — a hand-built OpusTags packet with a recomputed Ogg CRC reads back correctly in taglib — but hand-writing a tagger to work around a broken toolchain is not worth it. The library is Ogg Vorbis, which vorbiscomment handles correctly today. Opus stays a config option for later; the masters are lossless, so the library can be regenerated in any codec.

Vorbis -q:a 9 beats Opus 160k

Measured on an 8 s synthetic signal (pink noise, 55/440/3000 Hz tones, tremolo), decoded to 44.1 kHz mono, aligned to the source by integer sample-lag search:

Format	Size	SNR	Peak error
Vorbis -q:a 6	100 kB	20.6 dB	0.067
Vorbis -q:a 8	121 kB	23.5 dB	0.041
Vorbis -q:a 9	153 kB	27.3 dB	0.028
Vorbis -q:a 10	239 kB	33.8 dB	0.012
Opus -b:a 160k	148 kB	15.9 dB	0.089
Opus -b:a 256k	238 kB	16.2 dB	0.084

All encodes measured the same integrated loudness, so none of them shifts level.

Honest caveats: a tone/noise signal flatters Vorbis's MDCT and punishes Opus's speech-oriented tuning, so on real music the two converge. Opus decodes at 48 kHz while Vorbis keeps 44.1 kHz, so part of Opus's error is an extra resample. This is a worst-case probe, not a universal claim — but it is enough to justify the default.

Kitty graphics

Cover previews use a virtual placement (U=1) so the image occupies real cells in the layout instead of being painted over the interface. The placeholders are ordinary text, so the host application can move them around without knowing anything about graphics.

Things that are easy to get wrong, all of which produced a silently blank preview:

• The image id lives in the cell's foreground colour, not in extra runes. A cell is the placeholder rune plus three diacritics: row, column, and the id's high byte. The low 24 bits go in an SGR foreground colour.
• The transmission declares f=100, which means PNG and nothing else. The protocol has no JPEG format, so a JPEG handed over unchanged is refused with EBADPNG:Not a PNG file. Embedded art in FLAC files is usually JPEG, so this is the common case.
• The renderer quantises colours to the terminal's profile. Under a 256-colour profile an id of 7800 is emitted as palette colour 18, which matches no image. The interface therefore imposes a truecolor profile when drawing images, which is safe because the protocol is only selected for terminals that support it.
• charmbracelet/x/ansi/kitty exposes only the first six row/column diacritics and silently returns the first for anything past five. The full table (297 marks, generated by scripts/gen-diacritics.py) is carried in internal/tui/diacritics.go.
• A placeholder cell must not be measured as ordinary text. Counting runes calls a one-column cell four, and the Unicode width tables call U+10EEEE three. Either makes the grid measure wider than it is and get clipped.
• The medium is t=f, not t=t; both name a path in the payload, but t=t claims a temporary file the terminal may delete.
• No C=1. It means "do not move the cursor after placing", which is meaningless for a virtual placement — it has no cursor position, it exists only in cells.

Transmissions go through the renderer as tea.Raw, never straight to stdout. Writing directly races the frame being drawn and produces nothing at all.

kitten icat --unicode-placeholder --transfer-mode=file is the reference to compare against, and it is the only reason any of this was found.

Pipeline

0. Discovery — list albums under staging/, ordered, with a filter.
1. Metadata — read tags, whitelist them, resolve album artist, report what will change. Album-level values are only used when every track agrees.
2. Cover art — gather candidates, show real previews, let the user pick, or take a placeholder.
3. Track naming — disc-track_title.ext, zero-padded to 2 digits widening to 3 at 100+ tracks, truncated on a dash boundary at 200 bytes.

Names are lowercase, dashes between words, underscores between sections. Symbols that every filesystem accepts — ( ) [ ] + & — are kept, so a title stays readable rather than collapsing into a run of dashes. Symbols that are illegal somewhere, which is to say the Windows set < > : " / \ | ? *, cannot be kept: inside a word they become an underscore (while(1<2) becomes while(1_2)) so the two characters they sat between stay distinct, and beside a space they are just a separator (Album: Title becomes album-title) so the words don't glue together as album_title. A name with no letters or digits in it counts as unusable and goes to the name editor, the same as an empty one. 4. Write masters — copy into a temp dir inside archive/, tag, rename, embed the cover, verify, then promote atomically and delete the staged originals. 5. Transcode — build the streaming copy from the promoted masters, with an optimised cover derived from the one the user actually chose. 6. Report — paths, counts, sizes, warnings, placeholder flag.

Verification before promotion checks count, codec, duration, tags, non-whitelist leakage, and decodes the first, middle and last file. A failure at any point leaves staging untouched.

An album is built in a temporary directory beside its destination and moved into place with a single rename, so nothing half-written is ever visible under the real name. Replacing or merging an existing album moves the old one aside first and puts it back if the new one cannot land; a merge checks every name for a clash before it moves anything. The library copy is swapped the same way.

Staging is cleared only after the album has been promoted, and a staged file that cannot be removed is reported without turning a finished archive into a failure. Cleanup removes the album's own folder, its leftover sidecars and any subdirectory it has emptied, and stops at the staging directory: an empty staging directory is the normal state, not clutter.

If the album's destination is a directory that already exists, the new copy is filed alongside it as <album> (2), and everything after that — the summary, the journal and the library built by the same run — uses the directory the album actually landed in.

Artist folders

Resolved from the archive tree, the journal and artists.toml, in that order. An exact folder name, a recorded alias, or a romanised spelling is followed without asking; every decision is remembered, so the same artist always lands in the same folder.

Similar-but-different names are not acted on. A fuzzy match ("Blur" vs "Blue", "The Cure" vs "The Cult", "RTL Artist" vs "Art Artist" — all at or above the 75% score) is listed interactively and can be chosen there, but it is never taken automatically: filing an album under another artist's folder is silent, passes every later check because nothing compares a folder name against ALBUMARTIST, and is permanent once the guess is written to artists.toml. An unattended run creates the folder the credit names and prints the exact artists.toml line to add if the two really are the same artist. A duplicate artist folder is visible in a directory listing; a wrong one is not.

Nothing is guessed silently. An unknown album or artist, an empty normalised name, two different titles collapsing to one name, or two tracks that would be written to the same file are all hard issues, and --yes refuses to proceed rather than inventing metadata.

Renaming and merging artists

Two spellings of one artist is the most common mess a rebuild produces. Fixing it by hand means moving two directories, editing every journal record underneath them, and finding every artists.toml entry that named the old name; missing any one leaves the journal describing directories that are not there, which library verify then reports as albums that have gone missing.

artist rename refuses when the destination already exists and says to merge instead. artist merge refuses when an album of that name is already in the destination, before it moves anything. Both move rather than copy, and both repoint the journal and the aliases.

The journal

One record per staging path, holding where the album ended up and what is known about it. Everything that reads it treats it as the only index of what has been filed, so two rules follow.

Every change is a read, a change and a write under a lock file beside it, and each save lands through a uniquely named temporary file. Without both, two runs on one root each wrote back a snapshot taken before the other, and an album could be filed while its record vanished.

A record is only deleted when the album it points at has gone. A run that fails is not evidence about an album recorded earlier.

Because the journal is the only index, the archive is also read and compared against it: an album that reached the archive without a record — the process was killed between the promotion and the write, or somebody copied it in — would otherwise be invisible, and a later run would file a second copy beside it. library verify and library audit list those, and the journal screen marks them.

An archive can also exist without a journal: albums filed by an older version, albums copied in by hand, or a record that was lost. library adopt builds records for those from the masters' own tags, because the folder names may have been normalised on the way in and no longer say what the album was called. It writes nothing to the audio, and an adopted album with no cover.jpg is flagged as needing art rather than looking finished.

Records can be removed on purpose. Forgetting one touches no files at all; un-filing moves the masters back to staging and removes the library copy, which is derived. Un-filing moves rather than copies, because after filing the archive holds the only copy of that audio.

Already having it

The destination path is not enough to answer "do I already have this": a re-download whose artist folder is spelled differently lands somewhere else entirely, and the user finds out after reviewing the cover of an album they already own. An album is matched to the journal on the folded artist and album names, and to the disk on the destination path, and the finding is shown before the review starts rather than at the end.

Keeping both is the default, because it is the only choice that cannot lose an album. It files the new copy beside the old one as <album> (2), and everything after that — the summary, the journal and the library built by the same run — follows the album to where it actually went.

Filing unattended

batch files every ready album without stopping for a decision. An album that needs a person is parked — recorded with the reason, left exactly where it is — and the run moves on. That is the whole point: one mistagged mp3 out of two hundred downloads must not hold up the other hundred and ninety-nine, and the album it parks has to still be findable afterwards, which it is, because the park is a journal record rather than a line in a log.

A genuine failure is reported and also not fatal, because the next album has nothing to do with it. The command exits non-zero at the end if anything failed, so a script can tell the difference between "finished" and "finished with problems".

Cover art

Candidates come from embedded pictures, images beside the tracks, images one level down (which is how scans arrive, in a Scans/ or Artwork folder), a configured local folder, and then the network sources. Art beside the tracks wins outright; the subfolder is only a fallback, so a stray back cover in extras/ cannot displace the real front cover.

library re-cover --placeholders runs the fetch again for every album still on a generated placeholder. Only a confident match is taken: replacing a placeholder with the wrong sleeve is worse than keeping the placeholder, so anything less is left alone and reported.

Replacing an album's art rebuilds its library copy, because the library has its own cover.jpg and its own embedded picture. Without that, the placeholder list stops mentioning the album while the copy people actually stream still carries the placeholder.

The album sheet

Everything intake is about to write is editable in one $EDITOR document, reachable from every screen that has a plan loaded except the one that is filing it. That includes the cover picker, because choosing a cover is when a wrong album artist tends to be noticed.

The format is field: value lines, whole-line # comments, and ## n to begin a track. Nothing is positional: a block carries its own numbers, so a block can be moved or deleted without the rest shifting, and a value may contain # and : freely. It replaced an editor that could change only the two folder names and the text after each track number — and whose "delete a line to skip the file" did not actually skip it.

Track identity is the number after ##, which is the track's position in the plan when the sheet was written. It is not the file name, which the user may change, nor the track number, which they may change, nor the row order, which they may sort.

Deleting a block drops that file from the album and the totals are counted again. A folder that still holds the name intake derived it from follows a rename of the album or artist; one that was typed on purpose does not.

Repairing drift

library audit compares the journal, the archive and the library, and reads the archive against the journal so an album that reached the archive without a record is visible rather than invisible. --fix rebuilds the library copies that are missing or short, and restores a missing cover.jpg by copying the archive's rather than re-encoding the album.

It never writes to the archive: the masters are the one thing a rebuild cannot replace, so a missing or empty archive directory is reported and left strictly alone.

Interface

Bubble Tea v2 (charm.land/bubbletea/v2), Bubbles v2 and Lipgloss v2. Screens: browse, review (metadata / track names / cover art / confirm), cover picker, working, done.

The interface sets App.Quiet and takes progress from the pipeline's return values. stdout is its drawing surface, so a progress line from the pipeline lands in the middle of a frame and corrupts it.

Not done

• Sixel and iTerm2 image protocols.
• Opus as a library codec.
• Markdown panes in the interface.