muvid.align

Lyric → audio alignment.

We have:

• a transcript (Scribe / faster-whisper) with word-level (text, start, end)

• a user-edited LyricsDoc with section labels + line text + optional manual line-start anchors

We want a lacing store with three tiers (sections, lines, words) so the rest of the system can ask “which lines fall in shot X” without re-implementing interval math.

Strategy (greedy token-match):

1. Tokenize each lyric line into normalized words.

2. Walk the transcript word stream once, assigning each transcript word to the next unmatched lyric word that matches (case- and punctuation-insensitive). Tolerate small mismatches (transcript word missing in lyrics, vice versa) with a small lookahead window.

3. From the matched words, derive line [start, end] as (first_matched_word.start, last_matched_word.end). If a line has no matched words, fall back to the user’s manual anchor (if any), then to a linear interpolation between neighboring anchored lines.

4. Sections inherit [start, end] from the union of their lines; if the user provided explicit start_s / end_s on a section, those win.

The result is written as a lacing.SqliteStore so it round-trips and can be edited by other tools.

Module Attributes

AlignerName	Built-in aligner names.
WHISPERX_LITE_MODEL_SIZE	tiny is fast but, on sung/repetitive audio, produces a transcript so garbled that the greedy matcher walks past whole minutes before finding a plausible match (muvid#101: measured 53 s late on a repeated-word lyric).

Functions

align_lyrics(lyrics, transcript, *[, ...])	Align a LyricsDoc to a transcript.
align_scribe_greedy(lyrics, transcript, *[, ...])	Greedy token-match against a word-timestamped transcript.
align_stars(lyrics, transcript, *[, duration_s])	Singing-specific alignment (STARS / similar).
align_user_provided(lyrics, transcript, *[, ...])	Use line-level timings the caller has already determined.
align_whisperx_lite(lyrics, transcript, *[, ...])	Local-only aligner that re-uses faster-whisper (no API).
aligner_info(name)
list_aligners()	Return all registered aligner names, sorted.
register_aligner(name, fn, *, description[, ...])	Register an aligner under name.
write_alignment_store(alignment, *, path[, ...])	Write alignment to a lacing.SqliteStore file (.annot).

Classes

AlignerSpec(*, name, description, fn[, requires])	One row in the aligner registry.
AlignmentResult(*, sections)
LineAlignment(*, line_index, section_label, ...)
SectionAlignment(*, label, title, start_s, ...)
WordAlignment(*, line_index, token_index, ...)	One alignment between a lyric token and a transcript word.

muvid.align.AlignerName

Built-in aligner names. align_lyrics(aligner=...) accepts these plus any name added later via register_aligner().

class muvid.align.AlignerSpec(*, name, description, fn, requires=())

Bases: object

One row in the aligner registry.

class muvid.align.AlignmentResult(*, sections)

Bases: object

lines_in(start_s, end_s)

Lines that fall (at least partially) inside [start_s, end_s].

Return type:

list[LineAlignment]

class muvid.align.LineAlignment(*, line_index, section_label, text, start_s, end_s, word_alignments)

Bases: object

class muvid.align.SectionAlignment(*, label, title, start_s, end_s, lines)

Bases: object

muvid.align.WHISPERX_LITE_MODEL_SIZE = 'small'

tiny is fast but, on sung/repetitive audio, produces a transcript so garbled that the greedy matcher walks past whole minutes before finding a plausible match (muvid#101: measured 53 s late on a repeated-word lyric). The lyric-video path is happy to wait, so default to a materially more accurate size; override per call or via this env var.

class muvid.align.WordAlignment(*, line_index, token_index, text, start_s, end_s, confidence=1.0)

Bases: object

One alignment between a lyric token and a transcript word.

muvid.align.align_lyrics(lyrics, transcript, *, duration_s=0.0, aligner='scribe-greedy', **aligner_kwargs)

Align a LyricsDoc to a transcript.

Parameters:

• lyrics (LyricsDoc) – User-edited lyrics document (the ground-truth text).

• transcript (dict) – Aligner-specific input. For "scribe-greedy" this is a Scribe / faster-whisper response with words: [...]. For "user" this can be empty if you pass user_line_timings=.... For "whisperx-lite" the transcript is ignored — the aligner runs on the audio directly (path passed via audio_path=).

• duration_s (float) – Used to extrapolate end times for lines with no matched words and no later anchor.

• aligner (str) – Name of a registered aligner. See list_aligners().

• **aligner_kwargs – Forwarded to the aligner.

Return type:

AlignmentResult

Returns:

AlignmentResult.

muvid.align.align_scribe_greedy(lyrics, transcript, *, duration_s=0.0, lookahead=6)

Greedy token-match against a word-timestamped transcript.

Cheap, network-only (assumes the transcript came from Scribe or similar). Tolerates 1-character mishears between sung and written text. duration_s is used only when extrapolating end times.

Return type:

AlignmentResult

muvid.align.align_stars(lyrics, transcript, *, duration_s=0.0, **kwargs)

Singing-specific alignment (STARS / similar). Not yet implemented.

See misc/docs/alignment_references.md for the literature motivating this aligner. The slot exists so callers can already write aligner="stars" and get a clear NotImplementedError.

Return type:

AlignmentResult

muvid.align.align_user_provided(lyrics, transcript, *, duration_s=0.0, user_line_timings=None)

Use line-level timings the caller has already determined.

Useful when the user has hand-anchored every line, or when an external aligner has produced line_index → (start, end) timings and you don’t want any token-matching.

user_line_timings is a list of {"line_index": int, "start_s": float, "end_s": float}. If omitted, this aligner falls back to the manual anchors already on lyrics (the // 12.5 end-of-line markers in lyrics.md).

Return type:

AlignmentResult

muvid.align.align_whisperx_lite(lyrics, transcript, *, duration_s=0.0, audio_path=None, model_size='small', lookahead=6)

Local-only aligner that re-uses faster-whisper (no API).

The transcript argument is ignored when audio_path is given: we re-transcribe locally with faster-whisper and then run the same greedy match used by scribe-greedy. When audio_path is not given, we fall through and just use the supplied transcript.

Trade-offs vs scribe-greedy: free + offline; slower; less accurate on singing; needs torch + faster-whisper installed.

model_size defaults to WHISPERX_LITE_MODEL_SIZE ("small", overridable via MUVID_WHISPERX_LITE_MODEL_SIZE) rather than faster-whisper’s own "tiny" default — see muvid#101.

Return type:

AlignmentResult

muvid.align.list_aligners()

Return all registered aligner names, sorted.

Return type:

list[str]

muvid.align.register_aligner(name, fn, *, description, requires=())

Register an aligner under name.

The function should accept (lyrics, transcript, *, duration_s, \*\*kw) -> AlignmentResult. Extra keyword arguments forwarded by align_lyrics() are passed through.

Return type:

None

muvid.align.write_alignment_store(alignment, *, path, asset_id='song:audio', rate=1000)

Write alignment to a lacing.SqliteStore file (.annot).

Uses lacing.tracks.subtitle.SubtitleBuilder for the standard (sections, lines, words) tier set.

Return type:

None