sound.speech

Overview

Lightweight English text-to-speech.

A Klatt-style formant synthesizer with a small rule-based front end: no models, no data files, deterministic output (a pure function of its inputs), DECtalk/SAM-class robotic timbre by design. Everything is pure Lua on top of the sound.core float32 buffer.

local sound = require("sound")
local buf = assert(sound.speech.say("Docking request granted.", {
    voice = "computer", rate = dev:rate(),
}))
mx:play(buf, { gain = 0.8 })

Pipeline (each stage is public — intercept where you need):

    text ──normalize──► tokens ──g2p+prosody──► events ──klatt──► buffer
           speech.phonemize(text, opts) ────────►│
                                                 ▼
           speech.synth(events, opts) ──────────► buffer

events is a flat list of { ph = "AA", stress = 0|1|2, dur = ms, f0 = Hz } (ARPAbet phonemes + SIL pauses). Editing dur/f0 between phonemize and synth is the inflection hook: chants, alarms, hand-fixed pronunciations — hand-set values are never rescaled by prosody.

Voices: opts.voice is a preset name from speech.voices (default "narrator") or a knob table; any knob passed directly in opts overrides the preset: { voice = "computer", f0 = 70 }.

Knob semantics (f0, f0_range, speed, formant, breath, flutter, growl, quantize, gain) are documented in sound.speech.voices.

Output: a MONO buffer at 16 kHz, resampled to opts.rate when given (pass dev:rate(), mirroring sound.wav.load). The mixer upmixes mono to stereo.

Module	Description
sound.speech.g2p	English grapheme-to-phoneme for the formant TTS (sound.speech).
sound.speech.klatt	Klatt-style cascade/parallel formant synthesizer (after Klatt 1980, JASA; simplified).
sound.speech.phonemes	Phoneme inventory for the formant TTS (sound.speech).
sound.speech.prosody	Prosody-lite for the formant TTS (sound.speech).
sound.speech.text	Text normalization for the formant TTS (sound.speech).
sound.speech.voices	Voice presets for the formant TTS (sound.speech) — named bundles of the synthesis knobs.

Functions

Name	Signature
phonemize	phonemize(`s`, `opts`) -> `events`, `err`
synth	synth(`events`, `opts`) -> `buf`, `info`/`err`
say	say(`s`, `opts`) -> `buf`, `info`/`err`
say_job	say_job(`s`, `opts`) -> `job`, `err`

phonemize(s, opts) -> events, err

Transcribe text into editable phoneme events

Runs normalization, G2P and prosody with the resolved voice, returning the event list speech.synth would render — every event carries ph, stress, dur (ms) and f0 (Hz), ready to be edited.

synth(events, opts) -> buf, info/err

Render phoneme events into a sample buffer

The phoneme-level entry: renders an event list (hand-authored or from phonemize).

Missing dur/f0 fields are filled from the phoneme tables and a flat contour at the voice's base pitch.

opts.rate resamples the output. info = { rate, dur (s), frames, events }.

say(s, opts) -> buf, info/err

Speak: text in, sample buffer out

opts field	default	description
`voice`	`"narrator"`	preset name or knob table
`rate`	16000	output sample rate — pass `dev:rate()`
any knob	from voice	`f0`, `f0_range`, `speed`, `formant`, `breath`, `flutter`, `growl`, `quantize`, `gain` override the voice (see `sound.speech.voices`)

Deterministic: the same text and opts always yield identical samples.

say_job(s, opts) -> job, err

Speak incrementally: a sliced render job

The cooperative sibling of say for callers living inside one event loop: a long line renders tens of ms of CPU, and a monolithic say blocks everything sharing the thread (an audio writer task starves, a frame stalls). The job slices that work: each job.step() renders opts.yield_every parameter frames (default 40 ≈ 200 ms of speech, ~1 ms of work) and returns

nil: still rendering — call again later
buf, info: done (the final step resamples to opts.rate, one C call)
false, err: failed — the job is dead

The stepped output is byte-identical to say(s, opts): the render state lives in locals a yield cannot touch. job.done() reports completion without stepping.

Overview

Submodules

Functions