Text normalization for sound.speech. It turns a string into the token
list the grapheme-to-phoneme stage reads:
{ word = "hello" [, emph = true ] } a lowercase word: a-z and apostrophe
{ punct = "." | "," | "?" | "!" } a phrase or sentence boundary
It spells out numbers (integers, decimals, negatives: "42" becomes "forty
two"; a comma between digit groups of three is a thousands separator),
reads a token mixing letters and digits run by run ("pad3" is "pad
three"), expands a small table of abbreviations, and reads *word* as
emphasis. .;:!? and em-dashes become boundaries; other punctuation is
dropped.
| Name | Signature |
|---|---|
| number_words | number_words(s) -> words |
| normalize | normalize(s) -> tokens |
number_words(
s) ->words
Expand a number literal into English words
s is a decimal literal: "42", "3.5", "-7". Returns lowercase
words: { "forty", "two" }. Integers expand up to the billions. Digits
after the point are spelled one by one after "point".
normalize(
s) ->tokens
Normalize a text string into word/punct tokens