< Portfolio

TypeTalker

TypeTalker is a "speech-to-text-to-speech" free editing interface developed with Dongwook Yoon and Francois Guimbretiere as part of the RichReview project. The interface allows users to gesture on diagrams while talking, then edit their speech as text freely in a text box. Metadata for the gestures is re-aligned automagically across edits and replayed on playback. Published at CSCW 2017.

In practice, I also built a back-end in Praat which allowed us to graft the speaker's prosody onto the synthesized voice, leading to a hybrid voice. Synthesized voices are used because in certain settings, anonymous voices may be preferred to one's own voice (especially for non-native speakers, or e.g. in future versions of peer review), but the loss of certain prosodic information often leads the voice to sound overly robotic, and struggled to capture the inflection of questions. Grafting select prosodic data back onto the synthesized voice lets us achieve the best of both worlds.