Click here to download the project base paper of the language translation project.
Abstract:
We investigate the project of deep learning – language translation of simultaneous machine translation of long-form speech content. We target a continuous speech-to-text scenario, generating translated captions for a live audio feed, such as a lecture or play-by-play commentary. As this scenario allows for revisions to our incremental translations, we adopt a re-translation approach to simultaneous translation, where the source is repeatedly translated from scratch as it grows. This approach naturally exhibits very low latency and high final quality, but at the cost of incremental instability as the output is continuously refined. We experiment with a pipeline of industry-grade speech recognition and translation tools, augmented with simple inference heuristics to improve stability. We use TED Talks as a source of multilingual test data, developing our techniques on English-to-German spoken language translation. Our minimalist approach to simultaneous translation allows us to easily scale our final evaluation to six more target languages, dramatically improving incremental stability for all of them.
Recent improvements in neural voice and text processing have significantly improved speech translation and even simultaneous speech translation. Many of these advancements take the shape of specialized models and training processes that function under a streaming constraint, where each update to the translation must finish the preceding translation. However, since the output mode for the translation is text (rather than, say, speech), we can make minor changes to earlier forecasts without being too disruptive to the user. Because of the flexibility to edit prior partial translations, just re-translating each successful source prefix is a realistic technique. This approach has the advantages of minimal latency (since it always attempts a translation of the entire source prefix) and high final translation quality because it is free to produce its final translation with knowledge of the entire source phrase.
