Click here to download the project abstract of artificial ai.
Abstract:
Language-queried audio source separation (LASS)
is a new paradigm in advanced deep learning for computational auditory scene analysis
(CASA). artificial ai LASS aims to separate a target sound from an audio
mixture given a natural language query, which provides a natural
and scalable interface for digital audio applications. Recent works
on LASS, despite attaining promising separation performance
on specific sources (e.g., musical instruments, limited classes of
audio events), are unable to separate audio concepts in the open
domain. In this work, we introduce Audio Sep, a foundation model
for open-domain audio source separation with natural language
queries. We train Audio Sep on large-scale multimodal datasets
and extensively evaluate its capabilities on numerous tasks including audio event separation, musical instrument separation, and speech enhancement. Audio Sep demonstrates strong separation
performance and impressive zero-shot generalization ability using
audio captions or text labels as queries, substantially outperforming previous audio-queried and language-queried sound separation models. For the reproducibility of this work, we released
the source code, evaluation benchmark and pre-trained model
at: https://github.com/Audio-AGI/AudioSep.
Index Terms—sound separation, language-queried audio
source separation (LASS), natural language processing. Recently, universal sound separation (USS) [4] has attracted a lot of research interest.USSaims to separate arbitrary sounds in real-world sound recordings. Separating every sound from a mixture is challenging due to the wide variety of sound sources existing in the world. As an alternative, query-based sound separation (QSS) has been proposed which aims to separate specific sound sources condition edonapiece of query information.QSSallowsusers
