紀要論文

構造化状態空間シーケンスモデルを用いた位置情報の長距離依存関係を利用したバイノーラル音声合成

Binaural Audio Synthesis with the Structured State Space sequence model

jpn

24321192

https://doi.org/10.15002/00030611

法政大学大学院情報科学研究科

法政大学大学院紀要. 情報科学研究科編

19

1

6

2024-03-24

Version of Record

The structured state-space sequence model (S4) is a recent innovation in sequence modeling that has shown excellent performance in handling long-range dependencies across a variety of tasks and modalities. In the field of speech processing, it has been found to be an alternative to the self-attention model in automatic speech recognition and in speech synthesis. In this study, a new model for synthesizing binaural speech is developed that represents the long relationship between mono speech using S4 and the latent state space between speaker and source location information. Each layer is conditioned with information common to both left and right sides of the speech, which is processed by location, binaural time difference, and pre-trained binaural speech. Compared to conventional methods, our model shows that speech synthesis is possible with similar quality. These results indicate that our model has the potential to extend the applicability of S4 in sequence modeling and into the domain of conditional speech synthesis.

Article

AA12746425