紀要論文

クローズドブック質問応答における言語モデルの知識強化

Enhancing Knowledge of Language Model in Closed-Book QA

jpn

24321192

https://doi.org/10.15002/00030622

法政大学大学院情報科学研究科

法政大学大学院紀要. 情報科学研究科編

19

1

6

2024-03-24

Version of Record

Language models such as Chat-GPT generate new answers from the data they learn, improving work efficiency and generating ideas. On the other hand, incorrect answers may be generated, and this is likely due to the language model not being able to maintain accurate knowledge. In this research, we aim to provide language models with more accurate knowledge by devising a learning method rather than increasing the scale of the model. Specifically, we propose Title Prediction as an intermediate pretraining for Closed-book question answering (QA). Title Prediction learns how to output the title of an article from any two paragraphs on Wikipedia. By doing this, we hope to improve the relationship between separate paragraphs and embed accurate knowledge about titles into the model, even for paragraphs where titles are omitted. In my experiments, we confirmed that title prediction improves the performance of Closed-book question answering.

Article

AA12746425