CasEnc: An Information Retrieval and Ranking System based on Cascaded Encoders
Keywords:Neural Information Retrieval; Natural Language Understanding; BERT; Deep Language Model; Learning-To-Rank.
The improvements in Natural Language Understanding benefit information retrieval techniques (NLU). One of the driving forces behind the creation of effective Lan- guage Models (LM), which significantly enhanced the functionality of the document retrieval and ranking system, was the development of deep neural networks. Despite these advancements, there are some serious problems with the current information retrieval and ranking system. We are focusing on three of them in this work. The three factors are 1)The system’s maximum document length processing capacity, 2) The cost of prior computation and inference computation, and 3)Cross-document information. The suggested CasEnc can handle larger documents, and because it can process most of the computation independently of the query, its inference computa- tion cost is also low. In addition to this, CasEnc uses multiple documents to create the ranked list of retrieval results, allowing it to encode cross-document information and enhance ranking performance. Comparing the proposed system’s performance to publicly accessible benchmarking datasets produces competitive and encourag- ing results. On the MS-MARCO dataset, the proposed CasEnc model scored 38.2 MRR@10, which is 4.6% higher than the large-BERT model, and 35.1 MAP@1000, which is 4.8% higher than large-BERT on the TREC-CAR dataset. Additionally, the suggested model outperformed the second-best performer by 12.1% with a 40.70 MAP@1000 score.