Maximizing video Data Retrieval Efficiency: Leveraging Lang Chain Integration of RAG with Cohere Reranker for Enhance Performance
- Version
- Download 12
- File Size 435.27 KB
- File Count 1
- Create Date 12 June 2024
- Last Updated 12 June 2024
Maximizing video Data Retrieval Efficiency: Leveraging Lang Chain Integration of RAG with Cohere Reranker for Enhance Performance
MUKKU SUMANTH
B-TECH CSE AMITY UNIVERSITY CHHATTISGARH, INDIA
Abstract - This paper addresses cutting-edge artificial intelligence and natural language processing algorithms for extracting and organizing material from videos.
Audio from videos is extracted, texts are transcribed, and PDFs are generated. The text is split using PyPDFLoader and Lang Chain’s Recursive Character Text Splitter. FAISS and embeddings provide fast similarity searches, while Cohere Reranker uses LLMs to boost search result relevancy. The "command-r" technique in Cohere simplifies the creation of question- and-answer apps. Markdown and qa.invoke are used in the last stage to generate acceptable responses. The results show how well these techniques improve query resolution and multimodal information retrieval.
Key Words: LLM, NLP, Cohere API, RAG
