Twinity: A Retrieval-Augmented AI Enabled Avatar-Based Co-Pilot Assistant with Multimodal Chat and Voice Interaction

1st Firoz Ahmed Siddiqui

Artificial Intelligence and Data Science Anjuman College of Engineering and

Technology

Nagpur, India 0009-0008-4139-8293

4th Md. Kounen

Artificial Intelligence and Data Science Anjuman College of Engineering and Technology

Nagpur, India 0009-0003-0664-6128

7th Faizan Ahmad Khan

Artificial Intelligence and Data Science Anjuman College of Engineering and Technology

Nagpur, India 0009-0002-7503-9711

2nd Md. Aadil Siddiqui

Artificial Intelligence and Data Science Anjuman College of Engineering and Technology

Nagpur, India 0009-0006-9931-4617

5th M. Amaan Ansari

Artificial Intelligence and Data Science Anjuman College of Engineering and Technology

Nagpur, India 0009-0003-0471-9303

3rd Md. Zaid

Artificial Intelligence and Data Science Anjuman College of Engineering and Technology

Nagpur, India 0009-0002-5694-5567

6th Md. Aaquib Sheikh

Artificial Intelligence and Data Science Anjuman College of Engineering and Technology

Nagpur, India 0009-0001-4642-1834

Abstract— With AI rapidly advancing into many areas of our daily lives, most people are aware that traditional forms of chatbots/virtual assistants have generally helped increase access and automation; however, many of the currently available forms still struggle to provide context-based responses (e.g., be aware of the surrounding environment), support multiple types (i.e., voice, visual, etc.) of communication, or have the ability to create more personalized and engaged responses with a human- like interaction style. Due to the introduction of new technologies, such as Retrieval-Augmented Generation (RAG), vector databases, and neural text-to-speech (or voice synthesis), there is now an opportunity for intelligent assistance to respond more accurately, contextually relevantly, and in a way that feels more human-like than previous generations of chatbots/virtual assistants.