AI Based Doctor-Patient Diarization System Using Speech Recognition and NLP with SOAP Notes Generation
AI Based Doctor-Patient Diarization System Using Speech Recognition and NLP with SOAP Notes Generation
Sanskar Darekar1 ,Vishal Wagh2 ,Yash Dhole3 ,Atharv Borhade4 ,Prof.Shruti Hatwar5
1,2,3,4dept. Computer Engineering PVPIT, Bavdhan
5dept. Computer Engineering Asst. Prof. at PVPIT, Bavdhan
ABSTRACT
Clinical documentation is an essential yet time-consuming task in modern healthcare systems. Doctors often spend a significant amount of time manually recording patient consultations, which increases administrative burden and reduces patient interaction time. This paper proposes an AI-based Automated Clinical Documentation System that converts doctor-patient conversations into structured medical records. The proposed system integrates Whisper Automatic Speech Recognition (ASR) for speech-to-text conversion, pyannote.audio for speaker diarization, spaCy for medical entity extraction, and Large Language Models (LLMs) for SOAP note generation and consultation summarization. The system accurately identifies speakers, transcribes medical conversations, extracts key clinical information such as symptoms, diagnosis, medications, and treatment plans, and automatically generates structured SOAP notes. MongoDB is used for secure storage and retrieval of consultation records. The generated clinical documentation can be accessed through dedicated doctor and patient dashboards, improving healthcare workflow efficiency and reducing manual documentation effort. Experimental results demonstrate the effectiveness of the proposed approach in automating clinical documentation while maintaining accuracy and scalability. The proposed system contributes towards intelligent healthcare management by combining speech processing, natural language processing, and generative AI technologies into a unified platform.
Key Words: Doctor Diarization System, Artificial Intelligence, Natural Language Processing (NLP), Speech-to-Text, Whisper, pyannote.audio, Speaker Diarization, SOAP Note Generation, Medical Conversation Analysis, MongoDB, Healthcare Management System, spaCy, Large Language Models (LLMs), Clinical Documentation, Patient Management System, Role-Based Authentication, Digital Healthcare, Audio Processing.