AI Enhanced Data Quality in Data Warehouses and Data Lakes for Efficient Data Driven Intelligence
- Version
- Download 14
- File Size 339.50 KB
- Download
AI Enhanced Data Quality in Data Warehouses and Data Lakes for Efficient Data Driven Intelligence
Authors:
Kiran Veernapu,
Kiran_veernapu@yahoo.com
Abstract: Data quality is paramount in data-driven decision-making processes, especially when dealing with large volumes of data in environments like data warehouses and data lakes. These systems store vast amounts of raw and processed data from multiple sources, making data management and quality assurance complex yet critical. With the growing adoption of Artificial Intelligence (AI), new techniques and tools have emerged that can significantly enhance data quality. This paper discusses how AI can improve the quality of data within both data warehouses and data lakes by automating data cleansing, validation, anomaly detection, and ensuring consistency. It explores the benefits, challenges, and methodologies for integrating AI tools into these systems.
Keywords: Data quality, AI in data quality, data warehouse, data lakes, big data, data processing, data cleansing, data profiling.