AI-Driven Secure Top-K Association Rule Mining for Horizontally and Vertically Distributed Big Data Using Hybrid Machine Learning Frameworks

Authors:

Dr. Santosh Kumar Byraboina

Associate Professor, Wesley Pg College, Secunderabad, Telangana, India.

bskumar7@gmail.com

ABSTRACT

 The unprecedented expansion of distributed big data originating from e-commerce ecosystems, healthcare infrastructures, cloud computing environments, financial platforms, and social networking services has introduced substantial challenges in secure knowledge extraction and large-scale transactional analytics. Conventional association rule mining techniques, including Apriori and FP-Growth algorithms, encounter severe limitations such as excessive computational overhead, repeated database scanning, poor scalability, extensive candidate generation, and inadequate privacy preservation in distributed database environments. Moreover, most existing methodologies independently address either vertically or horizontally partitioned databases, thereby restricting their effectiveness in heterogeneous and large-scale distributed systems.

This research presents a secure Top-K association rule mining framework for vertically, horizontally, and hybrid distributed databases using optimized distributed mining and privacy-preserving computational strategies. The proposed framework integrates secure distributed frequency counting mechanisms, enhanced optimization models, and protected transactional communication protocols to achieve efficient frequent itemset extraction without compromising sensitive information. An enhanced FP-Growth architecture combined with Genetic Optimization and Particle Swarm Optimization techniques is employed to improve mining precision, scalability, execution efficiency, and rule selection performance while minimizing redundant pattern generation.

To reinforce transactional confidentiality and secure distributed processing, the framework incorporates cryptographic security mechanisms, encrypted communication protocols, and privacy-preserving distributed operations during data partitioning and mining procedures. Experimental analysis demonstrates that the proposed framework substantially reduces execution time, communication complexity, and memory utilization while significantly improving support confidence, precision, recall, scalability, and overall mining efficiency compared with conventional Apriori and FP-Growth approaches. The proposed methodology provides a secure, scalable, and computationally efficient solution for distributed data mining applications in healthcare analytics, cloud infrastructures, cybersecurity systems, financial intelligence, and large-scale transactional environments.

Keywords: Association Rule Mining, Top-K Frequent Itemset Mining, Hybrid Data Partitioning and Scalable Data Mining