Authors :

Haider Sadiq and Nabeel Sharif

Address :

Department of Computer Engineering Technologies, Al-Kitab University, Iraq haidedsadiq@gmail.com

Abstract :

The potential for changing agriculture and tackling the issues of sustainable food production is significant with the combination of big data analytics and advanced data mining techniques. This study introduces a system that integrates Apache Spark, a distributed computing platform, with dual clustering, a data mining technique that enables the simultaneous clustering of data instances and features. The proposed framework aims to achieve scalability and efficiency. By utilizing the parallel processing capabilities of Apache Spark and the ability of dual clustering to reveal intricate patterns, this methodology facilitates the examination of extensive and diverse agricultural datasets originating from various sources such as weather stations, soil sensors, satellite imagery, farm machinery, and manual records. The objective of the proposed framework is to detect uniform clusters of farms, fields, or crops, as well as the corresponding feature subspaces that define them. This will offer practical insights for making data-based decisions and promoting sustainable farming methods. During the analysis, a dual clustering process is incorporated with Apache Spark that uses the master and worker nodes to predict the clusters with maximum efficiency. The clustering process is performed until convergence is reached, which is used to make effective decisions while analyzing a large volume of data. Then, the system's excellence is evaluated using experimental results and discussions.

Keywords :

Agriculture, big data analytics, data mining, Apache Spark computing environment, dual clustering, weather station, soil sensors and farm machinery.