IoT-Enabled Multi-Modal AI System for Detecting and Predicting the Freshness and Ripeness of Produce

Abstract

Accurate evaluation of the quality of produce is essential for food safety, consumer acceptance, and reduction of post-harvest losses. Manual inspection and chemical tests, which are considered traditional methods, are subjective, tedious, and destructive. The proposed approach is a Multi-Modal AI System that detects and predicts the freshness and ripeness of produce through a non-destructive intelligent system, helping to overcome these limitations. The methodology employs a two-stage hierarchical pipeline that integrates visual information from a camera module with data from a sensor array comprising MQ4 (methane), MQ135 (CO2/ammonia), TGS2602 (Volatile Organic Compounds), and an AS7263 Near- Infrared (NIR) spectrometer. In Stage 1, an object detection model identifies the type of produce. In Stage 2, visual features extracted by a Convolutional Neural Network (CNN) are combined with normalized sensor-array readings in a novel early fusion model to determine freshness and ripeness. The findings clearly demonstrate the strength of data fusion, with the multi-modal framework achieving an overall prediction accuracy of 95%. This approach outperformed the unimodal methods, which achieved accuracies of 93% for image-only and 88% for sensor-only data, respectively. Additionally, the Vision Transformer (ViT) model, designed specifically to detect artificially ripened fruit, achieved accuracies of 94% for bananas and 92% for mangoes. The system is deployed through three applications, including a “GreenMate” physical device, an “AI Food Assistant” mobile application, and a standalone portable device specifically designed for the classification of artificial ripening in mangoes and bananas. Together, these solutions provide a low-cost, real-time, and practical approach to enhancing supply chain transparency, protecting consumers, and reducing food waste.