Combining near-infrared spectroscopic signatures and physical traits based on machine vision to enhance accuracy in identification of the geographical origins of agricultural products

Abstract

This study was the first to explore the combination of near-infrared (NIR) spectroscopy and machine vision to enhance accuracy in the identification of the geographical origins of agricultural products. To ensure accuracy, machine vision recognizing dissimilar physical traits of grains (appearance, shape, and texture) by their geographical origins was combined with NIR analysis. For this purpose, the NIR spectra and visual images of packed grains from six different agricultural products were analyzed. For the image analysis, the Segment Anything Model (SAM) was used to determine the proper cropping sizes for each product because the grain sizes differed by product. When the augmented images were input to Swin Transformer (ST), the average discrimination accuracy was 95.36 % and comparable to that using the NIR data (95.53 %). When multimodal approaches combining the NIR signatures and visual features of the samples were attempted, intermediate and late fusions enhanced the average accuracy to 98.2 %.