An Artificial Intelligence System for Combined Fruit Detection and Georeferencin

An Artificial Intelligence System for Combined Fruit Detection and Georeferencing, Using RTK-Based Perspective Projection in Drone Imagery

This work presents an Artificial Intelligence (AI) system, based on the Faster Region-Based Convolution Neural Network (Faster R-CNN) framework, which detects and counts apples from oblique, aerial drone imagery of giant commercial orchards. To reduce computational cost, a novel precursory stage to the network is designed to preprocess raw imagery into cropped images of individual trees.Unique geospatial identifiers are allocated to these using the perspective projection model. This employs Real-Time Kinematic (RTK) data, Digital Terrain and Surface Models (DTM and DSM), as well as internal and external camera parameters. The bulk of experiments however focus on tuning hyperparameters in the detection network itself. Apples which are on trees and apples which are on the ground are treated as separate classes. A mean Average Precision (mAP) metric, calibrated by the size of the two classes, is devised to mitigate spurious results. Anchor box design is of key interest due to the scale of the apples. As such, a k-means clustering approach, never before seen in literature for Faster R-CNN, resulted in the most significant improvements to calibrated mAP. Other experiments showed that the maximum number of box proposals should be 225; the initial learning rate of 0.001 is best applied to the adaptive RMS Prop optimiser; and ResNet 101 is the ideal base feature extractor when considering mAP and, to a lesser extent, inference time. The amalgamation of the optimal hyperparameters leads to a model with a calibrated mAP of 0.7627.

基于RTK的无人机影像透视投影结合水果检测和地理配准的人工智能系统

这项工作提出了一个基于快速区域卷积神经网络（Faster R-CNN）框架的人工智能（AI）系统，该系统可以从大型商业果园的倾斜，空中无人机图像中检测和计数苹果。为了减少计算成本，网络的一个新的前期阶段被设计为将原始图像预处理为单个树木的裁剪图像。.. 使用透视投影模型将唯一的地理空间标识符分配给这些标识符。它使用实时运动（RTK）数据，数字地形和曲面模型（DTM和DSM）以及内部和外部摄像机参数。但是，大量实验着重于调整检测网络本身中的超参数。树木上的苹果和地面上的苹果被视为单独的类。设计了通过两类的大小校准的平均平均精度（mAP）度量，以减轻虚假结果。由于苹果的大小，锚盒的设计非常重要。因此，k均值聚类方法在Faster R-CNN的文献中从未见过，它对标定的mAP进行了最显着的改进。其他实验表明，箱式提案的最大数量应为225；0.001的初始学习率最好应用于自适应RMS Prop优化器；ResNet 101是考虑mAP并在较小程度上考虑推理时间时的理想基础特征提取器。最佳超参数的合并导致模型的mAP校准值为0.7627。 （阅读更多）