Devising a method for evaluating the accuracy of coordinate and photometric measurements of moving objects in computer vision systems

Abstract

This study explores processes that measure and evaluate coordinate and photo-metric characteristics of moving objects in computer vision tasks on digital images and series of frames. The task addressed relates to the lack of a comprehensive formalized method for end-to-end evaluation of the accuracy of coordi¬nate and photometric measurements, which integrates high-fidelity simulation modeling, automatic data identification, and statistical analysis of the laws of error distribution in terms of structurally heterogeneous subsamples. To solve the task, a computational method for evaluating measurement accuracy indicators based on data from the SCVModel simulation modeling system has been devised. The method is based on a formalized representation of observation conditions through a morphological table with the use of cross-consistency testing to cut off physically unrealistic scenarios. An extended set of statis¬tical characteristics has been defined and spatial-energy criteria for grouping measurements into subsamples have been introduced. A distinctive feature of the devised method is that, unlike the generally accepted threshold metrics of integral quality, it enables studying measurement errors. They include full-fledged random variables with the calculation of their mathematical expec-tations, variances, asymmetry, and kur¬tosis. The results indicate that the mecha¬nism behind spatial censoring of subsamples effectively isolates anomalous deviations caused by optical distortion or overlapping of objects. It is quantitatively proven that rejecting 5–10% of such measurements leads to a sharp decrease in the mean square devi¬ation (from 6.30 to 0.40 pixels). The scope of practical use of the method is verification and comparative analysis of moving object detectors. This method is implemented in the COMPARATOR software toolkit (Ukraine) and has been successfully applied to evaluate the Amanita (Ukraine) and YOLOv8 (USA) systems.