Method of Intelligent Assessment of the Quality of Open Data with Automated Generation of Recommendations and Reports

Abstract

Introduction. The rapid growth of open data published by public authorities, research institutions and business organizations increases the need for reliable mechanisms of data quality assessment. Although open data are formally available for reuse, their practical analytical value is often limited by missing values, structural inconsistencies, format errors, semantic contradictions, duplicates and anomalous observations. These defects reduce the reliability of decision-making, complicate integration into digital platforms and weaken the effectiveness of intelligent services based on open datasets. Existing approaches to data quality assessment are often fragmented, static and mainly descriptive. They usually focus on isolated metrics and do not provide a complete automated cycle that includes defect detection, quality improvement recommendations and analytical reporting. Therefore, the development of intelligent methods capable of combining formal validation rules, statistical procedures and machine learning algorithms is an urgent scientific and practical task.

The purpose of the paper is to develop a method for intelligent assessment of open data quality with automated generation of recommendations and analytical reports. The proposed method is aimed at increasing the analytical usability and practical value of open datasets by integrating multidimensional quality metrics, adaptive weighting, anomaly detection, missing value imputation and interpretable reporting within a single methodological and software framework.

Methods. The study proposes a mathematical model for intelligent open data quality assessment. The model represents a dataset as a set of records and attributes and evaluates it through a vector of quality metrics, including completeness, consistency, correctness, uniqueness and semantic validity. Each metric is calculated using a combination of rule-based validation, statistical analysis and machine learning techniques. A key feature of the approach is the construction of an integral quality indicator based on adaptive metric weights. Unlike traditional models with fixed weights, the proposed method adjusts the importance of each metric according to the detected defects, variability of indicators and their influence on the analytical suitability of the dataset. The model also includes a defect function for identifying missing values, anomalies and format inconsistencies, as well as a recommendation function that transforms diagnostic results into corrective actions. To verify the proposed method, a modular software prototype was developed. It supports data loading, preprocessing, quality assessment, defect detection, recommendation generation and automated analytical reporting.

Results. Experimental validation confirmed the effectiveness of the proposed approach. After applying the developed method, the integral quality indicator of the open dataset increased from 0.9362 to 0.9477, which corresponds to an improvement of approximately 1.23% compared with the initial level. This result is significant because the original dataset already had a relatively high quality score, making further improvement difficult without intelligent processing. The most noticeable improvement was observed in the structural quality dimension, which increased from approximately 0.84 to 0.87 due to normalization, missing value processing and correction of format inconsistencies. Semantic, operational and behavioral dimensions remained close to one, indicating that the proposed method improved problematic components without degrading already correct data aspects. In anomaly detection, Isolation Forest showed the best result under the experimental conditions with injected anomalies, achieving an overall classification accuracy of 0.943. In the missing value imputation task, the Random Forest method achieved the lowest Root Mean Square Error (RMSE) value of 77.704, outperforming the k-Nearest Neighbors (KNN) method with RMSE 101.203 and median imputation with RMSE 125.079. Thus, Random Forest reduced the imputation error by 23.2% compared with KNN and by 37.9% compared with the median approach.

Conclusions. The proposed method enables a transition from fragmented and declarative data quality assessment to an integrated intelligent analysis of open datasets. Its scientific novelty lies in combining multidimensional quality assessment, adaptive weighting of metrics, formalized defect detection and automated recommendation generation. The developed software implementation confirms the practical applicability of the method and demonstrates that intelligent processing can provide measurable improvements even for datasets with initially high quality. The results prove the feasibility of using machine learning algorithms, especially ensemble models, for improving open data quality. The proposed approach can be used as a basis for decision support systems, open data portals and digital governance platforms where the reliability, completeness and analytical usability of data are critical.