On the progress, results and problems of the evaluation of scientific institutions of the National Academy of Sciences of Ukraine in 2023
DOI:
https://doi.org/10.15407/visn2024.06.014Keywords:
research organization, methodology for evaluating the performance of research organization, evaluation criteria, peer review, evaluation in points, National Academy of Sciences of Ukraine, research funding.Abstract
Organizational aspects of evaluating the effectiveness of the scientific activities of the NASU institutions and their divisions are presented in the article. The purpose of the article consists in the analysis of the evaluation results which presented the effectiveness of the scientific activities of the institutions and their divisions, with the evaluation process taking place in 2023 according to the new Evaluation Methodology, and the results of the testing of the evaluation in points, which was carried out at the same time. The tasks of this work also include results analysis for a survey of the experts’ opinions and those of members of the Standing Commissions for the NASU scientific directions of the NASU sections on the permanent conduct of evaluation in points. In addition, the features of the evaluation in 2023 and the problems of its implementation and testing of the scoring system are determined. It is difficult to fully formalize the evaluation process, because a lot of activities and their results can’t be presented in quantitative form. While the criteria and indicators for evaluation, as well as the methods of their determination, can be the subject of debate, the need for a faster solution to two organizational problems is not disputable. The first of them is the problem of forming a pool of experts with the involvement of foreign specialists. Formation of a pool of experts should ensure greater objectivity of generalized assessments. The second problem is the creation of universal information systems (in the form of user-friendly data arrays placed on the web) for conducting not only a "spot" evaluation every five years, but also for the introduction of monitoring by the Academy's governing bodies and providing information for state certification by the Ministry of Education and Science of Ukraine. This will make it possible to reduce the time for the preparation of evaluation materials, provided that the specifics of both procedures is preserved: while for the Ministry of Education and Science of Ukraine it is necessary to obtain a quantitative assessment of the institution as a whole, and then to compare it with the results of attestation in other agencies, for the Academy it is no less important to identify weak units and develop the ways to improve efficiency and propose measures for possible reorganization.
References
Bohdanov V.L., Kubalskyi O.N. About the concept of development of the National Academy of Sciences of Ukraine for 2021—2025. Visn. Nac. Akad. Nauk Ukr. 2021. (12): 38—45.
Methodology for evaluating the effectiveness of scientific institutions of the National Academy of Sciences of Ukraine. https://files.nas.gov.ua/text/Documents/Metodika_ocinuvanya_2023.pdf
Quality assurance by Scientific Advisory Boards at Leibniz institutions. Leibniz Association. Division Evaluation Committee. http://www.leibniz-gemeinschaft.de/en/about-us/evaluation/qualityassurance-by-scientific-advisory-boards-at-leibniz-institutions/
Fiala D. Science Evaluation in the Czech Republic: the Case of Universities. Societies. 2013. 3(3): 266—279. https://doi.org/10.3390/soc3030266
Xu F., Li X. The changing role of metrics in research institute еvaluation undertaken by the Chinese Academy of Sciences (CAS). Palgrave Communications. 2016. 2: 16078. https://doi.org/10.1057/palcomms.2016.78
Ortiz Núñez R. Altmetrics: alternative metrics for scientific, technological and innovation evaluation. Academia Letters. 2021. 1658. https://doi.org/10.20935/AL1658
Mryhlod O.I., Nazarovets S.A. Scientometrics and management of scientific activities: once again about the global and Ukrainian. Visn. Nac. Akad. Nauk Ukr. 2019. (9): 81—94. https://doi.org/10.15407/visn2019.09.081
Malitskyi B.A. Problems of Comprehensive Assessment of Scientific Results. Visn. Nac. Akad. Nauk Ukr. 2023. (9): 24—36. https://doi.org/10.15407/visn2023.09.024
Rybachuk V.P. Methodological problems of evaluating the productivity of scientific activity. Nauka Naukozn. 2013. (2): 46—52 (in Ukrainian).
Zhukovych I.A., Yegorov I.Yu. The Evolution of Methodical Approaches for Evaluation the Efficiency of the Activities of Scientific Research Organizations in the Czech Republic. Statistics of Ukraine. 2020. 89-90(2-3): 117—126. https://doi.org/10.31767/su.2-3(89-90)2020.02-03.13
Zhukovych I.A., Yegorov I.Yu. Evolution of Methodological Approaches to Evaluating the Efficiency of the Activities of Research Organizations in Ukraine. Statistics of Ukraine. 2021. 93(2): 4—15. http://doi.org/10.31767/su.2(93)2021.02.01
Zhukovych І.A. Peculiarities of Evaluating the Activity of Scientific Organizations in Some Countries of the Former USSR. Statistics of Ukraine. 2022. 97(2): 41—50. https://doi.org/10.31767/su.2(97)2022.02.05
Yegorov I.Yu., Zhukovych I.A. Evaluation of the results of researchers’ activities: new trends. Nauka Naukozn. 2023. (2): 42—58. https://doi.org/10.15407/sofs2023.02.042
Yegorov I.Yu., Zhukovych I.A. Performance evaluation in research organizations: evolution of methodologies. Nauka Naukozn. 2022. (3): 36—50. https://doi.org/10.15407/sofs2022.03.036
Zhukovych I.A., Krasnonosova O.M. Organization of Evaluation of Scientific Activity in China: Methodology and Experience. Biznes Inform. 2023. (10): 53—63. https://doi.org/10.32983/2222-4459-2023-10-53-63
Hughes A., Martin B. Enhancing Impact. The Value of Public Sector R&D. Summary report. Council for Industry and Higher Education. UK Innovation Research Centre. 2012. https://www.ncub.co.uk/wp-content/uploads/2012/08/CIHE-1208ImpactReportWebVersion-1.pdf
Goodhart C. Problems of Monetary Management: The UK experience. Papers in Monetary Economics. 1975. 1(1): 1.
The future of research evaluation: a synthesis of current debates and developments. Discussion paper. 2023. https://www.interacademies.org/publication/future-research-evaluation-synthesis-current-debates-and-developments
