Artificial intelligence, neural networks and spectral properties of random matrices
According to the materials of report at the meeting of the Presidium of the NAS of Ukraine, May 28, 2025
DOI:
https://doi.org/10.15407/visn2025.08.069Abstract
The report is devoted to recent results in the field of Artificial Intelligence and Neural Networks (NNs), both based on the spectral properties of random matrices. Nowadays, the development of Artificial Intelligence is skyrocketing, but we still lack an understanding of how it works. One of the modern approaches to solving this problem is the application of advanced mathematical theories, in particular, Random Matrix Theory. We present our results on the spectral properties of random matrices, obtained at the Department of Mathematical Physics at B. Verkin Institute for Low Temperature Physics and Engineering of the National Academy of Sciences of Ukraine. These matrices are closely connected with NNs: weight matrices of trained NNs could be represented in the form W = R + S, where R is random and S is deterministic. The spectrum of such matrices plays a key role in the rigorous underpinning of the novel pruning technique based on Random Matrix Theory.
Cite this article:
Afanasiev I.V. Artificial intelligence, neural networks and spectral properties of random matrices (according to the materials of report at the meeting of the Presidium of the NAS of Ukraine, May 28, 2025). Visn. Nac. Akad. Nauk Ukr. 2025. (8): 69—73. https://doi.org/10.15407/visn2025.08.069
References
Pennington J., Schoenholz S., Ganguli S. The emergence of spectral universality in deep networks. Proc. of the 21st Int. Conf. on Artificial Intelligence and Statistics (AISTATS). 2018. P. 1924. arXiv:1802.09979. https://doi.org/10.48550/arXiv.1802.09979
Pastur L. On random matrices arising in deep neural networks: Gaussian case. Pure Appl. Funct. Anal. 2020. 5(6): 1395—1424.
Pastur L., Slavin V. On random matrices arising in deep neural networks: general I.I.D. case. Random Matrices Theory Appl. 2023. 12(1): 2250046. https://doi.org/10.1142/S2010326322500460
Berlyand L., Sandier E., Shmalo Y., Zhang L. Enhancing Accuracy in Deep Learning Using Random Matrix Theory. Journal of Machine Learning Research. 2024. 3: 347—412. https://doi.org/10.4208/jml.231220
Price I., Tanner J. Dense for the Price of Sparse: Improved Performance of Sparsely Initialized Networks via a Subspace Offset. Proc. of the 38th Int. Conf. on Machine Learning. 2021. P. 8620—8629. https://doi.org/10.48550/arXiv.2102.07655
Saada T.N., Tanner J. On the Initialization of Wide Low-Rank Feedforward Neural Networks. arXiv:2301.13710. 2023. https://doi.org/10.48550/arXiv.2301.13710
Couillet R., Liao Z. Random Matrix Methods for Machine Learning. Cambridge University Press, 2022. https://doi.org/10.1017/9781009128490
Martin C.H., Mahoney M.W. Implicit self-regularization in deep neural networks: evidence from random matrix theory and implications for learning. Journal of Machine Learning Research. 2021. 22(165): 1—73. http://jmlr.org/papers/v22/20-410.html
Martin C.H., Mahoney M.W. Heavy-Tailed Universality Predicts Trends in Test Accuracies for Very Large Pre-Trained Deep Neural Networks. Proc. of the SIAM Int. Conf. on Data Mining. 2020. P. 505—513. https://doi.org/10.1137/1.9781611976236.57
Afanasiev I., Berlyand L., Kiyashko M. Asymptotic behavior of eigenvalues of large rank perturbations of large random matrices. arXiv:2507.12182. 2025. https://doi.org/10.48550/arXiv.2507.12182
Afanasiev I., Shcherbina T. Characteristic Polynomials of Sparse Non-Hermitian Random Matrices. J. Stat. Phys. 2025. 192: 12. https://doi.org/10.1007/s10955-024-03379-5
Afanasiev I. On the Correlation Functions of the Characteristic Polynomials of Non-Hermitian Random Matrices with Independent Entries. J. Stat. Phys. 2019. 176(6): 1561—1582. https://doi.org/10.1007/s10955-019-02353-w
Afanasiev I. On the Correlation Functions of the Characteristic Polynomials of the Sparse Hermitian Random Matrices. J. Stat. Phys. 2016. 163(2): 324—356. https://doi.org/10.1007/s10955-016-1486-z
Afanasiev I. On the Correlation Functions of the Characteristic Polynomials of Random Matrices with Independent Entries: Interpolation Between Complex and Real Cases. J. Math. Phys. Anal. Geom. 2022. 18(2): 159—181. https://doi.org/10.15407/mag18.02.159
Afanasiev I. On the Correlation Functions of the Characteristic Polynomials of Real Random Matrices with Independent Entries. J. Math. Phys. Anal. Geom. 2020. 16(2): 91—118. https://doi.org/10.15407/mag16.02.091
