Dimensionality reduction in multivariate nonparametric regression via nuclear norm penalization

Abstract

This paper reports on our study of a nonparametric reduced-rank regression method within an additive model framework. The nuclear norm of component functions is penalized to incorporate inherent low-dimensional structure into the estimation process. The proposed penalization scheme introduces sparsity to the singular values of coefficient matrices of basis functions, thereby enabling the identification of low-rank structures in the function space. A non-asymptotic oracle inequality is established to investigate the theoretical properties of the proposed estimator. Minimax upper and lower bounds are obtained to measure the complexity of the nonparametric multivariate regression problem with a low-rank structure under various high-dimensional asymptotic scenarios. The minimax analysis demonstrates the dimensionality reduction effects of the reduced-rank and additive modeling framework. The results also show that the proposed estimator is rate optimal in the minimax sense. The proposed method is implemented with the backfitting and the alternating direction method of multipliers algorithm. Simulation studies are conducted to complement the theoretical findings. To demonstrate the practicality of the proposed method, we apply it to the biochemical data and Arabidopsis thaliana data.