Variable Selection for Gaussian Process Regression

Variable selection is crucial in statistical modeling, as it helps identify the most relevant variables for constructing an effective and interpretable model. In this study, the main objective is to develop a novel framework that uses Gaussian Process Regression (GPR) for variable selection. Specifically, this framework sets spike-and-slab priors on the inverse length-scale parameters within the Automatic Relevance Determination (ARD) kernel, which is well-suited for selecting significant variables.

The significance of each variable can then be determined by examining its posterior inclusion probabilities, with higher values indicating more relevant variables. To sample the parameters from the posterior distribution efficiently, we propose a novel Metropolis independence sampler combined with Laplace approximation. This method has not been previously applied in GPR contexts and is expected to improve both sampling efficiency and convergence when dealing with complex posterior landscapes.

Additionally, we could extend the model by treating the covariates as latent variables. This leads to the use of a GP latent model, which offers flexibility in capturing complex, hidden structures within the data, especially when some covariates may not be directly observable. By incorporating a latent variable approach, we aim to enhance the model's capacity to account for unobserved factors that may drive underlying relationships between the variables and the response.