This paper proposes the Bootstrap Estimator with Variable Selection (BEVS) procedure to estimate the determinants of the probability of default (PD) in the Brazilian banking system as a case study. In this method, we combine techniques such as Lasso regression, Loess smoothing, and bagging, showing that this integrated approach yields improved results compared to those obtained through their individual performance. Our findings indicate that BEVS not only refines the estimate of PD but also offers a comprehensive view of the impact of macroeconomic factors over the study period.