arXiv
Parallel gradient boosting for flexible estimation of conditional distributions
Rémy Chapelle, Nicolas Vayatis, Bruno Falissard, Mohammed Sedki
Problem
Estimating a full conditional distribution (not just the mean) via multi-output gradient boosting is normally too slow, since classical implementations train one base model per target at every iteration.
Method
Parallel gradient boosting (PGB) forces all outputs to share one descent direction per iteration, so a single tree sets a shared magnitude while a randomly sampled coordinate sets its direction, needing only one base learner per step regardless of the number of quantiles.
Results
On a baseball salary dataset, XGBoost took over 50x longer to fit 128 quantiles versus one, while PGB's training time stayed nearly flat with comparable accuracy; across 10 real medical datasets PGB gave best-or-second-best CRPS on 9/10 and was fastest-or-second-fastest on 9/10 versus DRF, FlexCode and LinCDE.
Limitations
No proven convergence rate or Bayes-consistency, only asymptotic empirical-risk convergence; on the largest dataset tested (n=5875) PGB was actually the slowest method, so its speed edge can reverse at larger scale.
No new updates in the industry today.