Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

View PDF Abstract:Decision trees and diffusion models are ostensibly disparate model classes, one discrete and hierarchical, the other continuous and dynamic. This work unifies the two by establishing a crisp mathematical correspondence between hierarchical decision trees and diffusion processes in appropriate limiting regimes. Our unification reveals a shared optimization principle: \emph{Global Trajectory Score Matching (GTSM)}, for which gradient boosting (in an idealized version) is asymptotically optimal. We underscore the conceptual value of our work through two key practical instantiations: \treeflow, which achieves competitive generation quality on tabular data with higher fidelity and a 2\times computational speedup, and \dsmtree, a novel distillation method that transfers hierarchical decision logic into neural networks, matching teacher performance within 2\% on many benchmarks.

Comments: 12 pages (main), 68 pages (inclusive of appendix), Accepted in the Forty-Third International Conference on Machine Learning (ICML) 2026

Subjects: Machine Learning (cs.LG); Statistical Mechanics (cond-mat.stat-mech); Artificial Intelligence (cs.AI) Cite as: arXiv:2605.00414 [cs.LG]   (or arXiv:2605.00414v2 [cs.LG] for this version)   https://doi.org/10.48550/arXiv.2605.00414 arXiv-issued DOI via DataCite Submission history From: Sai Niranjan Ramachandran [view email] [v1] Fri, 1 May 2026 05:19:54 UTC (8,277 KB) [v2] Thu, 21 May 2026 04:49:57 UTC (8,277 KB)