My research focuses on encoding expert knowledge into hierarchical probabilistic models to facilitate inference and specify what to learn from data. Models become more data efficient and more trustworthy if the result of learning is a collection of expert-interpretable components. In my work, I explore how Bayesian non-parametric models can be composed to enforce abstract constraints, yield principled reasoning under uncertainty, and enable scalable and reliable inference.