Posterior Inference in Latent Space for Scalable Constrained Black-box Optimization (CiBO)

Abstract

Optimizing high-dimensional black-box functions under black-box constraints is a pervasive task in a wide range of scientific and engineering problems. These problems are typically harder than unconstrained problems due to hard-to-find feasible regions. In this work, we reformulate constrained black-box optimization as posterior inference, and perform this inference in the latent space of generative models. Our method iterates through two stages. First, we train flow-based models to capture the data distribution and surrogate models that predict both function values and constraint violations. Second, we cast the candidate selection problem as a posterior inference problem to effectively search for promising candidates that have high objective values while not violating the constraints. Concretely, we utilize outsourced diffusion models to amortize the sampling from the posterior distribution in the latent space of flow-based models, which can bypass the issue of mode collapse. We empirically demonstrate that our method achieves superior performance across synthetic and real-world tasks. Our code is available here.

For a visual explanation of the CiBO framework, please see the image below:

You can find the workshop paper on OpenReview, the latest preprint on arXiv, and the code on GitHub.

Paper

@inproceedings{om2025cibo,
  title={Posterior Inference in Latent Space for Scalable Constrained Black-box Optimization},
  author={Om, K. and Sim, K. and Yun, T. and Kang, H. and Park, J.},
  booktitle={NeurIPS Workshop on Structured Probabilistic Inference and Generative Modeling},
  year={2025},
  note={Oral}
}