View a PDF of the paper titled Instance Generation for Meta-Black-Box Optimization through Latent Space Reverse Engineering, by Chen Wang and three other authors.
Abstract:To relieve intensive human-expertise required to design optimization algorithms, recent Meta-Black-Box Optimization (MetaBBO) researches leverage generalization strength of meta-learning to train neural network-based algorithm design policies over a predefined training problem set, which automates the adaptability of the low-level optimizers on unseen problem instances. Currently, a common training problem set choice in existing MetaBBOs is well-known benchmark suites CoCo-BBOB. Although such choice facilitates the MetaBBO’s development, problem instances in CoCo-BBOB are more or less limited in diversity, raising the risk of overfitting of MetaBBOs, which might further results in poor generalization. In this paper, we propose an instance generation approach, termed as LSRE, which could generate diverse training problem instances for MetaBBOs to learn more generalizable policies. LSRE first trains an autoencoder which maps high-dimensional problem features into a 2-dimensional latent space. Uniform-grid sampling in this latent space leads to hidden representations of problem instances with sufficient diversity. By leveraging a genetic-programming approach to search function formulas with minimal L2-distance to these hidden representations, LSRE reverse engineers a diversified problem set, termed as Diverse-BBO. We validate the effectiveness of LSRE by training various MetaBBOs on Diverse-BBO and observe their generalization performances on either synthetic or realistic scenarios. Extensive experimental results underscore the superiority of Diverse-BBO to existing training set choices in MetaBBOs. Further ablation studies not only demonstrate the effectiveness of design choices in LSRE but also reveal interesting insights on instance diversity and MetaBBO’s generalization.
Submission History
From: Zeyuan Ma [view email]
[v1] Fri, 19 Sep 2025 09:37:48 UTC (2,582 KB)
[v2] Tue, 11 Nov 2025 06:55:10 UTC (513 KB)
—
### Understanding Meta-Black-Box Optimization (MetaBBO)
Meta-Black-Box Optimization (MetaBBO) is an innovative framework designed to automate the optimization process. Unlike traditional methods that often require expert knowledge, MetaBBO leverages the power of meta-learning. This strategic approach trains neural network-based policies that can adapt to various unseen optimization problems effectively.
### The Challenge of Limited Diversification in Benchmark Suites
More Read
A principal concern in training MetaBBOs is the reliance on established benchmark suites, such as CoCo-BBOB. While these suites serve to standardize testing and facilitate MetaBBO development, they typically present a limited range of optimization problems. This lack of diversity can lead to overfitting—where the model performs well on familiar instances but fails to generalize to new challenges.
### Introducing Latent Space Reverse Engineering (LSRE)
To tackle the issue of limited instance diversity, the authors propose an innovative instance generation approach known as Latent Space Reverse Engineering (LSRE). This novel methodology involves training an autoencoder, a type of neural network that compresses and reconstructs data, to map high-dimensional problem features into a 2-dimensional latent space. By working within this latent space, LSRE can uncover rich, diverse hidden representations of optimization problems.
### The Mechanics of LSRE
The process of LSRE begins with uniform-grid sampling in the latent space. This sampling creates numerous hidden representations, providing a foundation for generating diverse training instances. Subsequently, the authors employ a genetic programming approach to identify function formulas that closely resemble these hidden representations—measured by minimal L2-distance. This reverse engineering leads to the formulation of a diversified problem set named Diverse-BBO.
### Validating the Effectiveness of Diverse-BBO
To assess the efficacy of LSRE, the authors conduct extensive training of various MetaBBOs using the Diverse-BBO set. Their results highlight the increased generalization performances of these models, showcasing that Diverse-BBO significantly outperforms existing training sets. They offer compelling experimental data demonstrating that MetaBBOs trained on this diverse set can excel in both synthetic and real-world optimization scenarios.
### Insights Gained from Ablation Studies
Furthermore, the paper delves into ablation studies that explore the intricacies of LSRE’s design choices. These investigations not only confirm the effective methodologies used but also provide valuable insights into the relationships between instance diversity and the generalization capabilities of MetaBBOs.
### The Future of Optimization
As the demand for sophisticated optimization strategies continues to grow, the significance of innovations like LSRE cannot be overstated. By expanding the diversity of training problems, LSRE stands to improve the adaptability and efficacy of optimization algorithms across various applications. As explored in this paper, the future landscape of MetaBBOs will likely be shaped by exploratory methods that prioritize instance generation and diversity.
—
In summary, the work by Chen Wang and his colleagues emphasizes a pivotal shift in optimization research, steering towards a future where algorithms can learn from a broader spectrum of challenges, ultimately resulting in more robust and versatile optimization solutions.
Inspired by: Source
