The Sequential Parameter Optimization Toolbox (SPOT) provides a set of tools for model-based optimization and tuning of algorithms. It includes surrogate models, optimizers, and design of experiment approaches.
The main interface to the open-source toolbox is the function spot(), which uses sequentially updated surrogate models for the purpose of efficient optimization. SPOT reduces the number of objective function evaluations, when a single evaluation requires a significant amount of resources.
For more than 15 years, SPOT is used as an efficient and effective hyperparameter optimization (HPO) tool. It can be downloaded from CRAN: https://cran.r-project.org/package=SPOT
The paper "In a Nutshell -- The Sequential Parameter Optimization Toolbox" (Prof. Dr. Thomas Bartz-Beielstein, Dr. Martin Zaefferer, Frederik Rehbach) can be used as a starting point: https://arxiv.org/abs/1712.04076
More information about SPOT can be found here: https://www.spotseven.de