Colmena: AI-Steering for HPC
Colmena is a Python library for building applications that combine AI and simulation workflows on HPC. Its core feature is a communication library that simplifies tools for intelligently steering large ensemble simulations.
Colmena open-source and available on GitHub: https://github.com/exalearn/colmena
What does Colmena do?
The core concept of Colmena is a “thinking” application. The Thinking application is responsible for intelligently responding to new data, such as by updating a machine learning model or selecting a new simulation with Bayesian optimization.
Colmena provides a few main components to enable building thinking applications:
An extensible base class for building thinking applications with a dataflow-like programming model
A “Task Server” that provides a simplified interface to HPC-ready workflow systems
A high-performance queuing system communicating to tasks servers from thinking applications
The demo applications illustrate how to implement different thinking applications that solve optimization problems.
Why the name “Colmena?”
Colmena is Spanish for “beehive.” Colmena applications, like their namesake, feature independent agents that perform a variety of tasks over their lifetimes without complicated communication between each other.
Citing Colmena
If you use Colmena in academic research cite our 2021 paper: link bibtex