AI Grid lets a cloud provider serve AI models across a fleet of OpenShift clusters, managed from one Red Hat Advanced Cluster Management (ACM) hub.
The provider runs models on the clusters that have the GPUs, and sells access to organizations through subscriptions with token limits. The organizations' users call the models through a Models-as-a-Service (MaaS) gateway, which sends each request across the grid to a cluster that serves the model.
This organization builds the provider's side of AI Grid: the console and API a provider uses to run the grid, and the GitOps that builds and operates it.
flowchart LR
admin([Provider admin]) --> console[Provider console and API]
console --> hub[(ACM hub)]
console --> git[(Git)]
git --> argo[Argo CD and ACM]
argo --> maas[MaaS clusters]
argo --> inference[Inference clusters]
user([Organization user]) --> maas
maas -->|across the grid| inference
- One hub runs the grid. Provider admins enrol clusters, deploy models, manage organizations and subscriptions, and watch the fleet from a console on the ACM hub.
- Git is the source of truth. Models and subscriptions are written to Git, and the hub delivers them to the clusters, so every change can be reviewed and audited.
- Clusters have roles. MaaS clusters receive users' requests; inference clusters run the models. One cluster can be both.
- Organizations bring their own sign-in. Each organization keeps its identity provider, joined through one shared sign-in, and its users get access through the organization's subscriptions.
AI Grid is in early development. APIs and deployment layouts may change.
- Open Cluster Management and ACM for the fleet
- OpenShift AI and Models-as-a-Service for serving and access
- Argo CD for GitOps
- Keycloak for identity
Open a pull request in the repository you want to change; each repository's README says how to build and test it.