How It Fits Together
Core Components
mlOS Core
v6.2.0-alphaThe kernel-level ML runtime engine. Manages model lifecycle, plugin loading, resource allocation, and provides multi-protocol APIs for inference.
- Multi-protocol API (HTTP, gRPC, IPC)
- Plugin-based architecture
- NUMA-aware memory management
- Predictive prefetching
- Built-in ONNX & GGUF support
Axon CLI
v3.2.0Model management CLI. Downloads models from Hugging Face, handles format detection, ONNX conversion, and registration with mlOS Core.
- Format-agnostic installation (--format flag)
- Case-insensitive model lookup
- Multi-encoder ONNX (CLIP, T5)
- YOLO model support
- Universal ONNX conversion
System Test
ActiveEnd-to-end testing framework. Validates the entire pipeline from model download through inference with real models on real hardware.
- 42+ model test suite
- Hardware-aware benchmarking
- Historical performance tracking
- Automated GitHub Actions
- Public test reports
Core Releases
DistributionPre-built binaries for various platforms. Includes mlOS Core daemon, development headers, and system libraries.
- Linux x64 binaries
- macOS ARM64 (Apple Silicon)
- Docker images
- Development packages
- Automatic updates
All Repositories
| Repository | Description | Status | Version |
|---|---|---|---|
| mlOS-foundation/core | Kernel-level ML runtime engine | Active | v6.2.0-alpha |
| mlOS-foundation/axon | Model management CLI | Active | v3.2.0 |
| mlOS-foundation/system-test | E2E testing & benchmarking | Active | - |
| mlOS-foundation/core-releases | Pre-built binaries & Docker images | Active | - |
| mlOS-foundation/mlosfoundation.org | This website | Active | - |
| mlOS-foundation/sdk-python | Python SDK for mlOS Core | Planned | - |
| mlOS-foundation/sdk-js | JavaScript/TypeScript SDK | Planned | - |
Typical Integration
1. Install Axon
curl -sSL axon.mlosfoundation.org | sh
2. Download Model
axon install hf/bert-base
3. Start Core
mlos_core --config prod.yaml
4. Run Inference
POST /models/bert/inference