Laya MLX
Native MLX runtime for Laya — 7–14ms short decisions on M3 Max
A native MLX runtime for Laya typed decision models: 7–14ms short decisions on M3 Max. No text generation, no PyTorch, no cloud API — a showcase of on-device inference optimization.
Run open LLMs on your own machine — no API bills, no data leaving your device.
Key features
- Open-weight typed decisions running natively on Apple Silicon via MLX
- 13.4 ms median end-to-end for a short English decision, 0 output tokens
- No PyTorch, Transformers runtime, or cloud API required
- Ships a real-time terminal Snake demo driven by local inference
- Pre-converted FP16 checkpoints for three upstream Laya models
Good for
- Ultra-low-latency local decisions on Macs without a GPU bill
- Game AI and real-time agents needing sub-20ms inference
Stack & tags
PythonMLXmlxapple-silicondecision-enginelocal-inferencelow-latencyopen-weights
Quick start
pip install laya-mlxFrom the project README on GitHub.