Explore AI
Runtimes
The model is a file. Something has to run it. These are the programs that do.
Command line, one-command model downloads, wraps llama.cpp.
Best for: Getting started. The fastest path from nothing to a working model. macOS, Windows, LinuxA graphical app with a model browser built in.
Best for: Anyone who would rather not use a terminal. Also good on Windows. macOS, Windows, LinuxThe engine underneath most of the others. Maximum control over quantisation and offloading.
Best for: Squeezing a model onto hardware that should not fit it. Everything, including CPUsApple's own inference framework for Apple Silicon.
Best for: Mac users who want the most speed per watt. macOS (Apple Silicon)A production serving engine with high throughput.
Best for: Many requests at once, or a server other machines use. Linux, NVIDIAWhisper speech recognition optimised for local use.
Best for: Transcribing recordings on any machine. EverythingNode-based interface for image and video generation.
Best for: Image work. Steep, but nothing else comes close for control. Windows, Linux, macOSThe reference Python library.
Best for: Scripting and research, not for speed. EverythingWhich should you pick
Start with Ollama or LM Studio. They remove almost all of the setup friction, and both are wrappers around the same engine — llama.cpp — that you would end up using anyway.
Move to llama.cpp when you need control: choosing a specific quantisation, offloading layers to different devices, or making a model fit hardware it should not fit.
Use vLLM only if you are serving other people or running many requests at once. It is a server, not a desktop tool.
On Apple Silicon, try MLX. It is usually the fastest option on a Mac and uses less power.