Tag
8 catalogue entries use this topic; each Project's license policy is labeled separately.
Run open language models locally with one command
Ollama reduced local model hosting from an afternoon of CUDA troubleshooting to a single pull command. It handles model download, quantisation selection, GPU offloading, and memory management, and exposes an OpenAI-compatible endpoint on localhost.
Inference engine that runs LLMs on almost any hardware
llama.cpp is the layer most local model tooling is built on. It is a C and C++ inference engine with no Python runtime, aggressive quantisation support, and hand-tuned kernels for a remarkably wide range of hardware.
Private desktop chatbot that runs open LLMs fully on-device
GPT4All is a free desktop app from Nomic that runs open language models entirely on your own computer, with no cloud calls.
Speech recognition that runs anywhere, with no runtime
whisper.cpp is a C and C++ port of the Whisper speech recognition model with no Python dependency, quantised weights, and hardware acceleration across Apple Silicon, CUDA, and plain CPU.
Open-source orchestration for AI workloads across any GPU cloud
dstack is an open-source orchestrator for AI training and inference on GPUs across clouds. Teams declare resources in YAML, and dstack provisions matching capacity.
Serverless GPUs with fast cold starts for AI workloads
Beam is a serverless GPU platform for running AI inference, sandboxes, and background jobs without managing infrastructure. Functions deploy from a Python decorator, and the platform keeps idle containers warm so repeat calls skip the cold start.
Desktop application for running language models locally
LM Studio made local models approachable for people who do not want a terminal. Browse a catalogue, download a quantisation that fits the machine, and chat, with hardware requirements explained in plain language.