llama.cpp

The C and C++ inference engine that most local model runners are built on. Runs quantised models on ordinary CPUs and consumer GPUs.