LocalAI

About

Open-source, self-hosted OpenAI API alternative. Run LLMs, image generation, and audio models locally on consumer hardware.

• Unified REST API compatible with OpenAI endpoints (chat completions, embeddings, images, audio)
• Built-in model gallery for downloading and managing models from Hugging Face directly
• Support for multiple backends: llama.cpp, transformers, diffusers, and whisper.cpp
• Hardware acceleration via CUDA, OpenCL, Metal, and Vulkan for GPU inference
• Template system for customizing prompts with pre-defined structures and variables