About
Easy-to-use LLM inference tool for local use. One-click setup for running GGUF models on CPU/GPU with API and web interface.
Features
- Built-in web UI with editing, memory, and lorebook support.
- KoboldAI-compatible API for integration with third-party front-ends.
- Multi-user support with separate sessions and settings.
- Dynamic context compression for longer conversations without extending context window.
- Command-line and GUI modes for headless or visual operation.
Links
Categories
Reviews
4.2
Write a Review
Get new AI tools weekly
Subscribe to our newsletter and never miss a tool.
Get new AI tools weekly
Subscribe to our newsletter and never miss a tool.