About
Open-source mixture-of-experts LLM. Efficient and powerful AI model.
Features
- Mixture-of-experts architecture with 132B total parameters and 36B active per token
- Trained on 12 trillion tokens of web data and code
- Uses fine-grained MoE with 32 experts and top-4 routing for each token
- Supports context length of 32,768 tokens
- Integrated with Databricks Model Serving, Unity Catalog, and MLflow for end-to-end ML lifecycle
Links
Categories
Reviews
3.9
Write a Review
Get new AI tools weekly
Subscribe to our newsletter and never miss a tool.
Get new AI tools weekly
Subscribe to our newsletter and never miss a tool.