waste

We have hosted the application waste in order to run this application in our online workstations with Wine or directly.

Quick description about waste:

WASTE is an embeddable C inference engine for running extremely large mixture-of-experts models when the weights exceed available RAM. It keeps the shared model trunk in memory and streams only the experts selected for each token from fast NVMe storage. A bounded cache reuses recently needed experts, while lookahead routing begins disk reads before the next layer requires them. Its main target is the full 2.78-trillion-parameter Kimi K3 model, including multimodal image input. The engine has no third-party runtime dependency on its CPU inference path and exposes both a CLI and C library. An optional server provides an OpenAI-compatible chat API with streaming, tools, structured output, and image support. Validation compares layers, logits, vision output, and prompt rendering against reference implementations.

Features:

• NVMe-streamed mixture-of-experts inference
• Full Kimi K3 model support
• Bounded expert caching and lookahead routing
• Dependency-free C CPU inference path
• CLI, embeddable library, and HTTP server
• Multimodal text and image inference

Programming Language: C.
Categories: