openlit

We have hosted the application openlit in order to run this application in our online workstations with Wine or directly.

Quick description about openlit:

OpenLIT is an OpenTelemetry-native tool designed to help developers gain insights into the performance of their LLM applications in production. It automatically collects LLM input and output metadata and monitors GPU performance for self-hosted LLMs. OpenLIT makes integrating observability into GenAI projects effortless with just a single line of code. Whether you're working with popular LLM providers such as OpenAI and HuggingFace, or leveraging vector databases like ChromaDB, OpenLIT ensures your applications are monitored seamlessly, providing critical insights including GPU performance stats for self-hosted LLMs to improve performance and reliability. This project proudly follows the Semantic Conventions of the OpenTelemetry community, consistently updating to align with the latest standards in observability.

Features:

• Advanced Monitoring of LLM and VectorDB Performance
• Cost Tracking for Custom and Fine-Tuned Models
• Cost Tracking for Custom and Fine-Tuned Models
• Documentation available
• OpenTelemetry-native & vendor-neutral SDKs
• Examples available
• OpenLIT enables you to tailor cost tracking for specific models
• OpenLIT is built with native support for OpenTelemetry

Programming Language: Python.
Categories: