LiteLLM — a leading open-source project in the AI/LLM ecosystem.

Learn how to design, develop, deploy and iterate on production-grade ML applications.

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

Composio powers 1000+ toolkits, tool search, context management, authentication, and a sandboxed workbench to help you build AI agents that turn intent into action.

Label Studio is a multi-type data labeling and annotation tool with standardized output format

☁️ Build multimodal AI applications with cloud-native stack

A curated list of awesome open source libraries to deploy, monitor, version and scale your machine learning

Turns Data and AI algorithms into production-ready web applications in no time.

Machine Learning Engineering Open Book

<⚡️> SuperAGI - A dev-first open source autonomous AI agent framework. Enabling developers to build, manage & run useful autonomous agents quickly and reliably.

The absolute trainer to light up AI agents.

Workflow Engine for Kubernetes

Python SDK for Agent AI Observability, Monitoring and Evaluation Framework. Includes features like agent, llm and tools tracing, debugging multi-agentic system, self-hosted dashboard and advanced analytics with timeline and execution graph view

An orchestration platform for the development, production, and observation of data assets.

Free MLOps course from DataTalks.Club

Supercharge Your LLM Application Evaluations 🚀

An open source AutoML toolkit for automate machine learning lifecycle, including feature engineering, neural architecture search, model compression and hyper-parameter tuning.

A curated list of references for MLOps

A blazing fast AI Gateway with integrated guardrails. Route to 200+ LLMs, 50+ AI Guardrails with 1 fast & friendly API.

TensorZero is an open-source stack for industrial-grade LLM applications. It unifies an LLM gateway, observability, optimization, evaluation, and experimentation.

Always know what to expect from your data.

Always know what to expect from your data.

BISHENG is an open LLM devops platform for next generation Enterprise AI applications. Powerful and comprehensive features include: GenAI workflow, RAG, Agent, Unified model management, Evaluation, SFT, Dataset Management, Enterprise-level System Management, Observability and more.

The AI developer platform. Use Weights & Biases to train and fine-tune models, and manage models from experimentation to production.

Example 📓 Jupyter notebooks that demonstrate how to build, train, and deploy machine learning models using 🧠 Amazon SageMaker.

Kedro is a toolbox for production-ready data science. It uses software engineering best practices to help you create data engineering and data science pipelines that are reproducible, maintainable, and modular.

A booklet on machine learning systems design with exercises. NOT the repo for the book "Designing Machine Learning Systems", which is `dmls-book`

Build, Manage and Deploy AI/ML Systems

Open-source, low-code AutoML platform for Python. PyCaret 4.0: sklearn-native engine + React control plane.

Evidently is ​​an open-source ML and LLM observability framework. Evaluate, test, and monitor any AI-powered system or data pipeline. From tabular data to Gen AI. 100+ metrics.

Open-source observability for your GenAI or LLM application, based on OpenTelemetry

The Open Source Feature Store for AI/ML

ClearML - Auto-Magical CI/CD to streamline your AI workload. Experiment Management, Data Management, Pipeline, Orchestration, Scheduling & Serving in one MLOps/LLMOps solution

Ship AI Agents to Google Cloud in minutes, not months. Production-ready templates with built-in CI/CD, evaluation, and observability.

Fastest enterprise AI gateway (50x faster than LiteLLM) with adaptive load balancer, cluster mode, guardrails, 1000+ models support & <100 µs overhead at 5k RPS.

🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 🍓

An awesome & curated list of best LLMOps tools for developers

Next-generation AI Agent Optimization Platform: Cozeloop addresses challenges in AI agent development by providing full-lifecycle management capabilities from development, debugging, and evaluation to monitoring.

🐢 Open-Source Evaluation & Testing library for LLM Agents

ZenML 🙏: One AI Platform from Pipelines to Agents. https://zenml.io.

SkillFoundry — a leading open-source project in the AI/LLM ecosystem.

StackQuadrant — a leading open-source project in the AI/LLM ecosystem.