Optimizing inference proxy for LLMs

High-performance inference framework for large language models, focusing on efficiency, flexibility, and availability.

Just talk to your agent — it learns and EVOLVES.

OP Vault ChatGPT: Give ChatGPT long-term memory using the OP Stack (OpenAI + Pinecone Vector Database). Upload your own custom knowledge base files (PDF, txt, epub, etc) using a simple React frontend.

China Unicom's Yuanjing Wanwu Agent Platform is an enterprise-grade, multi-tenant AI agent development platform. It helps users build applications such as intelligent agents, workflows, and rag, and also supports model management. The platform features a developer-friendly license, and we welcome all developers to build upon the platform.

II-Agent: a new open-source framework to build and deploy intelligent agents

The Agent Memory Stack

Evaluation and Tracking for LLM Experiments and AI Agents

Build production-ready AI agents in both Python and Typescript.

Sparsity-aware deep learning inference runtime for CPUs

A modular Agentic RAG built with LangGraph — learn Retrieval-Augmented Generation Agents in minutes.

Implementation for MatMul-free LM.

AI Multi-Agent Framework in .NET

Open-source tools for prompt testing and experimentation, with support for both LLMs (e.g. OpenAI, LLaMA) and vector databases (e.g. Chroma, Weaviate, LanceDB).

AI agent microservice

AI Search & RAG Without Moving Your Data. Get instant answers from your company's knowledge across 100+ apps while keeping data secure. Deploy in minutes, not months.

🏆 Top-1 on 5+ benchmarks | Web UI | Supports MiroThinker, Claude, Kimi, OpenAI

MiroFlow is an agent framework that enables tool-use agent tasks, featuring a reproducible GAIA score of 82.4%.

InternLM-XComposer2.5-OmniLive: A Comprehensive Multimodal System for Long-term Streaming Video and Audio Interactions

LLM Finetuning with peft

Distributed LLM inference. Connect home devices into a powerful cluster to accelerate LLM inference. More devices means faster inference.

[KDD'2026] "VideoRAG: Chat with Your Videos"

A portable accelerated SQL query, search, and LLM-inference engine, written in Rust, for data-grounded AI apps and agents.

Fast, Accurate, Lightweight Python library to make State of the Art Embedding

🧠 AI-powered enterprise search engine 🔎

Laminar - open-source observability platform purpose-built for AI agents. YC S24.

RamaLama is an open-source developer tool that simplifies the local serving of AI models from any source and facilitates their use for inference in production, all through the familiar language of containers.

Medusa: Simple Framework for Accelerating LLM Generation with Multiple Decoding Heads

A framework-agnostic, git-native standard for defining AI agents

Build, personalize and control your own LLMs. From data pre-processing to fine-tuning, xTuring provides an easy way to personalize open-source LLMs. Join our discord community: https://discord.gg/TgHXuSJEk6

🚀 efficient approximate nearest neighbor search algorithm collections library written in Rust 🦀 .

Learn to build your Second Brain AI assistant with LLMs, agents, RAG, fine-tuning, LLMOps and AI systems techniques.

All-in-one platform for search, recommendations, RAG, and analytics offered via API

Project Page for "LISA: Reasoning Segmentation via Large Language Model"

Awesome papers about unifying LLMs and KGs

Turn any computer or edge device into a command center for your computer vision projects.

A simple, performant and scalable Jax LLM!

Awesome resources for in-context learning and prompt engineering: Mastery of the LLMs such as ChatGPT, GPT-3, and FlanT5, with up-to-date and cutting-edge updates.

Awesome things about LLM-powered agents. Papers / Repos / Blogs / ...

Comprehensive resources on Generative AI, including a detailed roadmap, projects, use cases, interview preparation, and coding preparation.

🏕️ Reproducible development environment for humans and agents

Mastering Applied AI, One Concept at a Time

⚡ Build your chatbot within minutes on your favorite device; offer SOTA compression techniques for LLMs; run LLMs efficiently on Intel Platforms⚡

NestJS Helper + AI Chatbot Development

Your Automatic Prompt Engineering Assistant for GenAI Applications

MoBA: Mixture of Block Attention for Long-Context LLMs

Distributed AI Model Training and LLM Fine-Tuning on Kubernetes

Deploy your agentic worfklows to production