📚A curated list of Awesome LLM/VLM Inference Papers with Codes: Flash-Attention, Paged-Attention, WINT8/4, Parallelism, etc.🎉

A High-Performance LLM Inference Engine with vLLM-Style Continuous Batching

Implementation of PagedAttention from vLLM paper - a breakthrough attention algorithm that treats KV cache like virtual memory. Eliminates memory fragmentation, increases batch sizes, and dramatically improves LLM serving throughput.

High-performance On-Device MoA (Mixture of Agents) Engine in C++. Optimized for CPU inference with RadixCache & PagedAttention. (Tiny-MoA Native)

A production-grade, native Rust speculative inference engine for Apple Silicon with Metal GPU acceleration and paged attention.

vLLM - High-throughput, memory-efficient LLM inference engine with PagedAttention, continuous batching, CUDA/HIP optimization, quantization (GPTQ/AWQ/INT4/INT8/FP8), tensor/pipeline parallelism, OpenAI-compatible API, multi-GPU/TPU/Neuron support, prefix caching, and multi-LoRA capabilities

A from scratch LLM inference engine build in PyTorch with custom GPT2 transformers, kv cache, paged kv cache, continuous batching and A100 benchmarks

High-Performance LLM Inference Engine with PagedAttention & Continuous Batching in Rust

🤖 Enhance task management with Tiny MoA, a GPU-free multi-agent system that plans, reasons, and collaborates efficiently in real time.

Discrete-event simulator for LLM inference serving — PagedAttention memory management and continuous batching

LangChain integration for Parallel Context-of-Experts Decoding (PCED)

AI Infra 手写算子实现Qwen2推理，支持Paged Attention

Intra-2MiB CUDA leaf packing (cuMem). GATE12 iron: workload_id=t1_leaf_physics_v1 + legacy t1-eval-20260522. 42% VRAM liberation @70% budget. Evaluation-Only.

A high-performance visual exploration platform for understanding LLM Inference, vLLM optimization, RAG architectures, and GPU warm startup concepts

🌱 A tiny, readable LLM serving engine with vLLM/SGLang-style features.

A compact benchmark lab for measuring TTFT, throughput, and KV-memory gains from GQA, KV cache, and paged KV management.