vLLM

@vllm_project51k followers

A high-throughput and memory-efficient inference and serving engine for LLMs. Join to discuss together with the community!

Open on X

Top outlier

212×

Best organic post

Peak views

1.5M

Single post

Bookmarks

14k

Across 99 organic posts

Last scan

Oct 17

Community cache

Posts 99 posts

#	Date	Post	↓ Outlier	Views	Likes	Reposts	Bookmarks
1	Oct 20, '25	🚀 DeepSeek-OCR — the new frontier of OCR from @deepseek_ai , exploring optical context compression for LLMs, is running blazingly fast on vLLM ⚡ (~2500 tokens/s on A100-40G) — pow	212×	1.5M	2.5k	355	1.3k
2	Sep 18, '25	Congrats to @deepseek_ai ! DeepSeek-R1 was published in Nature yesterday as the cover article, and vLLM is proud to have supported its RL training and inference🥰	29.5×	213k	1.2k	129	361
3	Apr 14, '25	🙏 @deepseek_ai's highly performant inference engine is built on top of vLLM. Now they are open-sourcing the engine the right way: instead of a separate repo, they are bringing cha	28.2×	204k	2.3k	327	739
4	Oct 22, '25	it’s tokenization again! 🤯 did you know tokenize(detokenize(token_ids)) ≠ token_ids? RL researchers from Agent Lightning coined the term Retokenization Drift — a subtle mismatch b	23.7×	171k	675	72	420
5	Nov 3, '25	Wow excited to see PewDiePie using vLLM to serve language models locally 😃 vLLM brings easy, fast, and cheap LLM serving for everyone 🥰	22.8×	164k	1.6k	42	313
6	Oct 16, '25	Announcing the completely reimagined vLLM TPU! In collaboration with @Google, we've launched a new high-performance TPU backend unifying @PyTorch and JAX under a single lowering pa	21.8×	157k	948	113	279
7	Oct 27, '25	🎉 Congrats to the @minimax_ai team for releasing MiniMax-M2 model! Built for advanced coding and agentic tasks, MiniMax-M2 is now available with Day-0 support on vLLM, bringing fa	20.6×	149k	410	40	64
8	Feb 21, '25	We're excited to receive our first #NVIDIADGX B200 system which we'll use for vLLM research and development! Thank you @nvidia!	16.5×	119k	946	38	85
9	Sep 29, '25	How does @deepseek_ai Sparse Attention (DSA) work? It has 2 components: the Lightning Indexer and Sparse Multi-Latent Attention (MLA). The indexer keeps a small key cache of 128 pe	14.3×	103k	698	106	430
10	Feb 6, '25	v0.7.2 is released! Featuring 🖼️ @Alibaba_Qwen Qwen2.5-VL, 🤗 @huggingface Transformers backend, and several @deepseek_ai performance enhancements.	14.2×	103k	371	52	103
11	Sep 5, '24	A month ago, we announced our performance roadmap. Today, we are happy to share that the latest release achieves 🚀2.7x higher throughput and is 5x faster for output latency on Lla	13.3×	96k	375	68	138
12	Jan 27, '25	🚀 With the v0.7.0 release today, we are excited to announce the alpha release of vLLM V1: A major architectural upgrade with 1.7x speedup! Clean code, optimized execution loop, ze	13.2×	95k	636	89	191
13	Nov 12, '25	🚀 No More Train–Inference Mismatch! We demonstrate bitwise consistent on-policy RL with TorchTitan (training) + vLLM (inference) — the first open-source run where training and inf	12.8×	92k	512	63	230
14	Feb 1, '25	We landed the 1st batch of enhancements to the @deepseek_ai models, starting MLA and cutlass fp8 kernels. Compared to v0.7.0, we offer ~3x the generation throughput, ~10x the memor	12.5×	90k	718	101	316
15	Oct 5, '25	🚀 The RL community keeps pushing boundaries — from better on-policy data and partial rollouts to in-flight weight updates that mix KV caches across models during inference. Contin	12.4×	89k	460	63	295
16	Apr 19, '25	After feedback about our v0.8.4 benchmark for @deepseek_ai R1, we rerun it with suggested changes: vLLM no EP, SGLang updated v0.4.5 -> post1 and EP -> DP, TensorRT-LLM uses	11.1×	80k	320	36	101
17	Apr 17, '25	vLLM🤝🤗! You can now deploy any @huggingface language model with vLLM's speed. This integration makes it possible for one consistent implementation of the model in HF for both tra	10.1×	73k	834	120	280
18	Aug 17, '25	🚀 Amazing community project! vLLM CLI — a command-line tool for serving LLMs with vLLM: ✅ Interactive menu-driven UI & scripting-friendly CLI ✅ Local + HuggingFace Hub model manag	9.9×	71k	1.1k	176	578
19	Oct 11, '25	🚀 vLLM x MinerU: Document Parsing at Lightning Speed! We’re excited to see MinerU fully powered by vLLM — bringing ultra-fast, accurate, and efficient document understanding to ev	9.8×	71k	582	84	518
20	Sep 28, '25	🚀 New in vLLM: dots.ocr 🔥 A powerful multilingual OCR model from @xiaohongshu hi lab is now officially supported in vLLM! 📝 Single end-to-end parser for text, tables (HTML), for	9.6×	70k	672	86	496
21	Aug 6, '25	👀 we care a lot about correctness, ran many evals and stared at many tensors to compare them. numerics of vLLM on hopper should be solid and verified! if you run into any correctn	9.3×	67k	318	27	112
22	Sep 11, '25	Welcome Qwen3-Next! You can run it efficiently on vLLM with accelerated kernels and native memory management for hybrid models.	8.5×	62k	299	39	49
23	Apr 29, '25	⬆️ pip install -U vLLM vllm serve Qwen/Qwen3-235B-A22B-FP8 --enable-reasoning --reasoning-parser deepseek_r1 --tensor-parallel-size 4 vLLM introduce Day 0 support for @Alibaba_Qwen	8.3×	60k	392	61	95
24	Nov 13, '25	uv helps vLLM out of dependency hell! The simple vLLM nightly index installation has been broken by xformers for a while and people have to use a fairly complicated command to work	7.9×	57k	557	39	160
25	Oct 21, '25	kvcached works directly with vLLM and you can use it to serve multiple models on the same GPU. They will share unused KV cache blocks. Check it out!	7.6×	55k	543	66	272
26	Apr 10, '25	spotted @vllm_project at @googlecloud next keynote today!	7.1×	51k	566	49	85
27	Jul 8, '25	vLLM runs on free-threaded Python! A group of engineers from @Meta’s Python runtime language team has shown that it’s possible to run vLLM on the nogil distribution of Python. We’r	6.9×	50k	592	76	236
28	Feb 27, '25	Just landed FlashMLA in vLLM and it is already boosting output throughput 2-16%! We expect more improvements in the coming days as we continue to optimize the code path.	6.8×	49k	461	58	96
29	Apr 24, '25	OpenRLHF is a pioneering framework to use vLLM for RLHF, driving many design and implementation of vLLM's features for RLHF, making vLLM a popular choice for many RLHF frameworks.	6.7×	48k	259	40	131
30	Aug 23, '25	Wow glad to see vLLM powers @jiawzhao 's DeepConf work, impressive results on AIME 2025! Do you think this sampling control makes sense? Have a try and leave a comment in that PR t	6.6×	48k	309	35	110
31	Sep 9, '25	The amazing blogpost from @gordic_aleksa is alive at vLLM's blogpost (after more proofreading and clarifications)! Looking forward to future series of tech deep dive blogposts😍	6.5×	47k	617	73	392
32	Mar 1, '25	Amazing system! It is now the north star for LLM inference 🌟. We will get there, quickly.	6.4×	46k	464	33	81
33	Aug 5, '25	Thank you @OpenAI for open-sourcing these great models! 🙌 We’re proud to be the official launch partner for gpt-oss (20B & 120B) – now supported in vLLM 🎉 ⚡ MXFP4 quant = fast &	6.1×	44k	415	60	92
34	Sep 24, '25	Pro Tip💡Fast and simple way to deploy DeepSeek-V3.1-Terminus with vLLM ⚡️ Run it with: vllm serve deepseek-ai/DeepSeek-V3.1-Terminus -tp 8 -dcp 8 (as simple as appending -dcp 8 af	6.0×	43k	411	52	153
35	Apr 17, '25	perf update: we are continuing to see benefits with vLLM V1 engine’s highly performant design. on 8xH200, vLLM leads in throughput for @deepseek_ai V3/R1 models. we expect further	5.8×	42k	415	40	89
36	Aug 8, '24	🙏 Thank you @nvidia for sponsoring vLLM development. The DGX H200 machine is marvelous! We plan to use the machine for benchmarking and performance enhancement 🏎️.	5.8×	42k	428	23	33
37	Oct 30, '25	🎉 Congrats to @Kimi_Moonshot! vLLM Day-0 model expands! Now supporting Kimi Linear — hybrid linear attention with Kimi Delta Attention(KDA): - RULER 128k context: 84.3 perf + 3.98	5.5×	40k	249	33	55
38	Feb 25, '25	👀 We just merged initial EP support in vLLM and will be integrating these collectives ASAP! 🏎️	5.5×	40k	324	23	55
39	Feb 20, '25	vLLM v0.7.3 now supports @deepseek_ai's Multi-Token Prediction module! It delivers up to 69% speedup boost. You can turn it on with `--num-speculative-tokens=1` and an optional `--	5.5×	39k	514	76	169
40	Oct 22, '25	🚀 Excited to share our work on batch-invariant inference in vLLM! Now you can get identical results regardless of batch size with just one flag: VLLM_BATCH_INVARIANT=1 No more sub	5.4×	39k	281	43	95
41	Oct 13, '25	🚀 vLLM just hit 60K GitHub stars! 🎉 From a small research idea to powering LLM inference everywhere — across NVIDIA, AMD, Intel, Apple, TPUs, and more — vLLM now supports almost	5.4×	39k	475	43	113
42	Mar 18, '25	Spotted @vllm_project during Jensen's Keynote @nvidia #GTC	5.3×	39k	216	13	21
43	Jul 6, '25	We genuinely want to solve this problem. As many (@Rxday000 @samsja19 @danielhanchen @_EldarKurtic and more!) chimed in, the reason includes attention kernels, matmul reduction ord	5.1×	37k	267	17	106
44	Sep 12, '24	🖼️ pip install -U vLLM vllm serve mistralai/Pixtral-12B-2409 --tokenizer_mode mistral --limit_mm_per_prompt 'image=4' --max_num_batched_tokens 16384	4.9×	36k	314	33	109
45	Dec 27, '24	⬆️pip install -U vLLM You can now run DeepSeek-V3 on latest vLLM many different ways: 💰 Tensor parallelism on 8xH200 or MI300x, or TP16 on IB connected nodes: `--tensor-parallel-s	4.9×	35k	294	47	116
46	Jul 11, '25	@Kimi_Moonshot just released a trillion-parameter model with great agentic capability, and it is already supported in vLLM! Have a try with a simple command, and check the doc for	4.8×	34k	200	22	51
47	Oct 28, '25	vLLM Sleep Mode 😴→ ⚡Zero-reload model switching for multi-model serving. Benchmarks: 18–200× faster switches and 61–88% faster first inference vs cold starts. Explanation Blog by	4.7×	34k	490	68	243
48	Aug 13, '25	Have you ever felt you are developing cuda kernels and your tests often run into illegal memory access (IMA for short) and you have no idea how to debug? We have collaborated with	4.7×	34k	169	23	101
49	Jul 22, '25	✅ Try out @Alibaba_Qwen 3 Coder on vLLM nightly with "qwen3_coder" tool call parser! Additionally, vLLM offers expert parallelism so you can run this model in flexible configuratio	4.7×	34k	232	19	50
50	Nov 6, '25	🚀 Day 0 support: Kimi K2 Thinking now running on vLLM! In partnership with @Kimi_Moonshot, we're proud to deliver official support for the state-of-the-art open thinking model wit	4.7×	34k	413	50	87
51	Jul 26, '25	This amazing Attention-FFN disaggregation implementation from @StepFun_ai , achieves decoding throughput of up to 4,039 tokens per second per GPU under 50ms TPOT SLA, for their 321	4.6×	33k	349	51	182
52	Sep 1, '25	Amazing blogpost from @gordic_aleksa explaining internals of vLLM😍	4.5×	33k	385	48	216
53	Nov 4, '25	Amazing work by @RidgerZhu and the ByteDance Seed team — Scaling Latent Reasoning via Looped LMs introduces looped reasoning as a new scaling dimension. 🔥 The Ouro model is now ru	4.5×	32k	227	30	100
54	Oct 22, '24	Speculative decoding is one of the best tool in the vLLM's suite of inference optimization tool box, accelerating the inference without accuracy loss. Checkout our blog post for mo	4.3×	31k	233	49	130
55	Sep 10, '25	Wow, thanks to @charles_irl , you can understand internals of vLLM with a live notebook from @modal 🥰	4.2×	30k	327	31	184
56	Jul 31, '25	🔥 vLLM @ PyTorch Conference 2025 🔥 We’re excited to share that 5 talks at this year’s PyTorch Conference will feature vLLM! Topics include: • Easy & Fast LLM Serving • Open-Sourc	4.0×	29k	184	21	47
57	Nov 5, '25	😉	3.8×	27k	230	5	16
58	Nov 11, '25	vLLM now optimized for @Intel Arc Pro B-Series GPUs 🚀 Delivering accessible, high-performance LLM serving with exceptional price-to-performance. Key highlights: • 80%+ MoE hardwar	3.7×	26k	190	23	48
59	Sep 9, '24	We are excited to see @vllm_project as an option for local apps in the @huggingface hub! It comes with easy snippets to quickly test out the model.	3.5×	25k	159	27	24
60	Apr 6, '25	vLLM v0.8.3 now supports @AIatMeta's latest Llama 4 Scout and Maverick. We see these open source models as a major step forward in efficiency with long context feature, native mult	3.4×	25k	284	35	86
61	Oct 27, '25	🚀 New in vLLM Semantic Router: • Parallel LoRA execution • Lock-free concurrency with OnceLock • Flash Attention 2 for 3–4× faster inference • Rust × Go FFI for cloud-native ML Sm	3.4×	24k	414	69	151
62	Dec 9, '24	Open-source innovation is part of the vLLM’s DNA, and we love the PyTorch ecosystem! Together, let's push the boundaries of AI innovation and make it accessible to all💪	3.3×	24k	200	23	33
63	Sep 10, '25	⚡️ Efficient weight updates for RL at trillion-parameter scale 💡 Best practice from Kimi @Kimi_Moonshot vLLM is proud to collaborate with checkpoint-engine: • Broadcast weight syn	3.3×	24k	321	33	118
64	Nov 4, '25	Wow Quantization-enhanced Reinforcement Learning using vLLM! Great job by @yukangchen_ 😃	3.2×	23k	231	30	87
65	Aug 21, '25	🚀 Exciting news: DeepSeek-V3.1 from @deepseek_ai now runs on vLLM! 🧠 Seamlessly toggle Think / Non-Think mode per request ⚡ Powered by vLLM’s efficient serving — scale to multi-G	3.2×	23k	309	33	78
66	Feb 26, '25	👀 @vllm_project will be testing and integrating these GEMM kernels ASAP as well.	3.1×	23k	249	15	23
67	Nov 17, '25	Check out a great framework built upon vLLM, to serve Any-to-Any multimodal models!	3.1×	22k	198	19	81
68	Jul 22, '25	The @huggingface Transformers ↔️ @vllm_project integration just leveled up: Vision-Language Models are now supported out of the box! If the model is integrated into Transformers, y	3.1×	22k	263	43	59
69	Jun 10, '25	👀 Look what just arrived at @BerkeleySky! 🌟 A shiny MI355X system. Huge thanks to @AMD for supporting open source and we are looking forward to getting it set up in the next few	2.8×	20k	270	21	12
70	Sep 25, '24	⬆️ pip install -U vLLM 🦙 vllm serve meta-llama/Llama-3.2-11B-Vision-Instruct --enforce-eager --max-num-seqs 16 We are excited to announce day 1 support for @AIatMeta's Llama 3.2 v	2.8×	20k	191	41	93
71	Oct 24, '25	vLLM 🤝 @nvidia = open, scalable, agentic AI you can run anywhere. 🧵 Strengthening our partnership with @nvidia: vLLM serves the NVIDIA Nemotron family. This new blog from NVIDIA	2.6×	19k	160	20	47
72	Jun 16, '25	Congrats on the launch! vLLM is proud to support this great model on day 1, looking forward to the following releases!	2.6×	19k	189	12	39
73	Sep 5, '25	vLLM is proud to support the great Kimi update from @Kimi_Moonshot , better tool-calling, longer context, and more! Check the deployment guide at 🔥	2.6×	18k	202	23	31
74	Sep 10, '25	Deep dive into optimizing weight transfer step by step and improving it 60x!	2.5×	18k	237	22	130
75	Jun 19, '25	vLLM has just reached 50K github stars! Huge thanks to the community!🚀 Together let's bring easy, fast, and cheap LLM serving for everyone✌🏻	2.5×	18k	263	21	21
76	Oct 24, '25	Wow OCR models are taking off in vLLM 😍 Small but powerful 💪 Enjoy this fast OCR model from @staghado ✌️	2.4×	18k	217	28	84
77	Sep 16, '25	v0.10.2 marks the first release with official aarch64 support for vLLM! You can now install vLLM directly onto @nvidia 's GB200. Along with the PyPI release, our docker image is al	2.4×	18k	263	21	44
78	Oct 2, '25	Keeping BERT alive in vLLM, via transformers!!	2.4×	18k	179	18	41
79	May 28, '25	⬆️ uv pip install -U vLLM The latest release features 649 commits from 215 contributors. vLLM is now ready for @nvidia Blackwell with the latest @PyTorch 2.7 upgrade. Huge thanks t	2.4×	17k	287	29	71
80	Jan 10, '25	Thanks for the community support, vLLM now supports MacOS natively🤩	2.4×	17k	175	15	29
81	Jun 13, '25	Thank you @AMD @LisaSu @AnushElangovan for Advancing AI together with @vllm_project! We look forward to the continued partnership and pushing the boundary of inference.	2.4×	17k	212	18	15
82	Oct 24, '25	🎉@Kimi_Moonshot co-founder @ppwwyyxx talking about Moonshot’s Decode Context Parallel open source contribution to @vllm_project at @PyTorch conf.	2.4×	17k	157	17	18
83	Oct 28, '25	✨ At vLLM, we strive for correctness, reliability, and open collaboration — every detail matters. Together with @Kimi_Moonshot , we verified Kimi K2’s tool-calling accuracy on vLLM	2.3×	17k	140	13	32
84	Aug 19, '25	🚀 GLM-4.5 meets vLLM @Zai_org 's latest GLM-4.5 & GLM-4.5V models bring hybrid reasoning, coding & intelligent agent capabilities—now fully supported in vLLM for fast, eff	2.3×	17k	232	35	53
85	Aug 26, '25	🚀 LLM Compressor v0.7.0 is here! This release brings powerful new features for quantizing large language models, including transform support (QuIP, SpinQuant), mixed precision com	2.3×	17k	245	44	95
86	Sep 10, '25	Thank you @cHHillee for the great explanation and demo of how to implement deterministic inference on vLLM!!!	2.2×	16k	185	18	64
87	Jul 19, '25	Thanks for the great write-up! 🙌 Prefix caching is critical for agentic workflows like @manusai , and vLLM makes it seamless. ✅ prefix caching is enabled by default with an effici	2.0×	15k	189	18	82
88	Jul 1, '25	"The second most intelligent open weights model after DeepSeek R1, with a much longer 1M token context window!" Checkout the blog post from @minimax_ai on how the model is implemen	1.9×	14k	152	23	55
89	Apr 25, '25	looking amazing! We found the quality of the documentation and diagram to be great and accurate. Will recommend :D	1.9×	14k	206	24	76
90	May 7, '25	Great work! We love how @vllm_project is used in the rollout process with with offloading the engine to CPU and give the GPU back to the kernel to be benchmarked! This is a small f	1.8×	13k	179	17	39
91	Jul 23, '24	vLLM, running on the latest @nvidia H200 GPUs, is delivering high token throughput per user on Llama 3.1 405B out of the gate! Check it out 👇	1.8×	13k	115	13	23
92	Sep 29, '25	Getting ready to try DeepSeek-V3.2-Exp from @deepseek_ai ? vLLM is here to help! We have verified that it works on H200 machines, and many other hardwares thanks to the hardware pl	1.7×	12k	134	14	36
93	Jun 5, '25	Congrats on the launch! vLLM is proud to support the new Qwen3 embedding models, check it out 👉🏻	1.7×	12k	175	21	46
94	Aug 25, '25	🌏 vLLM Shanghai Meetup Recap 🚀 Last weekend, we gathered with the community in Shanghai to dive into: → Contributing to vLLM → Distributed inference → ERNIE 4.5 integration → Moo	1.7×	12k	174	32	31
95	Jun 14, '24	We just released v0.5.0 of vLLM, one week away from our one year anniversary! The new release marks many features entering beta phase ready for testing and usage. These features bo	1.6×	12k	124	25	35
96	Jun 10, '25	⬆️ uv pip install -U vllm --extra-index-url --torch-backend=auto Try out Magistral on with vLLM 0.9.1rc1 today! 🔮	1.6×	11k	146	21	42
97	May 12, '25	"inference uses @vllm_project"!	1.5×	11k	142	5	12
98	Sep 1, '25	🚀 vLLM now supports Kwai Keye-VL-1.5! With sharper video 📹 & image 🖼️ comprehension, stronger reasoning, and an extended 128K context length, this model unlocks richer conversat	1.5×	11k	157	17	57
99	Apr 5, '24	vLLM 0.4.0 release has a suite of new additions! To start, we added support for @cohere's Command+R, @DbrxMosaicAI's DBRX, @Alibaba_Qwen's latest MoE. Notably all of these models a	1.2×	9k	124	26	31
100	May 2, '25	Very cool spec decode technique built on top of @vllm_project ! We love the idea of suffix decoding 👀	1.0×	7.2k	116	8	35

Scan another creator

Run this on any account.

XOutlier builds this table for every public profile you open on x.com, with live data and every post, not just the top 50.

Add to Chrome Download .zip