Scaling Intelligence: Agentic Workflows with vLLM
Jun Kang ChowEmbedded LLM
Why run LLMs yourself, where vLLM gets its speed — continuous batching, torch.compile and fusion passes, prefill/decode disaggregation, quantization, speculative decoding — why agentic workflows hit the engine harder than chat, and a live SQL agent over the Northwind database.
In the deck
- Why run LLMs yourself? Loading a model is easy; serving it is hard
- What vLLM is: the project, 100+ model architectures, broad hardware support
- The secret sauce: continuous batching, torch.compile and fusion passes, PD disaggregated inference, quantization, speculative decoding
- What agents and tools are, and how LLM capability has progressed
- Why agentic workflows turn one request into many inference calls and hit the engine harder every turn
- Serving with vllm serve and choosing a model with vLLM recipes
- Demo: a SQL agent over the Northwind database, and a virtual workspace built by agents
No recording of this session is available — the slides are the full story.