Recap · Saturday, 24 October 2026 · ASIC Room, 31st floor, MS Siam Tower

Ray Summit 2026 Recap

On Saturday 24 October 2026, six speakers from the community retold the keynotes and the Ray and vLLM news from Ray Summit 2026 for teams serving models in Thailand. Here are their slides, talk by talk, and photos from the morning.

Hosted by vLLM Bangkok with AGICAFET · In partnership with Anyscale ↗ · Venue sponsored by SVOA ↗

Slides, talk by talk

Each talk with the deck its speaker shared, in programme order.

  1. Talk 01 ·

    Opening & What’s Ray and vLLM + Serving a Frontier Open Model with Ray & vLLM

    Natdhanai Praneenatthavee · CEO & CAIO · AGICAFET

    Original session: Eyal Lantzman, Nscale ↗

    Natdhanai Praneenatthavee opens the morning with Ray Summit 2026 (San Francisco, 24–26 August): the keynotes, the breakouts, the first vLLM Conference and the business moves behind Ray and vLLM, then a beginner’s tour of Ray, vLLM and Ray Serve LLM. Part 3 retells the Ray Summit breakout by Eyal Lantzman of Nscale on serving DeepSeek-V4-Flash with Ray and vLLM: four deployment shapes on 8x B200, prefill/decode disaggregation and the caches behind a 1M-token window.

    Open PDF32 pages · 1.0 MB
    In the deck
    • Opening: why a Thai recap of Ray Summit, and vLLM Bangkok so far: three events in 2026
    • Ray Summit 2026 in six numbers, keynotes where RL became a production problem, and breakouts sharing one pattern: one Ray cluster underneath data, training and serving
    • The first vLLM Conference and which of today’s talks retell its sessions, then three business moves: Nscale agrees to acquire Anyscale, Inferact launches with a $150M seed, Ray joins the PyTorch Foundation
    • Ray for beginners: ordinary Python on many machines as if they were one, tasks, actors and objects, one core with five libraries, and Anyscale, the company behind Ray
    • What vLLM is, why vLLM is the engine and Ray the control plane, what happens to one request in Ray Serve LLM, and its whole API on one slide
    • Part 3, retold from Eyal Lantzman (Nscale): DeepSeek-V4-Flash, 284B total and 13B active parameters with a 1M-token window, and the four shapes tested: TEP8, DP8xEP, prefill/decode disaggregation and whole replicas
    • Long context, where the cache is the product: KV handed from prefill to decode over NIXL, fp8 KV with block-size 256, the fp4 indexer cache, and max-model-len as the context-vs-throughput dial
    • Measured on 8x B200 with vLLM v0.27.1: plain TEP8 beat the published reference shape, 2,209 vs 1,634 tok/s at 32 concurrent; evaluate the outputs, not just tok/s, and pin the engine version

    Links

  2. Talk 02 ·

    State of vLLM 2026

    Paphanin Amnuaikitwanit · AI Engineer · vLLM Bangkok

    Original session: Woosuk Kwon, Zachary Xi (Inferact)

    Paphanin Amnuaikitwanit (Toon) retells the State of vLLM 2026 talk by Woosuk Kwon and Zachary Xi: why owning inference matters as open models keep arriving, and the project’s three areas of focus — a core redesign (Model Runner V2, asynchronous scheduling, a Rust API server), model optimization (kernel fusion, composable parallelism, speculative decoding) and agentic workloads (KV reuse, Mooncake, session-aware routing, smarter cache eviction) — then what comes next.

    Open PDF35 pages · 2.1 MB
    In the deck
    • Open models and inference ownership: about 10 days between open-model releases; vLLM at 89.7K GitHub stars and 3,251 contributors (GitHub, August 2026)
    • Core redesign: Model Runner V2 moves selected bookkeeping to the GPU (16K to 25K output tok/s, +56.2%), and asynchronous scheduling lets the CPU schedule the next step while the GPU is still executing
    • A ModelState API for standard LLMs, diffusion LMs and Whisper; a Rust API server with ~7 ms startup and under 100 MB peak memory at 1400 concurrency; one standard KV cache layout for GQA, MLA, CSA and KDA
    • Model optimization: why custom GPU kernels, and fusing GEMM with reduce-scatter so compute and communication can overlap
    • Parallelism: why tensor parallelism alone is not always enough for modern models, composable parallelism (TP, PP, DP, EP, SP, PCP, DCP), and GLM-5.2 with prefill context parallelism at ~3.5x faster TTFT than TP in the reported test
    • Speculative decoding: Eagle, MTP, DFlash 1 & 2, DSpark and RecoverSSM, compatible with logprobs, structured outputs and PD disaggregation, with training integration through Speculators + TorchSpec
    • Agentic workloads: one user task can need several inference requests, so multi-turn KV reuse, the Mooncake distributed KV pool, session-aware routing (Dynamo, llm-d, SMG), smarter cache eviction and sparse retention
    • What’s next: simpler KV management APIs, performance metrics and models with granular metrics, broader hardware support (AMD GPU, Google TPU), and RL and multimodal integration

    Links

  3. Talk 03 ·

    Beyond Text-to-Video: Omni-Context Generation in a 33B Model

    Sattaya Singkul · Specialist Lead of AI Solution and Platform · True Digital Group

    MiniMax H3, an open-weights 33B model ranked #1 in video editing, generates video and audio natively in one sequence, and that choice has a price: a 10 s clip took 82.2 s on 8x B300. Sattaya Singkul starts from MiniMax’s H3 session at Ray Summit 2026 and follows how each bottleneck was engineered down — precompute and quantization, Ulysses across GPUs, the vLLM-Omni runtime, FastH3 distillation to 8.7 s, and Ray Serve for many users at once.

    Open PDF22 pages · 1.9 MB
    In the deck
    • The hook: H3 is #1 in video editing (Elo 1,132 on Artificial Analysis, 1,390 on Arena Video Edit) with open 33B weights to download and self-host — why it wins, and what it costs
    • Act 1, the choice: multimodality should be native, not assembled — one model for text, image, video and audio instead of a T2V, TTS, lip-sync and upscale chain; the H3 system is Context-IR (private), H3-Base at 768p (open weights) and Regenerate-2K (private)
    • Act 2, the cost: H3-Base encodes, packs everything into one sequence, generates with a 33B Omni Transformer and decodes, so a 10 s clip takes 82.2 s on 8x B300 with ~128 GB of memory, 58,758 tokens and 49 denoise steps
    • Fix 1, memory: turn the AdaLN MLP into a precomputed lookup table and quantize the checkpoint from 123.6 GB (BF16) to ~42.5 GB (pruned INT8 + NVFP4), down to 16 GB cards
    • Fix 2, sequence length: Ulysses splits the 59K-token attention across GPUs (the GPU count must divide 56 heads), plus layerwise offload (37.5% less HBM for 5.1% more latency) and a disaggregated encoder
    • Fix 3, latency: the diffusion-native vLLM-Omni runtime takes 82.2 s to 56.9 s, lossless; FastH3 distils 49 steps to 4 (DMD2 + VSA) for a 10 s clip in 8.7 s, faster than playback
    • Fix 4, concurrency: Ray Serve runs H3 as a deployment graph with a queue and backpressure, autoscaling on queue depth and replica recovery — vLLM-Omni makes one request fast, Ray keeps many requests alive
    • Act 4, before and after: 82.2 s to 8.7 s, 123.6 GB to ~42.5 GB, 8 GPUs to a 16 GB card (different test runs, so read it as the scale of change), and three questions for your next model: the choice, the cost, the fixes

    Links

  4. Talk 04 ·

    Accelerating Large Language Models: vLLM on TPUs

    Kanin Kearpimy · Fellow · Python Software Foundation

    Original session: Qi Zhou, Google

    How does vLLM run on Google TPUs? Kanin Kearpimy builds the answer up from first principles: how a CPU, a GPU and a TPU each compute (SISD, SIMD, SIMT and the systolic array behind the TPU’s Matrix Multiply Unit), the two ideas that make vLLM fast (continuous batching and PagedAttention), and the TPU software stack, from TorchAX, the current vLLM runtime on TPU, to TorchTPU.

    Open PDF32 pages · 0.7 MB
    In the deck
    • The project: vllm-project/tpu-inference, a unified backend supporting PyTorch and JAX on TPU
    • How many AI accelerators are on the market? Beyond TPU and GPU, 50+ startups are building their own chips
    • Scalars, vectors, matrices and tensors, and how hardware computes them: SISD on a CPU (memory bound), SIMD with vector registers, SIMT across GPU threads
    • Large tensors: a neural network comes down to multiplying inputs by weights
    • Inside the TPU: processing elements in a systolic array form the Matrix Multiply Unit (MXU), fed from host memory, DRAM and unified memory (SRAM)
    • How vLLM works: continuous batching uses ~100% of GPU computing, and PagedAttention keeps memory fragmentation to a small proportion
    • Programming the TPU: TorchAX, the current vLLM runtime on TPU, goes through JAX; TorchTPU goes from PyTorch straight to StableHLO and the graph IR (XLA), with faster compilation and speed gains case by case
    • Key takeaways: vLLM has run on TPU for a long time, there is no silver bullet across GPU, TPU and CPU as we head towards heterogeneous hardware, and TorchTPU has come for vLLM on TPU

    Links

  5. Talk 05 ·

    AMD and vLLM: What’s New

    Passin Pornvoraphat · PhD Candidate · Chulalongkorn University

    Original session: Douglas Lehr, AMD

    In 2026 AMD moved its vLLM work from a fork to upstream. Retelling AMD and vLLM: What’s New, the session Douglas Lehr of AMD gave at the first vLLM Conference inside Ray Summit 2026, Passin Pornvoraphat shows what that changed: 1,041 AMD and ROCm PRs merged in a year (14.2% of all vLLM PRs in July 2026), AMD CI grown from 29 to 128 nodes, the path a kernel takes from FlyDSL through AITER and ATOM into vLLM, Quark and TraceLens, vLLM on Radeon, and what comes next.

    Open PDF36 pages · 0.8 MB
    In the deck
    • AMD in vLLM today, Aug 2025 to Jul 2026: 1,041 AMD and ROCm PRs merged from 174 authors, 73% of them written by AMD engineers; ROCm PRs per month up from 25 to 163, 14.2% of all vLLM PRs in July 2026; 48 AMD engineers contributing for the first time this year, and five AMD-side code owners
    • 2026, the year AMD moved upstream: the ROCm/vllm fork (Nov 2023 to Sep 2025) is retired; vLLM itself builds and releases the ROCm wheels and the vllm/vllm-openai-rocm image, model settings live in vllm-project/recipes, and fixes go upstream first
    • AMD CI inside vLLM: test groups from 96 to 193, 73 of them gating merges, Docker build per PR down from 40 to 5 minutes, the AMD pass rate from 63.9% in May to 100% in August, and CI nodes on Instinct GPUs from 29 to 128
    • The Instinct family from MI250 to MI450, speed and cost per token against NVIDIA on DeepSeek R1 (SemiAnalysis InferenceX), and Day 0 support: new models run on ROCm on launch day, and MI450 (gfx1250) is already supported upstream
    • How a kernel reaches users: written in FlyDSL (a Python DSL with an MLIR compiler; 24 PRs merged, 28 in review), assembly, Triton or Composable Kernel, collected in AITER, which picks the fastest one, proven in ATOM, then upstreamed into vLLM; MoRI-IO moves the KV cache for prefill/decode disaggregation, and ATOM runs as a vLLM plugin without touching vLLM core
    • Tools: Quark for quantization (MXFP4, MXFP6 and MXFP8 upstream, NVFP4 emulated on MI300 and MI355X, a 4-bit KV cache with TurboQuant), TraceLens to see where the time goes in a trace, and NIXL shared by both vendors now that AMD has retired its RIXL fork
    • vLLM on Navi: from Ryzen AI laptops and mini PCs (up to 128 GB of unified memory on Ryzen AI Halo) to Radeon 7000 and 9000 desktops and 1 to 8 GPU servers, with RDNA3 and RDNA4 running upstream vLLM
    • What comes next: ROCm through pip install vllm with TheRock, Mooncake as another transport for prefill/decode, and batch invariance; then the repos to start from and #sig-amd on the vLLM Slack

    Links

  6. Talk 06 ·

    NVIDIA and vLLM Full-Stack Collaboration for MoE Model Performance

    Arisa Umparat · AI Engineer & NLP Researcher · Ascend Group

    Original session: Siyuan Fu, NVIDIA

    The strongest GPU alone does not make a Mixture-of-Experts model answer fast. Retelling the Ray Summit talk by Siyuan Fu of NVIDIA, Arisa Umparat climbs four levels of bottleneck with a fix for each: kernel fusion with FlashInfer for Kimi K3 (72 to 111 tokens/sec), one-shot all-reduce for MiniMax M2.5 (latency down 52%), prefill/decode disaggregation for DeepSeek R1 on GB200 NVL72 (10,000 tokens/sec per GPU), and NVIDIA Dynamo for KV-aware routing across the cluster.

    Open PDF61 pages · 15.1 MB
    In the deck
    • Is the strongest GPU enough? Not always, especially for Mixture-of-Experts models: how a router sends each token to its experts, and splitting a model across GPUs with tensor parallelism (TP) or expert parallelism (EP)
    • The NVIDIA x vLLM open-source community: 60+ Nvidian vLLM contributors, 110+ PRs merged during the past 3 months, and Day 0 support for 12 models so far
    • Four levels of bottleneck, micro to macro: kernel compute, cross-GPU communication, memory and hardware, system orchestration, and the full-stack optimization matrix that pairs each level with a model and a fix
    • Kimi K3, latent MoE on GB300 (TP8): the MoE tail collectives fused with FlashInfer by splitting the all-reduce, overlapping communication with compute and fusing all-gather with GEMM; the MoE tail drops about 19.5% and speed goes from 72 to 111 tokens/sec
    • MiniMax M2.5, full-heads RMSNorm: tiny payloads, big syncs; a one-shot all-reduce cuts 2 sync barriers to 1, from 15.5 µs to 7.4 µs (latency down 52%), and FlashInfer picks one-shot below batch size 40 and two-shot otherwise
    • DeepSeek R1, PD disaggregation on GB200 NVL72: NVFP4 quantization before all-gather cuts network traffic 4x, the V2 weight offloader shrinks prefill from EP4 to EP2, and 4 prefill workers (EP2) to one decode worker (EP8) reach 10,000 tokens/sec per GPU
    • Take-aways: split all-reduce into reduce-scatter and all-gather and hide each one’s latency, fuse all-gather with GEMM, use a one-shot all-reduce kernel when traffic is small, and use a smaller parallel size in prefill workers
    • NVIDIA Dynamo for distributed serving: requests coordinated between vLLM, SGLang or TensorRT-LLM workers on Kubernetes, a KV-aware router weighing prefix match against load, NIXL moving the KV cache, fault tolerance, InferenceX results on NVL72 and the Dynamo 2.0 roadmap

    Links

Programme and event details

Photos

Moments from the event, from the opening to the group photo.

Natdhanai Praneenatthavee (Job) opens the Ray Summit 2026 Recap in the ASIC Room, beside the slide “Why a Thai recap of Ray Summit”, with the audience seated in front
  • Natdhanai Praneenatthavee (Job) looks back on vLLM Bangkok’s events of 2026, from IBM TechXchange in April to this recap
  • Natdhanai Praneenatthavee (Job) points to the slide “Ray in one minute: tasks, actors, objects”, a short code example of Ray’s core ideas
  • Paphanin Amnuaikitwanit (Toon) opens State of vLLM 2026 with a slide on open models and owning inference
  • Paphanin Amnuaikitwanit (Toon) points to a diagram of the Mooncake integration and its distributed KV cache pool in vLLM
  • Sattaya Singkul (Jo) explains why MiniMax H3 tops the video-editing leaderboard, in the talk Beyond Text-to-Video
  • Sattaya Singkul (Jo) shows how Ray Serve runs MiniMax H3 in production for many users at once, from the request queue to vLLM-Omni
  • Kanin Kearpimy (James) at the presenter table, speaking on vLLM on TPUs
  • A wide view of the ASIC Room as Passin Pornvoraphat presents AMD and vLLM, here on the Quark and TraceLens tools
  • Arisa Umparat compares tensor parallelism and expert parallelism for splitting a model across GPUs, in the NVIDIA and vLLM talk
  • The opening’s thank-you slide for venue sponsor SVOA, with the hosts vLLM Bangkok and AGICAFET and partner Anyscale
  • Attendees, speakers and organisers of the Ray Summit 2026 Recap give a thumbs-up for the group photo

More vLLM sessions in Thailand

Meetups and talks from other events, with the slides.

All events →
  • Speakers and organisers together on stage under the vLLM Bangkok Day 2026 LED wall

    Past eventOur flagship event

    · 14:00–19:00 · JAS Green Village Ramkhamhaeng

    vLLM Bangkok Day

    Our flagship afternoon at JAS Green Village: talks from vLLM maintainers, Thai production stories, two panels and a full house. The recording, slides and photos are on the recap page.

    Recording, slides & photos →5 decks

  • Pie & AI title slide: a sweet meetup for deep learners, by DeepLearning.AI

    Past eventOrganised by our community

    · 13:00–18:00 · Cleverse Rama 9

    Pie & AI Bangkok 2026

    A DeepLearning.AI community meetup hosted by AGICAFET

    An afternoon on AI at the edge, organised by AGICAFET as a DeepLearning.AI Pie & AI meetup at Cleverse Rama 9: an opening from Andrew Ng, then talks on NVIDIA Isaac and physical AI (AGICAFET), agentic orchestration on local hardware (IBM), KV-cache reuse (Tensormesh), practical edge AI (True Digital), local AI for autonomous vehicles (Aueaphum Aueawatthanaphisut), Mamba inference on embedded systems (Friso Smit) and letting vLLM pick the right skill from a large collection (Datakarate), with networking to close.

    Slides & details →5 decks

  • Title slide: Scaling Intelligence — Agentic Workflows with vLLM, Jun Kang Chow, TechXchange 28 April 2026

    Past eventA vLLM session at another organisation’s event

    · 15:00–19:00 · IBM Thailand (SP Building)

    IBM TechXchange Thailand 2026

    The Agentic Workforce: Scaling Multi-Agent Orchestration

    A half-day on multi-agent orchestration hosted by IBM Thailand with the AI Alliance at the SP Building. Speakers from KBTG, ETDA and its AI Governance Clinic, SCG Digital, CJ More, EGG Digital, NECTEC, IBM and Embedded LLM, plus panels on Responsible AI frameworks and enterprise security. Embedded LLM gave the vLLM session.

    Slides & details →1 deck

Stay in the loop

One email when the next event is announced or its registration opens, plus the occasional note on what the Thai vLLM community is building. No spam, unsubscribe any time.