---
title: "Full Sail on Asynchronous Inference"
description: "The inference market is the largest in software. We invested in Sail's Series A to build queuing infrastructure for long-horizon agents that run for hours."
categories: ["AI","infrastructure"]
keywords: ["long-horizon agents","inference queuing","Sail Research","AI infrastructure","async inference","fleet-aware orchestration"]
ai_summary: "The inference market is the largest in software. AI workloads are shifting from synchronous chat to asynchronous, multi-turn agents that run for hours. Sail Research builds fleet-aware orchestration to maximize throughput per dollar of inference spend."
date: 2026-06-25
lastmod: 2026-07-21
canonical_url: https://www.tomtunguz.com/sail-inference-queue/
author: "Tomasz Tunguz"
---


Today all inference is real-time. A human types, a model responds, & the clock starts over. The infrastructure is built for someone waiting on the other end. Every millisecond of latency costs money because the serving stack optimizes for cold-start, not throughput.

As we built internal AI systems at Theory, we embraced queueing. Parallelize ten agents on a single task, let them run for hours, & the productivity gains are enormous. It is the product of token-maxxing,[^2] pushing every dollar of compute to do more work. But the cost was unsustainable.
{{< email_image src="ebfpqjsvkxhlhnpb2x9c" alt="sail" width="540" height="360" >}}
That is when we met [Neil Movva](https://www.linkedin.com/in/nmovva) & [Samir Menon](https://www.linkedin.com/in/samir-menon-27954214b) of Sail Research.[^5]

[Neil Movva](https://www.linkedin.com/in/nmovva) built one of the fastest LLM inference stacks at Together AI. [Samir Menon](https://www.linkedin.com/in/samir-menon-27954214b) ran LLMs inside hardware enclaves at Blyss. Both are systems engineers to the core. They were building the system we needed.

As the inference market segments into real-time, near-real-time, & batch, async inference sits in the batch tier & carries a massive cost advantage.[^3] The key is model selection & routing.

Sail distributes requests across open models like DeepSeek, Qwen, Kimi, & GLM, picking the cheapest capable model for each task. GLM-5.1 on Sail costs 6x less per token than Anthropic's Haiku.[^4] Wait two minutes instead of two seconds for a code review, & the same token costs 6x less.

Sail uses spot capacity when it is available & fails over to reliable compute when it is not. Fleet-aware orchestration keeps utilization high & cost low.

Real-time stacks reserve capacity per request. Queued stacks pack requests into idle capacity. Different architecture, different economics.

Sailboxes are cloud computers for the bursty rhythm of agents. A sailbox stays alive as long as the agent needs, holds state across the entire task, pauses when it waits on inference, & resumes in seconds when the response arrives. You pay for active time. No paying for idle.

Sail has served trillions of tokens to customers in code review, deep research, & cybersecurity.[^5]

Today we announced our Series A investment in Sail alongside Kleiner Perkins, Redpoint, & Sequoia.

As agents grow from chat assistants into background workers scanning codebases overnight, enriching every CRM row, processing every document, the vast majority of tokens will flow through a queue. The future runs in the background. We are thrilled to partner with Neil, Samir, & the entire Sail team.

If you're building agents, [get started here](https://docs.sailresearch.com/).

[^1]: [Grand View Research : AI Inference Market Size 2024](https://www.grandviewresearch.com/industry-analysis/artificial-intelligence-ai-inference-market-report) - Estimated at $97.24b in 2024.

[^2]: [Token-maxxing](https://tomtunguz.com/tokenmaxxing/)

[^3]: [Darwinian Specialization in AI](https://tomtunguz.com/inference-market-segmentation/)

[^4]: [Sail Research : Cost Efficiency](https://www.sailresearch.com/) - Input tokens per dollar, comparing Sail's GLM-5.1 to Anthropic Haiku 4.5 & OpenAI GPT-5.4-mini.

[^5]: [Sail Research](https://www.sailresearch.com/)