The Shift From Search at Inference to Search in Training

For the last few years, web search has mostly been an inference-time capability: a model encounters a question, reaches beyond its weights, retrieves fresh information, and uses it to produce an answer. Now search is moving upstream into training. Models are increasingly learning from trajectories in which they search, evaluate evidence, reformulate queries, use tools, and decide when they have enough information to stop.

That changes what search infrastructure needs to be. It can no longer be optimized only for relevance, latency, and reliability at inference time; it also needs to support the generation, evaluation, and reproduction of the experiences models learn from. In other words, AI is beginning to require training-grade web search.

Training-grade search means search that can function as a learning environment: preserving what a model saw, providing sufficiently consistent retrieval behavior for agents to learn from it, and allowing important interactions to be replayed or reconstructed when needed.

NVIDIA’s work on Nemotron 3 Ultra, which uses Tavily inside post-training and evaluation, offers an early example of this shift.

From answers to search trajectories

Traditional training data usually captures a relatively simple mapping: given an input, produce the desired output. Tool-using agents require something richer.

A research agent must decide whether it needs external information, what to search for, how to interpret the results, whether the evidence is sufficient, and what to do next. These are not simply pieces of knowledge. They are policies for interacting with an external environment.

The natural unit of training, therefore, becomes a trajectory: a sequence of actions and observations that leads from a task to a result.

NVIDIA’s Nemotron 3 Ultra post-training pipeline illustrates this directly. Teacher models are given difficult questions and access to Tavily Search. The resulting training data can contain the full research process: an initial query, incomplete evidence, a reformulated search, another source, an intermediate conclusion, and eventually a final answer. NVIDIA also uses longer research trajectories and scientific reasoning traces in which models can combine Tavily with a Python execution environment for computation and verification.

The distinction is important. A conventional dataset teaches the model that a question has an answer. A search trajectory can teach the model how to discover that answer.

Search becomes part of the learning environment

Once search appears inside a training trajectory, retrieval is no longer just a tool used by a finished model. It becomes part of the environment against which the model learns.

Consider an agent in the middle of a task. It has a partial understanding of the problem and chooses to issue a query. The retrieval system determines what evidence it observes next. That evidence changes the model’s state, which influences its next action, which may generate another query.

The training trajectory is therefore partly determined by the retrieval system.

This creates an important difference between inference-time and training-time search. At inference time, poor retrieval can degrade one answer. During training, poor retrieval can alter the experiences from which the model learns its future behavior.

A weak search environment may teach a model that searching is rarely useful. Noisy retrieval may encourage repeated querying because useful evidence requires many attempts. Higher-quality retrieval can support more efficient policies in which the model learns when one query is enough and when further investigation is necessary.

Search quality therefore affects more than retrieval quality. It can influence the search policy the model develops.

Models may also begin to learn regularities in the search environment itself. A query asking for an SEC filing, for example, should reliably surface authoritative primary sources. Other classes of queries may repeatedly surface particular types of sources or evidence. Over many trajectories, these patterns give an agent expectations about what different searches are likely to return.

This does not mean results need to be identical every time. But a training environment benefits from enough consistency in source quality and retrieval behavior that models can form useful policies around search.

The web becomes part of the curriculum.

Training and evaluation start to converge

Once models learn through interactions with tools, the infrastructure for training and evaluation begins to overlap.

An agent evaluation requires a task, a model, tools, observations, and some definition of success. Reinforcement learning and synthetic trajectory generation require nearly the same components. This means the same search-enabled environment can increasingly be used to generate training data, measure model behavior, and diagnose failures.

NVIDIA uses Tavily across both post-training and evaluation environments. Search can also operate alongside more specialized tools, for example, structured access to domain-specific data such as SEC filings - allowing agents to combine general web retrieval with purpose-built information sources.

This creates a natural feedback loop: an evaluation exposes a failure, a stronger model generates a better trajectory, that trajectory becomes training data, and the updated model is evaluated again.

Frontier models, routers, and production telemetry extend this loop beyond the training environment itself. Frontier models can act as teachers, generating stronger search trajectories for tasks that smaller models fail to solve. Routers expose the boundary of those smaller models’ competence by escalating difficult cases, creating a stream of high-value examples for future training. Production telemetry can preserve the resulting trajectories, including queries, retrieved sources, tool calls, intermediate failures, and final outcomes — and turn them into new evaluations or training data.

These failed or escalated trajectories can be especially valuable because they reveal not only that the model failed, but where the search process failed: the initial query may have been poor, the wrong sources may have been selected, the model may have stopped too early, or additional verification may have been needed.

Search sits inside each stage of this process. A production failure may originate in a poor query or insufficient evidence; a teacher model may solve the same task by following a better search trajectory; that trajectory can then become supervision for the next model. In this sense, retrieval is not simply one tool among many in the learning loop. It is part of the mechanism through which external information is converted into training experience. As models are improved from their own interactions with the web, the quality and reproducibility of that search environment become increasingly important.

The live web creates a new training problem

Training on web search introduces a technical problem that matters much less during ordinary inference: the web changes.

Search rankings move. Pages are updated or deleted. New information appears. The same query may produce a different observation tomorrow.

At inference time, this is desirable. A model should operate on the current state of the world. During training and evaluation, however, it complicates reproducibility.

If an evaluation score changes, did the model improve, or did the retrieval environment change? If a useful production trajectory becomes training data six months later, can the original evidence still be reconstructed? If two model checkpoints are evaluated against different versions of the web, are the results comparable?

Fixed retrieval environments solve part of this problem because they can be replayed. Live search solves a different problem because it exposes models to real, changing information.

Training systems will increasingly need both: live environments for real-world dynamics and reproducible environments for controlled learning and evaluation.

In practice, this could mean a live search layer where models encounter current information and natural variation in the web, alongside a snapshot or replay layer that preserves specific retrieval states for evaluation, debugging, or future training. These solve different problems: one keeps the model connected to the real world, while the other makes experimentation comparable and repeatable.

Toward training-grade web search

This is what separates training-grade web search from ordinary search infrastructure.

Inference-time search is primarily optimized for relevance, accuracy, latency, and reliability. Training retains those requirements but introduces additional ones: provenance, stable interfaces, large-scale trajectory generation, reproducibility, and sufficiently predictable retrieval behavior.

Predictability does not mean returning the same pages forever. It means that the retrieval environment has enough consistency in source quality, result semantics, and behavior that models can form useful expectations about what different search actions are likely to produce.

The retrieval layer also becomes part of the experimental apparatus. If changing the search system changes the observations available to the model, then it may also change the policy that emerges from training. Researchers therefore need to distinguish changes in model capability from changes in the information environment.

Training-grade search is therefore not simply a search API capable of handling more traffic. It is a web environment designed to participate reliably in model development: delivering high-quality observations, behaving consistently enough for useful search policies to emerge, and providing the provenance and experimental control needed to reproduce important interactions.

That does not mean a model trained with one retrieval provider should remain dependent on it. Robust agents should generalize across tools. At the same time, retrieval systems have their own distributions — source preferences, ranking patterns, result structures, and tool semantics — and models trained extensively against one environment may learn some of those regularities.

The deeper point is that what a model experiences during training influences what it learns, and retrieval determines a significant part of that experience.

The first generation of AI used the web primarily to answer questions.

The next generation will increasingly use the web to learn how to answer them.