Most retrievers ask a query to explain itself before seeing the search results. ARR flips that sequence: it retrieves an item, learns from the feedback, and then searches again. Retrieval systems increasingly need to search across text, images, and their combinations, supporting applications including retrieval-augmented generation, multimedia platforms, and e-commerce search.

Universal multimodal retrieval addresses this need with a single model that maps heterogeneous queries and items into a shared embedding space. In the standard embedding-based approach, queries and items are encoded independently and matched by similarity.

This separation enables item embeddings to be computed in advance and reused across queries, making large collections searchable through a pre-computed index. However, this separation limits a query’s ability to leverage the collection being searched.

A query may omit important details, express its intent only implicitly, or contain visual and textual cues of unequal importance. An independently computed query embedding must resolve these ambiguities before observing any candidate items. At inference time, the query representation remains fixed and cannot adapt to candidates encountered during retrieval; this is the query–item gap.

The retriever should respond to newly encountered items, and retrieval feedback offers a complementary direction. Prior methods either directly refine dense query representations using retrieved documents, or rewrite queries and re-encode them. Yet retrieved items are typically processed by an auxiliary refinement mechanism rather than by the retriever itself.

The central question is whether the retriever can learn to use the items it actually encounters to decide how to search better. ARR is a multimodal retrieval model that alternates between retrieving an item and updating the query embedding; this iterative process is called autoregressive retrieval.

At each intermediate step, ARR selects an item by embedding similarity, appends its multimodal content to the query history, and encodes the expanded context for the next search. After a fixed number of steps, the final embedding ranks the collection.

Each selection depends on the original query and earlier feedback, while item embeddings remain independently indexable. The figure’s lower path makes the update mechanism explicit: the first causal multimodal language model receives only the query, the next receives the query plus item A, and the final receives the query with items A and B.

Above, the retrieved item changes from A to B before the process ends in a ranked list, while the item representations remain separately indexed. Learning this process requires distinguishing an item’s relevance as an answer from its usefulness as feedback.

An item that is not the target may still reveal information that helps retrieve the target later, while a plausible match may distract the encoder. The model receives item content without a relevance label and must learn which evidence to use.

Supervised fine-tuning first teaches the encoder to interpret feedback using stepwise contrastive supervision on offline trajectories. Reinforcement learning then learns feedback selection from trajectories sampled by the current policy, using final retrieval quality as the reward; its actions are retrieved items rather than generated text tokens, with a query-side adapter learning against a fixed item index.

The item corpus and the query can involve text, images, or a combination of the two. A query may also specify an instruction describing the desired relevance relation. Universal multimodal retrieval uses a shared retrieval model across these input configurations to rank items according to their relevance to the query.

A training example pairs a query with an annotated relevant item. An MLLM-based encoder maps queries and items into a shared embedding space. Text and images are converted into token representations and assembled using an input template.

The MLLM processes the resulting sequence, and an embedding readout extracts a fixed-dimensional vector from its hidden states. After normalization, the inner product of the embeddings measures cosine similarity in the shared embedding space. ARR conditions query representations on retrieved content while preserving independently encoded item embeddings.

The method defines the retrieval process, then describes supervised fine-tuning for interpreting feedback and reinforcement learning for selecting feedback that improves the final ranking. Figure 2 contrasts the two stages of ARR training. In supervised finetuning, the authors use offline feedback trajectories with stepwise contrastive losses and a degradation penalty, while only the initial-step loss updates standalone item embeddings.

In reinforcement learning, the SFT model and item embeddings are frozen, and a query-side LoRA adapter is trained from sampled trajectories, terminal reciprocal-rank rewards, group advantages, GRPO, and final-embedding supervision. ARR uses N query states with N minus one feedback selections.

It first encodes the original query, then repeatedly retrieves an item and appends its content and available metadata to the query context. The encoder processes this context to produce the next query embedding, and the Nth embedding determines the final retrieval results.

No relevance label is included in the feedback. Earlier selections affect both the next query embedding and subsequent selection probabilities, making retrieval autoregressive over item identities. After incorporating N minus one feedback items, ARR uses the final query representation to retrieve the top-k items.

Supervised fine-tuning teaches the encoder to use retrieved content while preserving the original query intent. To avoid repeatedly searching the corpus during this stage, an offline feedback pool is constructed using the pretrained encoder. The state is the complete history, the action is a feedback item, and the transition appends that item to the history.

This defines a finite-horizon Markov decision process with deterministic transitions. The item pool and precomputed static item embeddings support the terminal reward, which is the reciprocal rank of the relevant item under the final query embedding.

ARR is trained and evaluated on M-BEIR, which contains sixteen dataset–task combinations spanning eight query–item modality configurations. On the M-BEIR test set, ARR-RL achieves the highest average score among the compared methods at both scales: 56.7 for two billion and 61.0 for eight billion.

Reinforcement learning improves the reported averages over ARR-SFT by 0.7 and 1.2 points, respectively. At eight billion, ARR-RL exceeds the strongest baseline average, TRACE’s 58.8, by 2.2 points. The table shows reinforcement learning shifting performance selectively across retrieval directions, not simply increasing every metric.

Feedback use must be learned: ARR-SFT with one training state scores 55.23 initially but falls to 46.90 when extended to four inference states. In contrast, ARR-SFT with four training states improves from 55.63 to 56.01.

Most of this gain occurs after the first feedback item, at Iter-2 with 55.95, followed by smaller changes. Table two separates training with feedback from using feedback at inference time: rows indicate the number of training states, while columns show performance after each inference step.

ARR-SFT with one training state drops from 55.23 at Iter-1 to 46.90 at Iter-4, whereas ARR-SFT with four states changes from 55.63 to 56.01. The RL variants show smaller changes across steps, with ARR-RL, subscript four, reaching 56.74 at Iter-4, illustrating how the training objective affects whether additional feedback is useful.

The zero-shot evaluation covers seven benchmarks for eight billion models without task-specific fine-tuning. ARR-RL obtains the highest reported average, 79.01, compared with 74.10 for the strongest baseline average. Scores of 77.9 on Visual Dialog and 66.0 on Multi-round FashionIQ indicate transfer to dialogue and interactive retrieval.

Across the two ARR methods, the final-state rows show consistent feedback gains over Iter-1, especially in the CIRCO column: ARR-SFT rises from 39.3 to 41.6, while ARR-RL rises from 39.9 to 43.4. The final ARR-RL row also records 88.0 on Flickr and 98.3 on the third query-to-item column, both marked in bold.

ARR learns to select and use retrieved items as feedback through an autoregressive retrieval process. It preserves independently indexed item embeddings while conditioning the query on a sequence of retrieved observations. Supervised fine-tuning teaches feedback interpretation, while reinforcement learning optimizes item selection for the final ranking outcome.

Experiments show improvements in average in-domain and zero-shot retrieval performance. Using feedback during inference improves subsequent retrieval, while learning from feedback during training improves initial retrieval before any feedback is observed.

ARR keeps the efficiency of independently indexed items while making the query responsive to retrieved feedback. The results show gains both when feedback is used at inference and when the model learns from it during training.