What if a video model could generate several possible moments once, then decide afterward which ones deserve to be returned? This paper adds confidence without regenerating the timestamps. Video temporal grounding supports video search, content review, and automated editing by localizing events described in natural language.

Existing generative models typically output timestamps without explicit interval-level confidence scores to guide candidate selection. The method separates candidate generation from acceptance by scoring individual intervals within the original decoding pass, using a lightweight confidence head that reads pooled decoder states.

The resulting scores support ranking, threshold-based selection, and rejection without invoking an external verifier at inference. On a fixed OMTG-Bench candidate pool, confidence raises query-macro Recall@0.5 from 9.95% to 14.42% at a 10% global return budget, and from 26.48% to 31.12% at a 25% budget.

Video temporal grounding localizes events described by a language query in a video. Discriminative TVG models provide proposal or matching scores, while generative vision-language models express predictions as timestamp sequences that accommodate individual moments and repeated occurrences.

Yet generative responses typically provide no explicit confidence for each interval. Coordinates identify where to look, but provide no interval-level score explicitly trained for localization reliability. Applications differ in their tolerance for false and missed matches.

Content moderation, surveillance review, and comprehensive sports review may favor recall because uncertain segments can be reviewed, whereas missed events never reach review. Livestream highlight editing, content recommendations, and targeted video clipping may favor precision because false matches produce irrelevant clips or unwanted recommendations.

These requirements call for adjustable acceptance criteria over the same candidate pool, allowing applications to change which intervals they return without regenerating timestamps. At the top, the timestamp-only model produces several candidate intervals along the timeline, but every candidate is presented without a confidence value.

The center callouts show the consequence: the same timestamp set cannot be filtered reliably for automatic use, cautious manual review, or rejection when no event is present. Video moment retrieval also requires deciding which candidates to present first.

This ranking matters even when all proposed intervals are retained. Generation order need not reflect how reliably an interval matches the query, while sequence likelihood is not explicitly supervised for this purpose. Figure four shows ranking among decoded interval proposals, where the highest-confidence candidate is selected as the single answer without boundary refinement, NMS, or thresholding.

In panel a, the selected purple interval is C2, the later washing event, with tIoU zero point eight three three, rather than the first generated interval. In panel b, the selected cap-replacement interval is C5, with tIoU zero point nine zero five, while the earlier C2 has tIoU zero point six zero zero.

Queries for absent events expose a further limitation: a model may return plausible timestamps even when no matching content exists. Negative-query rejection is a recognized challenge in moment retrieval. The task therefore includes deciding whether to return any candidate at all, beyond choosing which candidate to rank first.

Figure six illustrates query-level rejection in two examples: an indoor activity video and travel footage, each paired with a matched query and a synthetic cross-video mismatch. The highest-confidence candidate is accepted for both matched queries, with scores of zero point nine six three and zero point nine seven five, while the mismatches return no result despite the generator producing candidates.

Using global thresholds of zero point five four nine for Charades and zero point five two four for QVHighlights, the figure shows why retrieval systems must be able to suppress unsupported answers. Adapting existing confidence estimators to this setting involves trade-offs.

Token likelihood scores the generated timestamp sequence without directly supervising whether its boundaries localize the queried event. Verbalized confidence adds score tokens to the response, while sampling-based semantic uncertainty requires multiple answers.

A separate visual verifier can assess individual intervals, but it costs additional model inference on candidate clips. The target is a score trained for individual temporal predictions and available from the original grounding pass. Each generated interval is paired with a continuous confidence score.

A lightweight head reads pooled decoder states for the interval, reusing features computed during the original autoregressive pass. Applications can adjust an acceptance threshold, rank candidates by reliability, and return an empty result when no candidate passes the threshold.

All three decisions use the same response, without regenerating timestamps or invoking an external verifier. Candidate generation determines what can be returned; confidence supplies the acceptance rule. Training gives the generator and confidence head distinct objectives.

The generator is trained to cover annotated occurrences, whereas confidence learning also uses imperfect and unsupported intervals. Unsupported candidates teach the head to reject without becoming part of the generator’s timestamp targets, so the two tasks use separate supervision sequences.

GT-anchored supervised fine-tuning retains every annotated occurrence and adds a bounded number of supported boundary alternatives. The confidence head learns from an offline visual verifier’s soft scores on precomputed candidate sequences, while set-level reinforcement learning optimizes candidate coverage and the localization quality of the confidence-selected set.

As proposals evolve during reinforcement learning, binary temporal-overlap labels adapt the head to its current candidates. Confidence losses update only the head, while the selected-set reward guides the generator. Experiments on OMTG-Bench assess fixed-pool selection under equal global return budgets, while additional analyses on Charades, ActivityNet, and QVHighlights examine single-answer ranking and rejection of synthetic cross-video mismatches.

The method introduces an interval-level confidence interface for generative TVG that separates candidate generation from acceptance. A lightweight head scores intervals from the original decoder states, enabling ranking, threshold-based selection, and rejection without re-decoding or an inference-time verifier.

GT-anchored targets train the generator, while offline verifier scores and online temporal-overlap labels train the confidence head. Set-level reinforcement learning connects the two by rewarding candidate coverage and the quality of the confidence-selected set. Figure 2 separates the generation and confidence-supervision paths in a single autoregressive decoding pass.

The decoder produces candidate intervals, whose span-pooled states feed a detached confidence head; generation-order NMS forms a pool, and thresholding selects the output set. During training, GT-anchored SFT and set-level reinforcement learning update the generator, while fixed replay candidates with verifier soft labels and online overlap labels train the head, allowing unsupported candidates to teach rejection without becoming generation targets.

Learning what to return increases the value of a single generation. On fixed OMTG-Bench candidates, confidence improves query-macro Recall@0.5 under tight budgets, gaining 4.47 points at a 10% global return budget and 4.64 points at a 25% budget over generation order. At inference, the head scores intervals from decoder states cached during timestamp generation, without adding confidence tokens.

Teacher-forced replay is used only for training. Single-pass scoring reuses decoding features, while generating additional candidates still incurs token cost. The same completed decode supports multiple global thresholds without regenerating intervals.

A threshold allows variable output cardinality, including an empty set, rather than imposing a fixed number of answers per query. The scores can also rank the same pool under a return budget, with thresholding fixing an acceptance score and budgeted selection fixing how many intervals may be returned.

Neither rule changes interval boundaries or recovers candidates removed by NMS. For each query, the offline supervision uses one precomputed candidate sequence including supported proposals, unmatched proposals, ground-truth completions, and duration-based negative candidates.

An offline visual verifier supplies the soft label for each candidate. These sequences are fixed before supervised fine-tuning and replayed with teacher forcing to obtain candidate features. They have zero language-model loss, and their labels are not transferred to newly synthesized generator alternatives.

The confidence head adapts to current rollouts using binary overlap labels on NMS survivors. Set-level reinforcement learning starts from the merged supervised-fine-tuning generator and paired head, with full-parameter actor updates.

Each sampled response uses the same NMS and confidence selector as evaluation. After a recall and localization warm start, the set-level reward is used for training. Recall uses maximum-cardinality one-to-one matching at temporal intersection over union of zero point five, so one proposal cannot recover two ground-truth occurrences.

The first reward term values coverage before confidence filtering, while the remaining terms reward occurrence-level F1 and temporal intersection over union of the predicted and ground-truth unions. One threshold maximizes mean F1 across the current training batch, and the same selected set supplies both quality terms.

No per-query oracle or test annotation enters training rewards. OMTG-Bench evaluates multi-occurrence grounding on 320 queries, 287 videos, and 1,173 official ground-truth intervals. The model is initialized from TimeLens2-4B; supervised fine-tuning uses 60,405 OMTG and TimeLens2 training units, and reinforcement learning uses 8,424 OMTG queries.

Single-interval TimeLens splits supply matched and synthetic unmatched queries. The evaluation reports official OMTG set temporal intersection over union, temporal F1 at zero point three and zero point five, occurrence recall, count accuracy, and EtF1, retaining all queries including empty or invalid responses.

Each model decodes once and selectors reuse its cached predictions. For full-system OMTG results, one global threshold maximizes test temporal F1 at zero point five; these are test-oracle operating points, not deployment-validated thresholds. Test labels enter offline metrics and selection feedback, never model inputs, gradients, or rewards.

Confidence improves official query-macro recall over generation order by 4.47 points at 10% retention and 4.64 points at 25% retention. The benefit is largest under tight budgets because, as more of the fixed pool is returned, selection has less room to help. Paired 95% source-video bootstrap intervals for these gains are 2.30 to 6.04 points and 2.26 to 7.09 points.

Look down the Confidence column: its recall values rise from fourteen point four two at ten percent retention to sixty-five point three zero at seventy-five percent. Across every row, the Confidence entry is also above the corresponding Order, Logp, and Verifier entries, with the widest visible separation at the tightest budget.

Figure three shows confidence-controlled selection without re-decoding. In panel a, threshold sweeps trace precision–recall curves for fixed candidate pools after non-maximum suppression, with stars marking test-F-one optima and the gray diamond showing GT-only SFT.

Panel b plots localization F-one against false-alarm rate on matched and unmatched synthetic queries, using colors for datasets and arrows to connect native models with the RL model. The figure matters because it makes the precision, coverage, localization, and false-alarm trade-offs explicit.

The supervised-fine-tuning and reinforcement-learning models share the backbone, visual budget, parser, NMS, and one-decode inference, each with its matching head. Reinforcement learning gains 3.52 points in temporal F1 at zero point three, 2.85 points in temporal F1 at zero point five, and 2.49 points in temporal intersection over union over supervised fine-tuning.

OMTG-4B retains stronger count accuracy and EtF1. Looking across the final two rows, the reinforcement-learning row is consistently above supervised fine-tuning in the temporal metrics: the gaps are visible in tF1 at both thresholds and in tIoU. In contrast, the C-Acc and EtF1 entries differ in the opposite direction, so the table shows a metric-specific tradeoff rather than uniform separation.

The central idea is simple: generation proposes intervals, while a lightweight confidence head decides which intervals to rank, accept, or reject. On fixed candidates, that separation improves recall most when return budgets are tight.