On the Clock: Towards Punctual and Productive Time-Budgeted AI Agents

AI summary

Overview

• Research area: Time-budgeted LLM agents and resource-aware AI.
• Technical level: Intermediate.
• Scope: The paper tests whether agents can both meet wall-clock deadlines and use extra time to improve their results, comparing harness support with reinforcement learning.

What This Paper Is About

The researchers ask whether an AI agent can follow a time limit and adjust its approach when given more or less time. They find that simply telling an agent its budget is not enough: agents struggle to track elapsed time and anticipate how long actions will take. Providing clock information or training can improve deadline adherence, but agents still struggle to turn extra time into better task performance.

Key Contributions

1. Separates two abilities: The paper distinguishes budget adherence—finishing on time—from productive use—using additional time to improve the result.
2. Tests clock and deadline tools: It evaluates harness features that show elapsed or remaining time and features that limit or block actions that would exceed the deadline.
3. Tests budget-aware reinforcement learning: It applies GRPO training with rewards that consider task results and time use in MLE-Bench Lite and Zork I.
4. Identifies a failure mode: Agents can learn when to stop without learning a different, more productive strategy for a longer budget. In Zork I, the trained policy can spend extra time repeating unproductive actions.

Main Findings

• Prompt-only budgets are unreliable. On MLE-Bench, Qwen3.6-27B used an average of 161% of a 15-minute budget, and only 15% of runs finished on time. In Zork I, agents also commonly overshot budgets when the prompt was their only timing cue.
• Harness clocks improve adherence. In MLE-Bench, adding timing information reduced average budget use from 161.4% to 99.7%, without a measurable change in quality. In the paper’s 15-minute comparison, the on-time rate rose from 15% with prompting alone to 62% with clocks.
• Enforcement improves deadline control further. In the same MLE-Bench comparison, command clamping raised the on-time rate to 74%, while refusing overlong commands raised it to 88%. Enforcement has trade-offs: cutting off or blocking work can affect quality or valid-submission rates.
• RL can teach stopping behavior, but not reliably better time allocation. GRPO-trained agents showed strong budget adherence on Zork I, including generalization to held-out budgets. On MLE-Bench, training improved adherence but did not improve task performance over the harness condition.
• More time often did not mean better results. In Zork I, policies trained on multiple budgets tended to repeat a strategy learned for the shortest budget, then fill extra time with actions that did not advance the game. On MLE-Bench, extra time also did not consistently lead to improved submissions.

Methodology in Plain English

The researchers studied two different kinds of tasks. In MLE-Bench Lite, agents had to work through machine-learning competitions by examining data, writing and running code, and producing submissions. The authors selected five competitions where additional time could plausibly support a better approach. In Jericho’s Zork I, agents explored a text-based game, where more time could allow more moves and puzzle-solving.

They compared several ways to give agents time budgets. Some runs stated the budget only in the prompt. Other runs added clock information through the harness, either when the agent requested it or automatically after each turn. They also tested enforcement features that shortened or blocked commands likely to run past the deadline. Separately, they trained policies with GRPO using rewards that accounted for task score and time use.

The evaluation considered both whether an agent stayed within its budget and whether performance improved as the budget grew. That distinction exposed a key problem: an agent may use nearly all its allotted time without spending that time on useful work.

Why This Matters

Research impact: The paper argues that deadline compliance and effective planning under a deadline are different capabilities. Improving time awareness may solve much of the first problem, but it does not automatically teach an agent how to use extra time well.

Potential applications:

• Coding and machine-learning assistants: Respect execution deadlines while deciding whether to run more experiments.
• Business workflow agents: Finish tasks within time limits set by users or coordinated systems.
• Robotics and interactive systems: Account for variable action durations when operating under real-time constraints.
• Research and data-analysis agents: Use a larger time allowance for meaningful additional investigation rather than redundant work.

Industry relevance: A harness can provide practical deadline controls without relying on an agent to estimate time accurately on its own. However, systems that need both punctuality and better results from longer runtimes will also need ways to encourage useful exploration and discourage repetitive or invalid actions.

Future Directions

• Develop rewards that credit novelty, state changes, or useful exploration rather than elapsed time alone.
• Vary game states and task environments during training so that memorizing a fixed solution is less effective.
• Distinguish repeated attempts from genuinely different strategies, especially in tasks where tool actions have variable durations.
• Evaluate across more models, environments, and tasks to test whether these findings hold beyond the settings studied.

Target Audience

This paper is most useful for researchers working on LLM agents, reinforcement learning, agent evaluation, and time-aware inference. It should also interest engineers building tool-using AI systems that need to meet deadlines, particularly those deciding how much timing feedback and enforcement to place in the harness.

Authors’ abstract

We study whether small LLM agents can operate effectively under explicit wall-clock time budgets by both respecting the allocated runtime and using available time productively. We evaluate Qwen3.6-27B on five competitions from MLE-Bench Lite and Qwen3-4B on Zork I (Jericho), two agentic benchmarks where additional computational time can meaningfully improve performance. In the simplest setting, where the budget is stated only in the prompt, agents fail to translate the stated budget into controlled use of time. These failures arise from gaps in time awareness, since the harness provides no timing feedback, but also because they cannot reliably anticipate the duration of actions, and do not have a learned mapping from available time to an appropriate strategy. We investigate two complementary classes of interventions: harness-based mechanisms that expose timing information and enforce deadlines, and reinforcement learning with budget-aware rewards. Injecting timing information through the harness substantially improves budget adherence for Qwen3.6-27B without measurable loss in performance, while enforcement hooks tighten adherence further. RL with GRPO achieves near-perfect budget adherence on Zork I and generalizes to held-out budgets not seen during training, but does not improve task performance over the untrained harness on MLE-Bench. Once agents are made to respect the budget, they still fail to use additional time to improve task performance. RL-trained policies learn when to stop but often fill extra time with repeated actions, and GRPO training on multiple budgets tends to collapse toward the strategy learned for the shortest budget. Our results reveal a gap between time adherence and productive time allocation, which remains a central challenge for budget-conditioned agents.