用于推理大型语言模型的强化学习的推出效率:分类法和未来方向
面向推理的强化学习使大型语言模型能够解决数学、编码和其他多步骤任务,但将很大一部分训练成本转移到了部署上,即为策略更新生成轨迹。高效的推出机制...
All published intelligence, sovereign stack research, ecosystem news, and AI foundation model reviews.
面向推理的强化学习使大型语言模型能够解决数学、编码和其他多步骤任务,但将很大一部分训练成本转移到了部署上,即为策略更新生成轨迹。高效的推出机制...
在线 A/B 实验是用户参与度的决策标准,但流量和读出时间限制了可以测试的对话式 AI 更改的数量。我们询问设计为无需治疗臂用户暴露即可计算的离线信号是否与……的结果一致。
企业环境中的结构化数据代理必须对复杂的数据环境进行推理,其相关证据分布在模式、关系、策略和重复业务角色中。现代代理系统通常通过可重用的降价风格来解决这个负担……
在语言模型中寻找回路通常意味着进行许多仔细的干预。我们尝试一些更简单的方法:将注意力视为来自一次前向传递的路由图,保留一小组指向答案的路由,并询问这些路由是否真的重要。他们…
语言模型中的对话失败通常被认为是记忆失败:上下文太长、摘要有损、忘记了约束。我们认为这忽略了一个更深层次的问题:在许多对话中,模型不会忘记,但它提交得太早了。一个暧昧的早期转折崩溃了……
社交代理最基本的决定(我应该对这篇文章做出反应吗?我应该联系谁?)并不纯粹是内容问题。正确的行动往往取决于人与人之间的潜在关系——纽带强度、互惠、相互联系——而不是……
仅当书面推理因果性地限制答案时,思想链 (CoT) 监控才有意义。我们引入基于连续的因果测试,这是一种消融补丁干预,它会扰乱一个推理步骤,截断链条,并迫使模型从……继续下去。
现代信息检索越来越多地采用嵌入模型和生成模型来处理复杂的查询。然而,当前的服务系统由于单独执行这些模型而面临吞吐量低和 GPU 利用率低的问题。粗粒度划分,...
基于失败的文本知识蒸馏旨在通过检查模型的任务错误来发现模型知识中的差距。提炼的知识可用于该模型(“源模型”)和其他模型的推理。然而,这种知识转移可能并不……
In Mixture-of-Experts language models, the router typically selects and weights experts based on the token's hidden state, utilizing limited contextual information. We propose Attention-Aware Routing (AAR), which augments the router with temporal and spectral features extracted…
Autonomous software generation (ASG) aims to turn human requirements into executable applications, but delivering these applications does not necessarily establish that their interacting components satisfy the specified behavioral requirements. We introduce GameASG-Bench, a…
Multimodal trajectory prediction improves behavioral coverage in end-to-end autonomous driving, but existing methods remain limited by sparse scene representations. Incomplete evidence leads to low-quality candidate generation and unreliable ranking among geometrically similar…
Sparse representation formulates the environment perception for the end-to-end driving system as a set of discrete elements like objects and lane lines. This formulation meets safety risks in crowded, occluded scenes dealing with unstructured obstacles, uncertain regions, and…
Post-training plays a pivotal role in enhancing the reasoning capabilities and task-specific expertise of large language models (LLMs). Despite recent advances in post-training methods, such as Group Relative Policy Optimization (GRPO), their practical deployment remains impeded…
Online agent deployments produce abundant execution traces, while task-specific verification and expert annotation are costly to scale. We study how to distill these traces into reusable feedback without post-hoc outcome labels, drawing on their evidence of local progress,…
Perceptual planning tasks require two key capabilities: accurately perceiving uncertain scenes and planning valid action sequences following logical rules. Conventional methods convert perception into discrete symbolic facts and then plan, discarding perceptual uncertainty and…
Agentic marketplaces are emerging where AI agents with varying capabilities autonomously complete specialized tasks for buyers. A major challenge of such marketplaces is that buyers cannot easily determine which agent will perform best on their tasks. Reported benchmark scores…
Air operations rely on complex rules, established procedures, and time-critical analysis under limited connectivity and strict security constraints. In such environments, analysts must combine written doctrine with images, often without access to external computing resources.…
Understanding and modeling human intelligence are parallel goals shared by artificial intelligence (AI) and cognitive science. As AI systems grow increasingly capable, in what ways do model responses resemble human responses, and where do they systematically diverge? The sheer…
Automated macro placement remains a fundamental challenge in VLSI physical design. Despite decades of research, existing approaches predominantly optimize hand-crafted proxy objectives, such as estimated wirelength, and typically produce placements through one-shot numerical…
Production LLM agents are evaluated repeatedly as they evolve, but full agent benchmarks are costly to rerun. We study efficient recurring evaluation for a production analytics agent serving tens of thousands of monthly active users and report first-hand deployment experience.…
Organizations deploying agentic artificial intelligence must determine more than whether a model is trustworthy; they must establish what to validate, control, and observe for a use case to deliver its intended outcome while meeting applicable obligations. This paper proposes…
The Abstraction and Reasoning Corpus and related benchmarks evaluate whether AI models can solve novel reasoning tasks, but often leave unclear whether success reflects inference of the intended underlying rule or reliance on shortcuts. We address this gap by studying test-time…
Off-target protein binding is a major source of adverse effects for small-molecule drugs, yet most structure-based molecular design methods focus on generating selective compounds de novo rather than improving the selectivity of existing, well- characterized drugs. We introduce…
Self-play methods that co-train a single language model as both coder and test author promise to move code-generation RL beyond fixed test suites, but they suffer from two coupled pathologies: permissiveness collapse, where pass-rate rewards are maximised by trivial,…
Large Language Model (LLM) agents are increasingly being explored for chip design, but most existing approaches operate directly at RTL. We ask whether agents can design better chips by leveraging higher-level abstractions. We compare Direct RTL Design, Agent-based HLS Design,…
Before patients can use AI-assisted psychiatric intake systems, health systems need practical ways to routinely evaluate these tools against their clinical standards for quality assurance. Because clinicians may use different intake styles, evaluation for this task must (1)…
Replacing attention in a pretrained language model is a compatibility problem: a plausible substitute may alter representations expected by later layers. TinyCeNN-LM introduces a \emph{quality-gated post-training conversion} framework using CeNN-inspired cellular-recurrent…
Fine-tuning has emerged as a widely adopted approach for adapting LLMs to a variety of downstream tasks. However, how it reshapes their internal mechanisms remains poorly understood. To address this, we investigate how fine-tuning alters internal representations in LLMs,…