arXiv:2611.08613v1 Announce Type: new Abstract: Normative competence, the ability to infer socially enforced expectations from interaction and adjust behavior accordingly, is essential for agents operating in unfamiliar human communities. Yet existing evaluations largely test whether agents can recognize or apply norms they may already know, rather than whether they can acquire previously unknown norms from social interaction. We introduce NormGym, an interactive benchmark for normative competence in which a newcomer must infer a hidden, arbitrary norm while pursuing its own task in a multi-agent community. Success requires distinguishing what is possible from what is permitted, inferring the norm from observed behavior and sanctions, and applying it in new situations. NormGym contains diverse norms and distinct environments, allowing systematic evaluation of current agents, including large language model (LLM) agents. We find that normative competence remains challenging: the strongest evaluated LLM achieves a joint task-and-compliance success rate of 55.3%. Our analysis reveals that models can sometimes revise their behavior following sanctions for their own actions, but struggle to use other agents' behavior and the sanctions they receive to infer the underlying norm. We further find that fine-tuning models may be insufficient to improve normative competence. These results suggest that current models lack a robust, transferable capability for acquiring normative knowledge through social interaction.
NormGym: Evaluating Normative Competence through Social Interaction
arXiv:2611.08613v1 Announce Type: new Abstract: Normative competence, the ability to infer socially enforced expectations from interaction and adjust behavior accordingly, is essential for agents operating in unfamiliar human communities. Yet existing evaluations largely test…
AI Summary & Key Takeaways
- arXiv:2611.08613v1 Announce Type: new Abstract: Normative competence, the ability to infer socially enforced expectations from interaction and adjust behavior accordingly, is essential for agents operating in unfamiliar human communities
- Yet existing evaluations largely test…
NormGym: Evaluating Normative Competence through Social Interaction
Related stories
Ripple-65B Preview: Model for Agentic Coding
🌟 Ripple-65B Preview: Model for Agentic Coding. Recursive Intelligence has released an open preview of a 65‑billion‑parameter agentic programming model focused on response speed. The company was founded by Anna Goldie and Azalia Mirhoseini, former Google researchers…
FLEX Platform Teaches Robots to Understand Hidden Human Intentions
MIT Unveils FLEX Platform: Robots Learn to Understand Hidden Human Intentions and Generalize Commands Based on Language Model.
Top AI News of the Week
📌 Top AI News of the Week 🔊 ElevenLabs Sound Effects v3 — generates sounds from descriptions: from footsteps and explosions to ambient backgrounds. 🎨 OpenAI Images 2.6 — creates GIF animations, generates images faster, and edits existing images more precisely. 🎬 Gemini Omni 1.2 Flash —…
Toward LLM Self-Play Under Data Scarcity: Effectiveness, Generalizability, and Trustworthiness
arXiv:2611.08908v1 Announce Type: new Abstract: The rapid development of large language models (LLMs) has been driven largely by scaling model size and human-curated training data, yet this paradigm is increasingly constrained by data scarcity and diminishing returns from human…