Материал агрегирован из первоисточника:
arXiv — cs.AI
Читать полностью в arXiv — cs.AIМатериал собран из открытых источников. Если размещение затрагивает ваши права → Удалить информацию
Заголовок на английскомRight Order, Wrong Scale: Auditing LLM Judges for Occupational AI Measurement
LLM judges are increasingly used to assess whether AI outputs meet workplace requirements, but agreement on response rankings does not establish agreement on acceptance rates or occupational aggregates. We introduce O*NET-BENCH, an audit suite derived from an existing survey of…
Кратко от AI: Главные тезисы
- LLM judges are increasingly used to assess whether AI outputs meet workplace requirements, but agreement on response rankings does not establish agreement on acceptance rates or occupational aggregates
- We introduce O*NET-BENCH, an audit suite derived from an existing survey of…
Right Order, Wrong Scale: Auditing LLM Judges for Occupational AI Measurement
Похожие материалы
ENA Controlled LLM Workflow for Evidence-Grounded Diagnosis of Risk Score Differences
arXiv:2610.23962v1 Announce Type: new Abstract: We propose a controlled Large Language Model (LLM) workflow for explaining risk score differences using similar historical transactions. The workflow combines reference retrieval and contrastive SHapley Additive exPlanations (SHAP)…
ENBeyond Explicit Intent: Discovering and Shaping Artistic Intent through Human-AI Co-Creation
arXiv:2610.24010v1 Announce Type: new Abstract: Generative AI is becoming an important medium for artistic creation, yet existing interactions often assume that creators can specify intentions in advance and accurately communicate them through text prompts. Artistic intentions,…
GardenBench: Лёгкая ежедневная оценка возможностей LLM
arXiv:2610.23866v1 Тип объявления: новый Абстракт: Мы представляем GardenBench, бенчмарк с 11 наборами тестов, оценивающий возможности больших языковых моделей (LLM) по ряду задач. Систематически запрашивая LLM с различными лёгкими запросами, мы документируем…
ENLogicVal:A Logic-Formalized Benchmark for Constraint Satisfaction in Text-to-Image Generation
arXiv:2610.24051v1 Announce Type: new Abstract: Text-to-Image (T2I) generative models have made significant progress in producing high-fidelity images from natural-language prompts. Alongside this progress, numerous benchmarks have been developed to evaluate capabilities such as…