As AWS Machine Learning reports, universal agents handle a wide range of tasks, but they need to follow the procedures that govern the business: compliance checks, document workflows, escalation policies, and engineering standards. Coding all of this into a single system request or application logic becomes difficult to maintain and update. Skills provide a modular alternative. A skill is a reusable set of instructions, typically stored in a SKILL.md file, that trains an agent on a specific task in a specific area, such as editing a contract, reconciling an account, or meeting team standards on merge requests. Because skills follow the open Agent Skills standard, they are portable between compatible systems, and the agent loads only the skill it needs at runtime, rather than carrying all the procedures in its main instructions. A skill packages one or more tools with the context an agent needs to use them correctly: instructions, tool bindings, knowledge, workflow, and protective measures. This modular approach helps teams develop faster
cialize agents, reuse proven procedures between agents and workflows, maintain consistent behavior, and update domain-specific recommendations without retraining the underlying model or rewriting agent logic. However, this composability of skills introduces two failure modes that can be missed by general output quality metrics: an agent calls a skill that is inappropriate for the task, and an agent calls a correct skill but misses or only partially follows its instructions. The assessment, therefore, cannot rely solely on the final answer. To make these failures measurable, the Strands Evals SDK and Amazon Bedrock AgentCore Evaluations are adding skill-based evaluators. Skill selection accuracy determines whether each skill evoked was an appropriate choice for the task. It returns a binary result for each skill invoked. Following Skill Instructions measures how fully the agent followed the instructions of the invoked skill and returns a five-level evidence-based score for each prescribed step. In this post, you will learn how to evaluate skill selection and following instructions from a recorded trajectory.
in Strands Evals, add deterministic routing checks to the test suite, evaluate skill behavior from OpenTelemetry traces using AgentCore Evaluations, and interpret skill results to select the correct patch via the AgentCore CLI. The agent receives a task and a catalog of skills, selects a skill, loads it and acts. The launch is recorded as a trajectory in Strands Evals or as an OpenTelemetry trace in your observability layer. This entry can now be used for all three skill evaluators. Skill Selection Accuracy checks whether each skill called is appropriate for the task and whether the correct skill was called by the agent. Additionally, Skill Invoked is a deterministic check to see if the specified skill was successfully loaded. Skill behavior assessment requires different fixes depending on the type of failure. Inappropriate selections often indicate overlapping or ambiguous skill descriptions, while incomplete following of instructions may require clearer steps, a different skill structure, or a more powerful agent model. So skills make agents cheap to specialize, but a plausible final answer doesn't prove
that the agent chose or followed the correct procedure. Accuracy in skill selection and following skill instructions separates these failure modes and returns evidence for each skill invoked. The material was prepared by a neural network based on the original publication. AI may make mistakes in details (numbers, names, dates, quotes) - for important information, check with the original source. Claude Opus 5.5 is now available on Amazon Bedrock GPT-6 Sol and GPT-6 Luna is now available on Amazon Bedrock Learn how parallelism sweeps help optimize generative AI on Amazon SageMaker. AWS Machine Learning shows how to combine data automation with Agentforce. AI infrastructure may be more valuable than the AI applications themselves.
