New evaluation methods required for agentic AI

- New benchmarks are emerging to evaluate agentic AI on multi-step, open-ended tasks that require complex reasoning and tool use. Frameworks like AgentBench and WebArena test agents in simulated environments such as operating systems, e-commerce websites, and content management systems. These evaluations focus on functional correctness and the ability to complete tasks, rather than just the accuracy of a single output. - The high cost and slow speed of collecting human feedback for Reinforcement Learning from Human Feedback (RLHF) has led to the development of Constitutional AI. This approach uses a set of principles, or a "constitution," to enable an AI model to critique and revise its own outputs, a process known as Reinforcement Learning from AI Feedback (RLAIF). This method has been shown to reduce feedback costs by over 100-fold, making alignment more scalable. - Synthetic data is increasingly used to train and test AI agents, especially when real-world data is scarce, sensitive, or expensive to obtain. By generating artificial datasets that mimic the statistical properties of real data, developers can cover a wider range of scenarios, including rare edge cases, without compromising user privacy. This is particularly useful for training agents on specialized tasks, like using a new command-line interface, where sufficient real-world data may not exist. - Data labeling for agentic AI is evolving from a manual process to automated workflows driven by specialized AI agents. These "agentic data workflows" use different agents for tasks like initial pre-labeling, quality assurance, and routing complex data to human experts. This approach can lead to significant reductions in manual labeling time and costs while improving data consistency. - Production-level evaluation of AI agents often involves a combination of public benchmarks, internal task suites, and manual review of the agent's decision-making process, known as its "trajectory". This multi-faceted approach is necessary because, unlike traditional models, agents can take multiple valid paths to achieve a goal, and their intermediate steps are as important as the final result. - Frameworks for building agentic AI systems, such as LangChain, AutoGPT, and Microsoft's AutoGen, offer different approaches to orchestrating AI agents. LangChain provides a modular structure for creating complex workflows, while AutoGPT allows an agent to autonomously break down and execute tasks to achieve a high-level goal. - Aligning agentic AI requires more than just feedback on the final output; it also involves supervising the process by which a model arrives at an answer. This "process supervision" is a component of newer alignment techniques that move beyond simple RLHF to create more robust and reliable systems. - Multi-agent systems, where specialized agents collaborate to achieve a common goal, are becoming a common design pattern for complex AI workflows. This approach allows for a separation of responsibilities, such as having distinct agents for planning, research, execution, and verification, which can improve the reliability and observability of the system.