Quote: Atlassian Co-Founder on Multi-Model AI Stacks
Mike Cannon-Brooks, co-founder of Atlassian, noted that enterprises are increasingly using multiple AI models from different providers. He stated, “Companies like Atlassian use multiple AI models (Anthropic, Gemini, OpenAI) and... technology budgets have historically grown and AI might drive further expansion rather than being zero-sum.” This suggests that demand for AI infrastructure, including data labeling, will grow as companies build complex, multi-provider AI systems.
- Reinforcement Learning from Human Feedback (RLHF) is a critical process for aligning large language models, where human evaluators rank or compare model outputs to create a preference dataset that trains a separate reward model. This method reduces the need for extensive manual data labeling by focusing annotation efforts on the most crucial aspects of model training. However, RLHF faces challenges with scalability, potential human bias in feedback, and ensuring the quality and consistency of feedback from annotators. - Constitutional AI, developed by Anthropic, offers a more scalable approach by training models with a "constitution" of principles, enabling the AI to critique and revise its own outputs without constant human feedback. This method, known as Reinforcement Learning from AI Feedback (RLAIF), aims to make the alignment process faster, more transparent, and less resource-intensive than traditional RLHF. - Evaluating agentic AI systems requires moving beyond traditional metrics like accuracy to assess multi-step reasoning, tool use, and the ability to recover from errors. Benchmarks such as AgentBench, WebArena, and GAIA are used to test these complex behaviors across various environments like web navigation and knowledge retrieval. - While synthetic data can be generated in vast quantities to scale training, it often lacks the nuance and ability to handle edge cases that human-annotated data provides. A hybrid approach is often most effective, where models trained primarily on synthetic data receive significant performance boosts from smaller amounts of high-quality, human-labeled data. - The fundraising climate for AI startups has seen a surge in investment, with AI companies raising a third of all venture capital in 2024. In the first three quarters of 2024, AI-focused climate tech ventures alone raised $6 billion. By 2025, AI captured nearly 50% of all global funding, with foundation model companies like OpenAI and Anthropic raising a significant portion of this capital. - Selling to technical buyers at AI labs, such as ML engineers, requires a deep understanding of their technical requirements, including integration with existing tech stacks and security compliance. Sales teams often need to partner with sales engineers and product teams from the beginning of the sales process to address the nitty-gritty details of implementation. Successful strategies involve providing detailed, data-driven content and offering demos or trials for independent exploration. - Data quality is a primary bottleneck in AI development, with most project failures rooted in poor data rather than flawed models. Inconsistent formats, labeling errors, and delays in data collection can force data science teams to spend significant time cleaning and reconciling data instead of building models. - The rise of AI is significantly impacting the future of work, with nearly 40% of global jobs exposed to AI-driven change. This shift is creating demand for new skills, particularly in IT, and while it creates anxiety about job displacement, it is also expected to create millions of new roles.
