AGI Bottleneck is Human, Not Compute

- The concept of "human feedback" as a core component of advancing AI is operationalized through techniques like Reinforcement Learning from Human Feedback (RLHF). In this process, human evaluators rank different outputs from a model, creating a preference dataset that is then used to train a separate "reward model". This reward model is then used to further train the AI, aligning it more closely with human goals and values. - The "scaling laws" for neural networks, empirically observed and formalized by labs like OpenAI and DeepMind, predict that model performance improves in a power-law fashion with increases in model size, dataset size, or compute power. This predictability allows researchers to forecast the performance of future, larger models and make informed decisions about resource allocation. - While model training costs are substantial and growing—projected to reach a billion dollars for a single training run—inference costs can quickly surpass them for popular AI services. This creates a dual-sided cost optimization challenge for companies, balancing the large, upfront capital expenditure of training with the ongoing operational expenditure of inference. - The market for AI chips is experiencing explosive growth, with revenue from AI semiconductors expected to increase by 33% in 2024 to $71 billion. This growth is not evenly distributed; companies focused on AI chips have seen significant valuation surges, while traditional semiconductor sectors have declined, signaling a new, AI-driven "supercycle". - In response to supply chain imbalances and the high cost of GPUs from market leaders like Nvidia, major hyperscalers including Google, Amazon, and Microsoft are increasingly designing their own custom silicon (ASICs) for AI workloads. This "build vs. buy" decision is driven by a need for greater flexibility, cost control, and a resilient supply chain for their data centers. - Venture capital investment is shifting from a primary focus on software to "deep tech" and hardware, particularly for AI infrastructure. In the fourth quarter of 2024 alone, AI hardware startups raised over $3 billion, with significant rounds for companies developing custom processors and interconnect technologies to address bandwidth bottlenecks in data centers. - A new ecosystem of AI-powered Go-To-Market (GTM) tools is emerging to automate and enhance sales and marketing workflows. Platforms like Demandbase, Outreach, and Regie.ai use AI for tasks such as account-based marketing, sales prospecting, and generating personalized outreach content. - The role of the human is shifting from building AI systems to reviewing and validating their outputs. At OpenAI, for instance, designers now often write and ship their own code with AI assistance, blurring the traditional lines between design and engineering and emphasizing the need for faster review processes.