Jeff Dean: The 1% Rule for Building in AI

Jeff Dean initially predicted in May 2023 that AI had reached the level of a junior engineer. He believes this assessment remains accurate, noting significant advancements in agent-based systems capable of longer coding tasks and problem solving. Dean underestimated the speed at which these complex task capabilities were developing, particularly outside of just coding applications.

Dean predicts that a major trend in the near future will be the automation of ML systems themselves. This involves creating systems capable of improving their own capabilities through automated experimentation, breaking down problems into sub-processes, and iteratively refining results. He believes this approach extends beyond ML to other scientific and engineering fields with measurable objectives.

In 2001, Jeff Dean and Sanjay Ghemawat realized that the entire Google search index could fit into RAM. They rapidly implemented this change, shifting from hard drive-based searching to in-memory processing, which significantly improved search speed and was a pivotal moment for Google's success. This highlights the impact of optimizing data storage and access.

Dean anticipates increased development in high-performance, low-energy inference hardware systems. He believes this is driven by the recognition that inference is crucial for making agent-based AI accessible to a wider audience and reducing latency. Specialization of hardware, rather than general-purpose devices like GPUs or TPUs, will be key to achieving these goals.

Jeff Dean notes a significant shift in AI progress. While initially focused on larger models with more parameters and data, the current emphasis is expanding to encompass everything *around* the model itself. This includes tools like retrieval mechanisms, memory systems, agent capabilities, and what's now termed 'context engineering,' signifying a broader system-level approach.

Dean emphasizes that providing clear context to AI models is more beneficial than the information contained within their training data. Unlike the vast, unstructured 'soup' of tokens used during training, direct contextual input allows models to understand and apply knowledge more effectively for specific problems or use cases. This clarity facilitates better problem-solving capabilities.

Unlike earlier stages of AI development that required substantial resources (GPUs, data), context engineering is now accessible to a wider audience. Individuals can begin experimenting with tools like Gemini using their APIs and build custom retrieval systems and tool call sequences, democratizing the ability to contribute to advanced AI solutions.

Dean describes a practical example of context engineering involving the creation of skills for an AI model to automate microbenchmark measurement and code optimization. This system allows the model to iteratively measure performance, suggest improvements, re-run benchmarks, and refine its approach—demonstrating a self-improving workflow for low-level library development.

Jeff Dean highlights the success of specialized AI models like AlphaFold, which focused on protein folding. These models demonstrate that highly accurate and effective solutions can be achieved by concentrating efforts on specific domains such as material science or chip design. While not general-purpose, these niche models enable tackling previously difficult tasks within their area of expertise.

Dean emphasizes the crucial role of clear and detailed specifications when working with AI agents. He explains that agents perform better when given precise instructions, reducing the need for them to infer intentions which can lead to unexpected outcomes. Drawing parallels to traditional software development, he argues that specifying desired functionality remains paramount even with agent-based systems.

As AI agents automate coding tasks, Dean posits that the most valuable skill will be discerning which problems are worth pursuing. He compares this to a researcher’s challenge of selecting impactful research questions and suggests that models may not yet possess this 'taste' effectively, highlighting the need for human guidance in directing AI-assisted computation.

Dean suggests that developing a sense of 'taste' in AI problem selection comes from experience and reflection. He recommends documenting potential areas of importance, selecting one to pursue initially, and then revisiting the list later to evaluate which problems proved significant or were addressed by others. This iterative process fosters a better understanding of what constitutes valuable work.

Jeff Dean discusses the potential of creating faster validation models, potentially learned models, to dramatically accelerate scientific discovery. He explains that these models could be 300,000 times faster than running full-scale simulations, enabling scientists to screen ten million possibilities while they are at lunch instead of a six-month endeavor. This shift would fundamentally change how science is conducted by allowing for rapid iteration and experimentation.

Dean recounts the rejection of a paper on distillation, a technique for creating smaller, more efficient models from larger ones. While acknowledging that program committees sometimes miss the broader implications of research, he emphasizes that the rejected paper ultimately became widely adopted in industry and is now used to create Google's Flash models. This illustrates the importance of perseverance even when facing rejection.

Dean mentions that Google’s Flash models used in Gemini are partly a result of distillation techniques described in the rejected paper. These models demonstrate impressive performance relative to their size and speed, showcasing the practical value of research even when initially overlooked by reviewers.

Reflecting on what a younger version of himself would do, Dean highlights the significance of pursuing work that aligns with personal values and has a positive impact on the world. He emphasizes the importance of collaborating with enjoyable colleagues to collectively solve problems and offer valuable services to various groups, whether it's biochemists, programmers, or consumers.