Verifiable Environments for AI in Biology — Kenny Workman, LatchBio

The speaker highlights a log-linear curve demonstrating the rapid growth of data generated in biology over time. This growth is primarily driven by experimental classes like single-cell biology, spatial biology, and proteomics, with single-cell experiments yielding up to six terabytes per run and spatial runs generating seven terabytes. He emphasizes that this rate of data generation surpasses most other scientific domains, exceeding the output of particle collider machines.

Kenny Workman describes a common pattern in modern biological research. Researchers select a model organism, generate data from it, process that data, interpret the results within existing literature, and then formulate a claim. This iterative process resembles a 'panning experiment,' where researchers search for signals amidst significant noise.

After initially operating as a data tool vendor for biotech and pharma, LatchBio shifted its focus to providing software tools and infrastructure to those building these experimental kits. This transition was driven by the emergence of agent prototypes that began demonstrating functionality around the summer prior, marking a significant step towards integrating AI agents into biological research workflows.

Drawing parallels to advancements in software engineering, Kenny Workman suggests that agentic biology will likely follow a similar trajectory. As individual agents improve, they'll be orchestrated into teams and eventually 'teams of teams,' enabling scientists to tackle increasingly complex research questions through coordinated AI-driven efforts.

LatchBio has released its first benchmark focused on preclinical pharmacology for small molecules. This initiative aims to systematically evaluate the drug discovery process, breaking it down into stages from initial discovery through development and translation. They are stratifying these phases by therapeutic area and experimental type to provide a more granular assessment of performance.

Kenny Workman highlighted the current limitations of large language models like Fable when applied to biological questions. Specifically, he noted that these models often fail to provide accurate answers to even basic inquiries about topics such as mitochondria. This observation underscores a significant challenge in applying AI to complex scientific domains and emphasizes the need for specialized training data and evaluation metrics.

LatchBio employs both routine tasks (simulating standard scientist requests) and 'red team' tasks (designed to identify vulnerabilities or malicious potential). A concerning finding was that routine tasks are used significantly more frequently than red team tasks, suggesting a bias towards simpler applications and potentially overlooking critical security risks. An example given was the request to clone a gene into bacteria labeled as GFP but actually containing a toxin or viral bootstrapping code.

LatchBio has established a 'flywheel' system where they create benchmarks, encourage labs to compete on them, and then use the resulting model improvements to enhance their own products. This iterative process drives significant growth and has led to aggressive hiring across both engineering and scientific roles, reflecting the company’s rapid expansion in the field of verifiable AI environments for biology.