fighting slop with slop — Vaibhav Gupta, Boundary

Vaibhav introduces the concept of 'slop,' defining it as any code that isn’t consistently reviewed. He emphasizes that a codebase will inevitably contain some level of slop, and the goal is to manage it rather than eliminate it entirely. This philosophy stems from their experience building a programming language where consistency and reliability are paramount, requiring them to actively combat 'slop' in their processes.

To address the issue of engineers using disparate AI tools, Boundary implemented an 'architecture.md' file. This document serves as a central repository for agreed-upon standards and guidelines, ensuring that all agents can understand and work with the same foundational elements, particularly layers within their compiler. The file is intentionally kept small and stable to minimize disruption.

Recognizing that code can be 'slop' but design documentation cannot, Boundary prioritized the quality of written design documents. This led to the creation of a dedicated design doc tool with versioning and commenting features, coupled with a Slack integration to promote visibility and engagement. The resulting channel became highly active, demonstrating the power of shared knowledge and accountability.

To ensure code quality and catch potential issues, Boundary implemented a system where agents automatically generate BAML programs. These programs are then used to test the codebase from scratch, allowing them to analyze tool usage and identify any deviations or inconsistencies in the development process. This continuous validation loop helps maintain stability and prevent regressions.

The speaker demonstrates how the system automatically infers error types, eliminating guesswork for developers. This allows the compiler to provide exhaustive guarantees regarding error handling – proving whether an error has been handled or not. This capability enables the design of APIs that guarantee they never throw errors by leveraging these inferred types and compiler-verified handling.

The speaker proposes a future where Machine Language (ML) functions are directly accessible from any programming language, including Python, TypeScript, Rust, Go, Ruby, and Java. This seamless integration allows for the passing of lambdas, generics, and closures across these languages, streamlining development workflows and ensuring type safety without requiring engineers to manually manage complexities.

The speaker argues that incredibly complex systems can be built without traditional processes like code reviews. This is enabled by parallel work streams and the freedom to utilize AI tools as desired, but crucially depends on a thoughtful approach to how engineering teams interact with underlying systems. This shift allows for greater flexibility and speed in software development.

The speaker emphasizes that engineering processes must evolve to match the pace of modern software development, particularly when aiming for 'agent speed.' He draws a parallel to the initial resistance towards CI/CD pipelines, highlighting how short-term slowdowns are outweighed by long-term gains in efficiency. This underscores the importance of proactively addressing and mitigating slop within codebases.