AI Works, Pull Requests Don’t: Revolutionizing the Software Development Lifecycle
At the inaugural QCon AI New York 2025, Michael Webster, Principal Engineer at CircleCI, delivered a thought-provoking presentation titled "AI Works, Pull Requests Don’t: How AI Is Breaking the SDLC and What To Do About It." This session highlighted the transformative role of AI in the Software Development Lifecycle (SDLC) and how it reshapes Continuous Integration/Continuous Delivery (CI/CD) processes.
The Current State of AI in Development
Webster kicked off his presentation by providing an insightful AI timeline, illustrating how we have arrived at this pivotal moment in software development. The integration of AI into development workflows is burgeoning, leading to significant advancements in code generation capabilities. As organizations embrace these technologies, understanding their implications on productivity and quality becomes essential.
The Impact of AI on Code Generation
Recent studies shed light on the dual-edged nature of AI-assisted coding tools. One significant research initiative led by Hao He, a Ph.D. student in Software Engineering at Carnegie Mellon University, analyzed the long-term effects of incorporating Cursor, a notable AI coding assistant. By comparing over 2,000 open-source software projects, it was found that while the adoption of Cursor initially bolstered development velocity and software quality, these improvements were fleeting—lasting merely a month before a decline in development speed was noted.
Similarly, the State of AI-Assisted Software Development report by DORA concluded that while AI tools enhance development velocity, they also introduce levels of instability, prompting developers to reassess their usage strategies.
The Challenge of Code Reviews
As projects continue to scale, the complexity of code reviews grows exponentially. The State of Code Review report by Graphite revealed some startling facts: Organizations often face scenarios where code additions can outnumber deletions by as much as 25 times. For smaller organizations, merging a pull request typically takes about four hours, compared to about 13 hours for larger teams. This discrepancy largely stems from smaller organizations often skipping formal code reviews altogether.
More Read
The Role of AI in Addressing SDLC Challenges
Webster explored how CircleCI leverages AI in their SDLC to enhance productivity. The integration of AI responses led to an initial three-to-five times increase in development velocity. However, after one month, teams began experiencing an accumulation of technical debt, raising questions about long-term sustainability.
Understanding Queuing Theory
To grasp the effects of workload on development processes, Webster introduced Queuing Theory, the mathematical study of waiting lines. Utilizing this framework, he illustrated the principle that delays occur when the arrival rate exceeds the processing rate. This mathematical model serves as a critical lens through which teams can analyze their workflows and identify bottlenecks.
Optimizing Testing with Test Impact Analysis (TIA)
Traditional testing methods often involve executing a complete suite of tests, which can be time-consuming. Webster emphasized a more efficient approach: Test Impact Analysis (TIA). This strategy allows teams to run only the tests impacted by recent code changes, thereby drastically improving CI/CD throughput. For example, in a recent CircleCI project involving 7,500 tests, the time needed for testing was slashed from 30 minutes to just 1.5 minutes. This agile approach can also be applied to code reviews, allowing developers to focus their efforts on critical changes without unnecessary scrutiny on less impactful code.
Introducing Chunk: AI-Powered Code Validation
Wrapping up his presentation, Webster unveiled Chunk, an AI agent designed by CircleCI to validate code at exceptional speeds. Chunk operates on a clear process: it validates code first and integrates into existing pipelines and environments. This tool also keeps software production-ready by addressing flaky tests and refining test coverage as the development progresses.
With AI’s promise of streamlining both coding and review processes, Chunk represents a significant step toward an optimized and efficient SDLC, which can keep pace with the demands of modern software development.
As the landscape of software development continues to evolve, especially with advances in AI, organizations must adapt and refine their workflows to harness these new tools effectively. The insights shared by Michael Webster at QCon AI New York 2025 offer a glimpse into the future of software engineering—a future where AI transforms every stage of the development lifecycle.
Inspired by: Source
