“If debugging is the process of removing software bugs, then programming must be the process of putting them in,” as said by Edsger Dijkstra, a Dutch systems scientist, programmer, software engineer, science essayist, and pioneer in computing science.
Every software developer’s Eureka! moment is to build a code which is bug free. While a bug-free code is an exaggeration since no software is developed as such. But to develop a stable software is also crucial. Converging to this as quickly as possible is the challenge every company and developers face. According to a recent analysis, bug fixing consumes around 70% of a developers' lifecycle. To deliver great products and features to customers, most companies dedicate ample time in testing and bug fixes. Automation in testing has been in practice to efficiently find bugs which otherwise is one of the most time-consuming tasks.
In recent years, Artificial intelligence has grown from futuristic hope to everyday existence. AI and ML applications are incorporated in most of the software domains. Automation in testing is accelerated with AI optimizing the SDLC process and reducing time spent on mundane tasks. But why stop at just testing? Why not extend it to fixing the bugs as well. As long as we have a similar set of test suites to run, there can be a similar set of bug fixes in particularly classified areas.
How AI/ML helps in automating bug fixing
Automated bug fix, with the help of AI and ML reduces developers' job of performing an effort-prone and time-consuming task of locating and fixing certain trivial bugs. This can be made possible by using "predefined fixes" based on the previous set of bug fixes.
ML models can be developed to review the current test status, recent code fixes, code coverage, data patterns, and other metrics, to decide which test sets to run. And when the system detects the bugs, the AI system can automatically generate fixes for specific bugs that the system is trained on. There are a plethora of open-source projects with an enormous number of bug fixes available in code repositories. This library of software development records can be used to learn how to fix common programming bugs.
To explore its potential, an empirical study can be performed to assess the feasibility of using Neural Machine Translation techniques to learn bug-fixing patches for real defects.
First, millions of bug-fixes from the change history of projects hosted on GitHub can be extracted, to fetch meaningful examples of such bug-fixes. Next, the defective code and corresponding fixed code are to be extracted, and use them to train an Encoder-Decoder model to be able to translate defective code into its fixed version.
Facebook's Sapfix, AI hybrid tool automatically generates fixes for specific bugs
Facebook’s SapFix, a new AI hybrid tool created by Facebook engineers, saves time on debugging, and speeds up the process of developing a new software. SapFix can automatically generate fixes for specific bugs, and then propose them to engineers for approval and deployment to production. To deal with high-priority bugs, SapFix creates patches that either fully or partially reverts the code submission that caused them. For more complex crashes, the system generates patches by picking from the pre-defined collection of fixes. These templates were automatically created from those developed by human engineers, based on past fixes.
If previously used human-designed templates do not fit, SapFix will attempt a mutation-based fix, whereby it performs small code modifications to the abstract syntax tree (AST) of the crash-causing statement, making adjustments to the patch until a potential solution is found.
SapFix improves upon previous auto-fix technology by using more powerful techniques to learn fixing patterns from past code changes. It uses a more powerful clustering algorithm and analyzes the scenario around the particular lines of defective code to find more appropriate fixes. SapFix is still under research and development and it has been reported that the tool has successfully generated patches that have been accepted by human reviewers and pushed to production.
The automation process can be further improved by incorporating reinforcement learning. Under reinforcement learning the model learns itself which fixes can eliminate the bug without affecting performance and speed. With an enormous amount of data available, RL agents can be trained to make judgements based on available fix options. Rewards can be assigned to the agent based on execution results and performance.
To sum it up it's really exciting to see how existing and upcoming tools apply ML/AI for automated bug-fixing. Maybe the future of Software vending machines is not too far where we just input our requirements and fully functional software will be delivered.
.png){kind=icon}