The impact of code review coverage and code review participation on software quality: a case study of the qt, VTK, and ITK projects

Authors: Shane McIntosh Yasutaka Kamei Bram Adams Ahmed E. Hassan

Venue: MSR   11th Working Conference on Mining Software Repositories, pp. 192–201, 2011

Year: 2011

Abstract: Software code review, i.e., the practice of having third-party team members critique changes to a software system, is a well-established best practice in both open source and proprietary software domains. Prior work has shown that the formal code inspections of the past tend to improve the quality of software delivered by students and small teams. However, the formal code inspection process mandates strict review criteria (e.g., in-person meetings and reviewer checklists) to ensure a base level of review quality, while the modern, lightweight code reviewing process does not. Although recent work explores the modern code review process qualitatively, little research quantitatively explores the relationship between properties of the modern code review process and software quality. Hence, in this paper, we study the relationship between software quality and: (1) code review coverage, i.e., the proportion of changes that have been code reviewed, and (2) code review participation, i.e., the degree of reviewer involvement in the code review process. Through a case study of the Qt, VTK, and ITK projects, we find that both code review coverage and participation share a significant link with software quality. Low code review coverage and participation are estimated to produce components with up to two and five additional post-release defects respectively. Our results empirically confirm the intuition that poorly reviewed code has a negative impact on software quality in large systems using modern reviewing tools.

BibTeX:

@inproceedings{shanemcintosh2011tiocrcacrposqacsotqvaip,
    author = "Shane McIntosh and Yasutaka Kamei and Bram Adams and Ahmed E. Hassan",
    title = "The impact of code review coverage and code review participation on software quality: a case study of the qt, VTK, and ITK projects",
    year = "2011",
    pages = "192–201",
    booktitle = "Proceedings of the 11th Working Conference on Mining Software Repositories"
}

Plain Text:

Shane McIntosh, Yasutaka Kamei, Bram Adams, and Ahmed E. Hassan, "The impact of code review coverage and code review participation on software quality: a case study of the qt, VTK, and ITK projects," 11th Working Conference on Mining Software Repositories, pp. 192–201