Authors: Farida El Zanaty Toshiki Hirao Shane McIntosh Akinori Ihara Kenichi Matsumoto
Venue: ESEM 12th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, pp. 11:1–11:10, 2018
Year: 2018
Abstract: Background: Code review is a well-established software quality practice where developers critique each others' changes. A shift towards automated detection of low-level issues (e.g., integration with linters) has, in theory, freed reviewers up to focus on higher level issues, such as software design. Yet in practice, little is known about the extent to which design is discussed during code review. Aim: To bridge this gap, in this paper, we set out to study the frequency and nature of design discussions in code reviews. Method: We perform an empirical study on the code reviews of the OpenStack Nova (provisioning management) and Neutron (networking abstraction) projects. We manually classify 2,817 review comments from a randomly selected sample of 220 code reviews. We then train and evaluate classifiers to automatically label review comments as design related or not. Finally, we apply the classifiers to a larger sample of 2,506,308 review comments to study the characteristics of reviews that include design discussions. Results: Our manual analysis indicates that (1) design discussions are still quite rare, with only 9% and 14% of Nova and Neutron review comments being related to software design, respectively; and (2) design feedback is often constructive, with 73% of the design-related comments also providing suggestions to address the concerns. Furthermore, our classifiers achieve a precision of 59%–66% and a recall of 70%–78%, outperforming baselines like zeroR by 43 percentage points in terms of F1-score. Finally, code changes that have design-related feedback have a statistically significantly increased rate of abandonment (PearsonΧ2 test, DF=1, p< 0.001). Conclusion: Design-related discussion during code review is still rare. Since design discussion is a primary motivation for conducting code review, more may need to be done to encourage such discussions among contributors.
BibTeX:
@inproceedings{faridaelzanaty2018aesoddicr,
author = "Farida El Zanaty and Toshiki Hirao and Shane McIntosh and Akinori Ihara and Kenichi Matsumoto",
title = "An Empirical Study of Design Discussions in Code Review",
year = "2018",
pages = "11:1–11:10",
booktitle = "Proceedings of the 12th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement"
}
Plain Text:
Farida El Zanaty, Toshiki Hirao, Shane McIntosh, Akinori Ihara, and Kenichi Matsumoto, "An Empirical Study of Design Discussions in Code Review," 12th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, pp. 11:1–11:10