Please fix this mutant: How do developers resolve mutants surfaced during code review?

This paper studies the effects of surfacing undetected mutants during code review. Based on a dataset of 633 merge requests and 78,000 mutants, it answers three research questions around the change in mutant location over the course of a merge request, how often mutants are resolved during code review, and the observed changes after mutant intervention. The results show that (1) for 64% of mutants, the mutated code changes as the merge request evolves; (2) overall, 38% of all mutants and 60% of productive mutants are resolved via code changes or test additions; (3) unresolved productive mutants stem from developers questioning the value of adding tests for surfaced mutants, mutants being later resolved in deferred code changes (atomicity of merge requests), and false positives (mutants being resolved by tests not considered in the experiment infrastructure); (4) resolved productive mutants are associated with more test and code changes, compared to unproductive mutants.

• Software Engineering