Bobby Dorward
Currently interested in algorithms and theory, working on advancing algorithms in the continuous integration space.
Before Google, did research in combinatorics and number theory.
Research Areas
Authored Publications
Google Publications
Other Publications
Sort By
Flake Aware Culprit Finding
Eric Nickell
Collin Johnston
Avi Kondareddy
Proceedings of the 16th IEEE International Conference on Software Testing, Verification and Validation (ICST 2023), IEEE (to appear)
Preview abstract
When a change introduces a bug into a large software repository, there is
often a delay between when the change is committed and when bug is detected.
This is true even when the bug causes an existing test to fail! These delays
are caused by resource constraints which prevent the organization from running
all of the tests on every change. Due to the delay, a Continuous Integration
system needs to locate buggy commits. Locating them is complicated by flaky
tests that pass and fail non-deterministically. The flaky tests introduce
noise into the CI system requiring costly reruns to determine if a failure was
caused by a bad code change or caused by non-deterministic test behavior. This
paper presents an algorithm, Flake Aware Culprit Finding, that locates
buggy commits more accurately than a traditional bisection search. The
algorithm is based on Bayesian inference and noisy binary search, utilizing
prior information about which changes are most likely to contain the bug. A
large scale empirical study was conducted at Google on 13,000+ test breakages.
The study evaluates the accuracy and cost of the new algorithm versus a
traditional deflaked bisection search.
View details
Preview abstract
Flaky, non-deterministic tests make culprit finding (or finding the version of code where a test started failing) in large scale repositories difficult. A naive binary search (or bisect) algorithm will be unreliable if the test being used for the bisection is flaky. If retries are conducted for each tested version the process becomes expensive. We propose a flake-aware culprit finding system which takes into account the prior flakiness of a test and uses a Bayesian probabilistic model to reduce the number of test executions needed to achieve accurate culprit finding faster and using fewer resources than binary search with retries.
View details
Restricted growth function patterns and statistics
Lindsey Campbell
Samantha Dahlberg
Jonathan Gerhard
Thomas Grubb
Carlin Purcell
Bruce E. Sagan
Advances in Applied Mathematics, vol. 100 (2018), pp. 1-42
A generalization of Zeckendorf's theorem via circumscribed m-gons
Pari L. Ford
Eva Fourakis
Pamela E. Harris
Steven J. Miller
Eyvindur Palsson
Hannah Paugh
Involve, vol. 10 (2017), pp. 125-150
Individual Gap Measures from Generalized Zeckendorf Decompositions
Pari L. Ford
Eva Fourakis
Pamela E. Harris
Steven J. Miller
Eyvindur Palsson
Hannah Paugh
Uniform Distribution Theory, vol. 12 (2017), pp. 27-36
One-level density for holomorphic cusp forms of arbitrary level
Owen Barrett
Paula Burkhardt
Jonathan DeWitt
Steven J. Miller
Research in Number Theory, vol. 3 (2017)
Set partition patterns and statistics
Samantha Dahlberg
Jonathan Gerhard
Thomas Grubb
Carlin Purcell
Lindsey Reppuhn
Bruce E. Sagan
Discrete Mathematics, vol. 339 (2016), pp. 1-16
