John Penix

John Penix

Works on deploying automated tools into developer workflows. He is on the Steering Committee of the IEEE/ACM International Conference on Automated Software Engineering, and has worked in the Automated Software Engineering R&D group at NASA's Ames Research Center before joining Google. He holds a Ph.D. in Computer Engineering from the University of Cincinnati.

Research Areas

Authored Publications
Google Publications
Other Publications
Sort By
  • Title
  • Title, descending
  • Year
  • Year, descending
    Techniques for improving regression testing in continuous integration development environments
    Sebastian Elbaum
    Gregg Rothermel
    Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering, ACM(2014), pp. 235-245
    Preview abstract In continuous integration development environments, software engineers frequently integrate new or changed code with the mainline codebase. This can reduce the amount of code rework that is needed as systems evolve and speed up development time. While continuous integration processes traditionally require that extensive testing be performed following the actual submission of code to the codebase, it is also important to ensure that enough testing is performed prior to code submission to avoid breaking builds and delaying the fast feedback that makes continuous integration desirable. In this work, we present algorithms that make continuous integration processes more cost-effective. In an initial pre-submit phase of testing, developers specify modules to be tested, and we use regression test selection techniques to select a subset of the test suites for those modules that render that phase more cost-effective. In a subsequent post-submit phase of testing, where dependent modules as well as changed modules are tested, we use test case prioritization techniques to ensure that failures are reported more quickly. In both cases, the techniques we utilize are novel, involving algorithms that are relatively inexpensive and do not rely on code coverage information -- two requirements for conducting testing cost-effectively in this context. To evaluate our approach, we conducted an empirical study on a large data set from Google that we make publicly available. The results of our study show that our selection and prioritization techniques can each lead to cost-effectiveness improvements in the continuous integration process. View details
    Introducing Continuous Systematic Testing of Evolving Software
    Mary Jean Harrold
    Darko Marinov
    Stephen Oney
    Mauro Pezzè
    Adam Porter
    Per Runeson
    Shin Yoo
    Dagstuhl Seminar Proceedings, Schloss Dagstuhl-Leibniz-Zentrum für Informatik(2010)
    Preview abstract In today's evolutionary development of software, continuous testing is needed to ensure that the software is still functioning after changes. Test automation helps partly managing the large number of executions needed, but there is also a limit for how much automated tests may be executed. Then systematic approaches for test selection are needed also for automated tests. This manuscript defines this situation and outlines a general method and tool framework for its solution. Experiences from different companies are collected to illustrate how it may be set into practice. View details
    Predicting Accurate and Actionable Static Analysis Warnings: An Experimental Approach
    Joseph Ruthruff
    J. David Morgenthaler
    Sebastian Elbaum
    Gregg Rothermel
    Proceedings of the International Conference on Software Engineering, ACM(2008), pp. 341-350
    Preview abstract Static analysis tools report software defects that may or may not be detected by other verification methods. Two challenges complicating the adoption of these tools are spurious false positive warnings and legitimate warnings that are not acted on. This paper reports automated support to help address these challenges using logistic regression models that predict the foregoing types of warnings from signals in the warnings and implicated code. Because examining many potential signaling factors in large software development settings can be expensive, we use a screening methodology to quickly discard factors with low predictive power and cost-effectively build predictive models. Our empirical evaluation indicates that these models can achieve high accuracy in predicting accurate and actionable static analysis warnings, and suggests that the models are competitive with alternative models built without screening. View details
    Experiences Using Static Analysis to Find Bugs
    Nathaniel Ayewah
    David Hovemeyer
    J. David Morgenthaler
    William Pugh
    IEEE Software, 25(2008), pp. 22-29
    Preview abstract Static analysis examines code in the absence of input data and without running the code, and can detect potential security violations (e.g., SQL injection), runtime errors (e.g., dereferencing a null pointer) and logical inconsistencies (e.g., a conditional test that cannot possibly be true). While there is a rich body of literature on algorithms and analytical frameworks used by such tools, reports describing experiences with such tools in industry are much harder to come by. In this paper, we describe FindBugs, an open source static analysis tool for Java, and experience using it in production settings. FindBugs does not push the envelope in terms of the sophistication of its analysis techniques. Rather, it is designed to evaluate what kinds of defects can be effectively detected with relatively simple techniques and to help us understand how such tools can be incorporated into the software development process. FindBugs has become very popular, downloaded more than 500,000 times and used by many major companies and software projects. We report on experience running FindBugs against Sun’s JDK implementation, using Findbugs at Google where it has been used for more than a year and a half and incorporated into their standard development process, and preliminary results from a survey of FindBugs users. View details
    Using FindBugs on Production Software
    Nathaniel Ayewah
    J. David Morgenthaler
    William Pugh
    YuQian Zhou
    Proc. OOPSLA'07, ACM, Montréal(2007)
    Preview
    Evaluating Static Analysis Defect Warnings on Production Software
    Nathaniel Ayewah
    William Pugh
    J. David Morgenthaler
    YuQian Zhou
    Proceedings of the 7th ACM SIGPLAN-SIGSOFT workshop on Program analysis for software tools and engineering, ACM Press, New York, NY, USA(2007), pp. 1-8
    Preview abstract Classification of static analysis warnings into false positive, trivial or serious bugs: Experience on Java JDK and Google codebase View details
    Guest Editors' Introduction
    John C. Grundy
    Autom. Softw. Eng., 13(2006), pp. 343-344
    Verifying Time Partitioning in the DEOS Scheduling Kernel
    Willem Visser
    Seungjoon Park
    Corina S. Pasareanu
    Eric Engstrom
    Aaron Larson
    Nicholas Weininger
    Formal Methods in System Design, 26(2005), pp. 103-135
    Management of Interdependencies in Collaborative Software Development
    Cleidson R. B. de Souza
    David F. Redmiles
    Gloria Mark
    Maarten Sierhuis
    International Symposium on Empirical Software Engineering (ISESE), IEEE(2003), pp. 294-303
    Editorial: The First International Workshop on Automated Program Analysis, Testing and Verification (WAPATV 2000)
    Nigel Tracey
    Willem Visser
    Softw. Test., Verif. Reliab., 11(2001), pp. 63-64
    Amphion/NAV: Deductive Synthesis of State Estimation Software
    Jon Whittle
    Jeffrey Van Baalen
    Johann Schumann
    Peter Robinson
    Thomas Pressburger
    Phil Oh
    Michael R. Lowry
    Guillaume P. Brat
    International Conference on Automated Software Engineering(2001), pp. 395-399
    The Second International Workshop on Automated Program Analysis, Testing and Verification
    Nigel Tracey
    Willem Visser
    Proceedings of the International Conference on Software Engineering(2001), pp. 755-755
    Formal Analysis of a Space-Craft Controller Using SPIN
    Klaus Havelund
    Michael R. Lowry
    IEEE Trans. Software Eng., 27(2001), pp. 749-765
    Using predicate abstraction to reduce object-oriented programs for model checking
    Willem Visser
    Seungjoon Park
    Proceedings of the ACM Workshop on Formal Methods in Software Practice(2000), pp. 3-182
    The First International Workshop on Automated Program Analysis, Testing and Verification
    Nigel Tracey
    Willem Visser
    Proceedings of the International Conference on Software Engineering(2000), pp. 796
    Verification of time partitioning in the DEOS scheduler kernel
    Willem Visser
    Eric Engstrom
    Aaron Larson
    Nicholas Weininger
    ICSE(2000), pp. 488-497
    SPIN Model Checking and Software Verification, 7th International SPIN Workshop, Stanford, CA, USA, August 30 - September 1, 2000, Proceedings
    Klaus Havelund
    Willem Visser
    Springer
    Efficient Specification-Based Component Retrieval
    Perry Alexander
    Autom. Softw. Eng., 6(1999), pp. 139-170
    Deductive Synthesis of Event-Based Software Architectures
    International Conference on Automated Software Engineering, IEEE/ACM(1999), pp. 311-314
    Experiences in verifying parallel simulation algorithms
    Dale E. Martin
    Peter Frey
    Ramanan Radhakrishnan
    Perry Alexander
    Philip A. Wilsey
    Proceedings of the ACM Workshop on Formal Methods in Software Practice, ACM(1998), pp. 16-23
    Gravity: An Object-Oriented Framework for Hardware/Software Tool Integration
    Murali Rangarajan
    Perry Alexander
    Philip A. Wilsey
    Annual Simulation Symposium(1997), pp. 24-30
    Verification and Validation of AI Systems that Control Deep-Space Spacecraft
    Michael R. Lowry
    Klaus Havelund
    ISMIS(1997), pp. 35-47
    Declarative Specification of Software Architectures
    Perry Alexander
    Klaus Havelund
    International Conference on Automated Software Engineering, IEEE(1997), pp. 201-
    Classification and Retrieval of Reusable Components Using Semantic Features
    Phillip Baraona
    Perry Alexander
    Knowledge-Based Software Engineering Conference, IEEE(1995), pp. 131-138