Publication: Noise and Heterogeneity in Historical B...
Master data
Title: | Noise and Heterogeneity in Historical Build Data |
Subtitle: | An Empirical Study of Travis CI |
Abstract: | Automated builds, which may pass or fail, provide feedback to a development team about changes to the codebase. A passing build indicates that the change compiles cleanly and tests (continue to) pass. A failing (a.k.a., broken) build indicates that there are issues that require attention. Without a closer analysis of the nature of build outcome data, practitioners and researchers are likely to make two critical assumptions: (1) build results are not noisy; however, passing builds may contain failing or skipped jobs that are actively or passively ignored; and (2) builds are equal; however, builds vary in terms of the number of jobs and conigurations. To investigate the degree to which these assumptions about build breakage hold, we perform an empirical study of 3.7 million build jobs spanning 1,276 open source projects. We find that: (1) 12% of passing builds have an actively ignored failure; (2) 9% of builds have a misleading or incorrect outcome on average; and (3) at least 44% of the broken builds contain passing jobs, i.e., the breakage is local to a subset of build variants. Like other software archives, build data is noisy and complex. Analysis of build data requires nuance. |
Keywords: |
Publication type: | Article in Proceedings (Authorship) |
Publication date: | 03.09.2018 (Online) |
Published by: |
Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering, ASE 2018
Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering, ASE 2018
(
ACM New York;
)
to publication |
Title of the series: | - |
Volume number: | - |
First publication: | Yes |
Version: | - |
Page: | pp. 87 - 97 |
Versionen
Keine Version vorhanden |
Publication date: | 03.09.2018 |
ISBN (e-book): |
|
eISSN: | - |
DOI: | http://dx.doi.org/10.1145/3238147.3238171 |
Homepage: | https://dl.acm.org/citation.cfm?id=3238171 |
Open access |
|
Authors
Keheliya Gallaba
|
||||
Christian Macho (internal) | ||||
Martin Pinzger (internal) | ||||
Shane McIntosh (external) |
Assignment
Organisation | Address | ||||
---|---|---|---|---|---|
Fakultät für Technische Wissenschaften
Institut für Informatik-Systeme
|
AT - A-9020 Klagenfurt |
Categorisation
Subject areas | |
Research Cluster | No research Research Cluster selected |
Peer reviewed |
|
Publication focus |
Classification raster of the assigned organisational units:
|
working groups |
|
Cooperations
Organisation | Address | ||
---|---|---|---|
McGill University
|
CA Montreal |
Research activities
Projects: | No related projects |
Publications: | No related publications |
Events: | No related events |
Lectures: | No related lectures |