Stammdaten

Noise and Heterogeneity in Historical Build Data
Untertitel: An Empirical Study of Travis CI
Kurzfassung:

Automated builds, which may pass or fail, provide feedback to a development team about changes to the codebase. A passing build indicates that the change compiles cleanly and tests (continue to) pass. A failing (a.k.a., broken) build indicates that there are issues that require attention. Without a closer analysis of the nature of build outcome data, practitioners and researchers are likely to make two critical assumptions: (1) build results are not noisy; however, passing builds may contain failing or skipped jobs that are actively or passively ignored; and (2) builds are equal; however, builds vary in terms of the number of jobs and conigurations.

To investigate the degree to which these assumptions about build breakage hold, we perform an empirical study of 3.7 million build jobs spanning 1,276 open source projects. We find that: (1) 12% of passing builds have an actively ignored failure; (2) 9% of builds have a misleading or incorrect outcome on average; and (3) at least 44% of the broken builds contain passing jobs, i.e., the breakage is local to a subset of build variants. Like other software archives, build data is noisy and complex. Analysis of build data requires nuance.

Schlagworte:
Publikationstyp: Beitrag in Proceedings (Autorenschaft)
Art der Veröffentlichung Online Publikation
Erschienen in: Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering, ASE 2018
Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering, ASE 2018
zur Publikation
 ( ACM New York; )
Erscheinungdatum: 03.09.2018
Titel der Serie: -
Bandnummer: -
Erstveröffentlichung: Ja
Version: -
Seite: S. 87 - 97

Identifikatoren

ISBN:
  • 978-1-4503-5937-5
ISSN: -
DOI: http://dx.doi.org/10.1145/3238147.3238171
AC-Nummer: -
Homepage: https://dl.acm.org/citation.cfm?id=3238171
Open Access
  • Online verfügbar (Open Access)

Zuordnung

Organisation Adresse
Fakultät für Technische Wissenschaften
 
Institut für Informatik-Systeme
Universitätsstr. 65-67
A-9020  Klagenfurt
Österreich
  -993502
   sek-eder@isys.uni-klu.ac.at
https://www.aau.at/isys/
zur Organisation
Universitätsstr. 65-67
AT - A-9020  Klagenfurt

Kategorisierung

Sachgebiete
  • 102022 - Softwareentwicklung
Forschungscluster Kein Forschungscluster ausgewählt
Peer Reviewed
  • Ja
Publikationsfokus
  • Science to Science (Qualitätsindikator: I)
Klassifikationsraster der zugeordneten Organisationseinheiten:
Arbeitsgruppen
  • Software Engineering Research Group

Kooperationen

Organisation Adresse
McGill University
845 Sherbrooke Street
Montreal
Kanada
845 Sherbrooke Street
CA  Montreal

Beiträge der Publikation

Keine verknüpften Publikationen vorhanden