Please use this identifier to cite or link to this item: http://dx.doi.org/10.14279/depositonce-8721
Main Title: Optimizing for Recall in Automatic Requirements Classification: An Empirical Study
Author(s): Winkler, Jonas Paul
Grönberg, Jannis
Vogelsang, Andreas
Type: Conference Object
Language Code: en
Abstract: Using Machine Learning to solve requirements engineering problems can be a tricky task. Even though certain algorithms have exceptional performance, their recall is usually below 100%. One key aspect in the implementation of machine learning tools is the balance between recall and precision. Tools that do not find all correct answers may be considered useless. However, some tasks are very complicated and even requirements engineers struggle to solve them perfectly. If a tool achieves performance comparable to a trained engineer while reducing her workload considerably, it is considered to be useful. One such task is the classification of specification content elements into requirements and non-requirements. In this paper, we analyze this specific requirements classification problem and assess the importance of recall by performing an empirical study. We compared two groups of students who performed this task with and without tool support, respectively. We use the results to compute an estimate of β for the Fβ score, allowing us to choose the optimal balance between precision and recall. Furthermore, we use the results to assess the practical time savings realized by the approach. By using the tool, users may not be able to find all defects in a document, however, they will be able to find close to all of them in a fraction of the time necessary. This demonstrates the practical usefulness of our approach and machine learning tools in general.
URI: https://depositonce.tu-berlin.de/handle/11303/9678
http://dx.doi.org/10.14279/depositonce-8721
Issue Date: 2019
Date Available: 31-Jul-2019
DDC Class: 004 Datenverarbeitung; Informatik
006 Spezielle Computerverfahren
Subject(s): empirical research
controlled experiment
machine learning
automation
License: http://rightsstatements.org/vocab/InC/1.0/
Proceedings Title: 27th IEEE International Requirements Engineering Conference (RE'19)
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
Publisher Place: Jeju Island
Appears in Collections:FG IT-basierte Fahrzeuginnovationen » Publications

Files in This Item:
File Description SizeFormat 
winkler_etal_2019.pdf716.88 kBAdobe PDFThumbnail
View/Open


Items in DepositOnce are protected by copyright, with all rights reserved, unless otherwise indicated.