Please use this identifier to cite or link to this item: http://dx.doi.org/10.14279/depositonce-8721.2
For citation please use:
Main Title: Optimizing for Recall in Automatic Requirements Classification: An Empirical Study
Author(s): Winkler, Jonas Paul
Grönberg, Jannis
Vogelsang, Andreas
Type: Conference Object
URI: https://depositonce.tu-berlin.de/handle/11303/9678.2
http://dx.doi.org/10.14279/depositonce-8721.2
License: http://rightsstatements.org/vocab/InC/1.0/
Abstract: Using Machine Learning to solve requirements engineering problems can be a tricky task. Even though certain algorithms have exceptional performance, their recall is usually below 100%. One key aspect in the implementation of machine learning tools is the balance between recall and precision. Tools that do not find all correct answers may be considered useless. However, some tasks are very complicated and even requirements engineers struggle to solve them perfectly. If a tool achieves performance comparable to a trained engineer while reducing her workload considerably, it is considered to be useful. One such task is the classification of specification content elements into requirements and non-requirements. In this paper, we analyze this specific requirements classification problem and assess the importance of recall by performing an empirical study. We compared two groups of students who performed this task with and without tool support, respectively. We use the results to compute an estimate of β for the Fβ score, allowing us to choose the optimal balance between precision and recall. Furthermore, we use the results to assess the practical time savings realized by the approach. By using the tool, users may not be able to find all defects in a document, however, they will be able to find close to all of them in a fraction of the time necessary. This demonstrates the practical usefulness of our approach and machine learning tools in general.
Subject(s): empirical research
controlled experiment
machine learning
automation
Issue Date: 5-Dec-2019
Date Available: 12-Apr-2021
Language Code: en
DDC Class: 004 Datenverarbeitung; Informatik
006 Spezielle Computerverfahren
Proceedings Title: 2019 IEEE 27th International Requirements Engineering Conference (RE)
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
Publisher DOI: 10.1109/RE.2019.00016
Page Start: 40
Page End: 50
EISSN: 2332-6441
ISBN: 978-1-7281-3912-8
TU Affiliation(s): Fak. 4 Elektrotechnik und Informatik » Inst. Telekommunikationssysteme » FG IT-basierte Fahrzeuginnovationen
Appears in Collections:Technische Universität Berlin » Publications

Files in This Item:
winkler_etal_RE2019.pdf
Format: Adobe PDF | Size: 744.87 kB
DownloadShow Preview
Thumbnail

Version History
Version Item Date Summary
2 10.14279/depositonce-8721.2 2021-04-12 15:05:50.506 Paper published, include DOI
1 10.14279/depositonce-8721 2019-07-31 17:05:19.0
Item Export Bar

Items in DepositOnce are protected by copyright, with all rights reserved, unless otherwise indicated.