Please use this identifier to cite or link to this item: http://dx.doi.org/10.14279/depositonce-11624
For citation please use:
Main Title: When algorithm selection meets Bi-linear Learning to Rank: accuracy and inference time trade off with candidates expansion
Author(s): Yuan, Jing
Geissler, Christian
Shao, Weijia
Lommatzsch, Andreas
Jain, Brijnesh
Type: Article
URI: https://depositonce.tu-berlin.de/handle/11303/12824
http://dx.doi.org/10.14279/depositonce-11624
License: https://creativecommons.org/licenses/by/4.0/
Abstract: Algorithm selection (AS) tasks are dedicated to find the optimal algorithm for an unseen problem instance. With the knowledge of problem instances’ meta-features and algorithms’ landmark performances, Machine Learning (ML) approaches are applied to solve AS problems. However, the standard training process of benchmark ML approaches in AS either needs to train the models specifically for every algorithm or relies on the sparse one-hot encoding as the algorithms’ representation. To escape these intermediate steps and form the mapping function directly, we borrow the learning to rank framework from Recommender System (RS) and embed the bi-linear factorization to model the algorithms’ performances in AS. This Bi-linear Learning to Rank (BLR) has proven to work with competence in some AS scenarios and thus is also proposed as a benchmark approach. Thinking from the evaluation perspective in the modern AS challenges, precisely predicting the performance is usually the measuring goal. Though approaches’ inference time also needs to be counted for the running time cost calculation, it’s always overlooked in the evaluation process. The multi-objective evaluation metric Adjusted Ratio of Root Ratios (A3R) is therefore advocated in this paper to balance the trade-off between the accuracy and inference time in AS. Concerning A3R, BLR outperforms other benchmarks when expanding the candidates range to TOP 3. The better effect of this candidates expansion results from the cumulative optimum performance during the AS process. We take the further step in the experimentation to represent the advantage of such TOPK expansion, and illustrate that such expansion can be considered as the supplement for the convention of TOP 1 selection during the evaluation process.
Subject(s): algorithm selection
bi-linear learning to rank
candidates expansion
multi-object evaluation
machine learning
Issue Date: 9-Oct-2020
Date Available: 15-Mar-2021
Language Code: en
DDC Class: 004 Datenverarbeitung; Informatik
Sponsor/Funder: TU Berlin, Open-Access-Mittel – 2020
BMBF, 01IS16046, CODA: Cognitive Data Analytics Framework
Journal Title: International Journal of Data Science and Analytics
Publisher: SpringerNature
Publisher DOI: 10.1007/s41060-020-00229-x
EISSN: 2364-4168
ISSN: 2364-415X
TU Affiliation(s): Fak. 4 Elektrotechnik und Informatik » Inst. Wirtschaftsinformatik und Quantitative Methoden » FG Agententechnologien in betrieblichen Anwendungen und der Telekommunikation (AOT)
Appears in Collections:Technische Universität Berlin » Publications

Files in This Item:
Yuan_etal_When_2020.pdf
Format: Adobe PDF | Size: 1.39 MB
DownloadShow Preview
Thumbnail

Item Export Bar

This item is licensed under a Creative Commons License Creative Commons