NDNetGaming - development of a no-reference deep CNN for gaming video quality prediction

Simple item page
dc.contributor.authorUtke, Markus
dc.contributor.authorZadtootaghaj, Saman
dc.contributor.authorSchmidt, Steven
dc.contributor.authorBosse, Sebastian
dc.contributor.authorMöller, Sebastian
dc.date.accessioned2021-03-18T08:48:25Z
dc.date.available2021-03-18T08:48:25Z
dc.date.issued2020-07-24
dc.description.abstractGaming video streaming services are growing rapidly due to new services such as passive video streaming of gaming content, e.g. Twitch.tv, as well as cloud gaming, e.g. Nvidia GeForce NOW and Google Stadia. In contrast to traditional video content, gaming content has special characteristics such as extremely high and special motion patterns, synthetic content and repetitive content, which poses new opportunities for the design of machine learning-based models to outperform the state-of-the-art video and image quality approaches for this special computer generated content. In this paper, we train a Convolutional Neural Network (CNN) based on an objective quality model, VMAF, as ground truth and fine-tuned it based on subjective image quality ratings. In addition, we propose a new temporal pooling method to predict gaming video quality based on frame-level predictions. Finally, the paper also describes how an appropriate CNN architecture can be chosen and how well the model performs on different contents. Our result shows that among four popular network architectures that we investigated, DenseNet performs best for image quality assessment based on the training dataset. By training the last 57 convolutional layers of DenseNet based on VMAF values, we obtained a high performance model to predict VMAF of distorted frames of video games with a Spearman’s Rank correlation (SRCC) of 0.945 and Root Mean Score Error (RMSE) of 7.07 on the image level, while achieving a higher performance on the video level leading to a SRCC of 0.967 and RMSE of 5.47 for the KUGVD dataset. Furthermore, we fine-tuned the model based on subjective quality ratings of images from gaming content which resulted in a SRCC of 0.93 and RMSE of 0.46 using one-hold-out cross validation. Finally, on the video level, using the proposed pooling method, the model achieves a very good performance indicated by a SRCC of 0.968 and RMSE of 0.30 for the used gaming video dataset.en
dc.description.sponsorshipTU Berlin, Open-Access-Mittel – 2020en
dc.description.sponsorshipEC/H2020/871793/EU/Adaptive edge/cloud compute and network continuum over a heterogeneous sparse edge infrastructure to support nextgen applications/ACCORDIONen
dc.identifier.eissn1573-7721
dc.identifier.issn1380-7501
dc.identifier.urihttps://depositonce.tu-berlin.de/handle/11303/12852
dc.identifier.urihttp://dx.doi.org/10.14279/depositonce-11652
dc.language.isoen
dc.rights.urihttps://creativecommons.org/licenses/by/4.0/
dc.subject.ddc004 Datenverarbeitung; Informatiken
dc.subject.otherconvolutional neural networken
dc.subject.othergaming videoen
dc.subject.otherquality assessmenten
dc.subject.otherNDNetGamingen
dc.subject.otherCNNen
dc.titleNDNetGaming - development of a no-reference deep CNN for gaming video quality predictionen
dc.typeArticleen
dc.type.versionpublishedVersionen
dcterms.bibliographicCitation.doi10.1007/s11042-020-09144-6en
dcterms.bibliographicCitation.journaltitleMultimedia Tools and Applicationsen
dcterms.bibliographicCitation.originalpublishernameSpringerNatureen
dcterms.bibliographicCitation.originalpublisherplaceLondon [u.a.]en
tub.accessrights.dnbfreeen
tub.affiliationFak. 4 Elektrotechnik und Informatik::Inst. Softwaretechnik und Theoretische Informatik::Quality and Usability Labde
tub.affiliation.facultyFak. 4 Elektrotechnik und Informatikde
tub.affiliation.groupQuality and Usability Labde
tub.affiliation.instituteInst. Softwaretechnik und Theoretische Informatikde
tub.publisher.universityorinstitutionTechnische Universität Berlinen

Files

Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
Utke_etal_NDNetGaming_2020.pdf
Size:
1.77 MB
Format:
Adobe Portable Document Format
Download

Collections

Publications