Semantic Prior Based Generative Adversarial Network for Video Super-Resolution

Semantic Prior Based Generative Adversarial Network for Video Super-Resolution. Wu, X., Lucas, A., Lopez-Tapia, S., Wang, X., Kim, Y. H., Molina, R., & Katsaggelos, A. K. In 2019 27th European Signal Processing Conference (EUSIPCO), volume 2019-Septe, pages 1–5, sep, 2019. IEEE.

Paper doi abstract bibtex

Semantic information is widely used in the deep learning literature to improve the performance of visual media processing. In this work, we propose a semantic prior based Generative Adversarial Network (GAN) model for video super-resolution. The model fully utilizes various texture styles from different semantic categories of video-frame patches, contributing to more accurate and efficient learning for the generator. Based on the GAN framework, we introduce the semantic prior by making use of the spatial feature transform during the learning process of the generator. The patch-wise semantic prior is extracted on the whole video frame by a semantic segmentation network. A hybrid loss function is designed to guide the learning performance. Experimental results show that our proposed model is advantageous in sharpening video frames, reducing noise and artifacts, and recovering realistic textures.

@inproceedings{Xinyi2019,
abstract = {Semantic information is widely used in the deep learning literature to improve the performance of visual media processing. In this work, we propose a semantic prior based Generative Adversarial Network (GAN) model for video super-resolution. The model fully utilizes various texture styles from different semantic categories of video-frame patches, contributing to more accurate and efficient learning for the generator. Based on the GAN framework, we introduce the semantic prior by making use of the spatial feature transform during the learning process of the generator. The patch-wise semantic prior is extracted on the whole video frame by a semantic segmentation network. A hybrid loss function is designed to guide the learning performance. Experimental results show that our proposed model is advantageous in sharpening video frames, reducing noise and artifacts, and recovering realistic textures.},
author = {Wu, Xinyi and Lucas, Alice and Lopez-Tapia, Santiago and Wang, Xijun and Kim, Yul Hee and Molina, Rafael and Katsaggelos, Aggelos K.},
booktitle = {2019 27th European Signal Processing Conference (EUSIPCO)},
doi = {10.23919/EUSIPCO.2019.8902987},
isbn = {978-9-0827-9703-9},
issn = {22195491},
keywords = {Generative Adversarial Networks,Hybrid loss function,Semantic Segmentation,Spatial Feature Transform,Video Super-Resolution},
month = {sep},
pages = {1--5},
publisher = {IEEE},
title = {{Semantic Prior Based Generative Adversarial Network for Video Super-Resolution}},
url = {https://ieeexplore.ieee.org/document/8902987/},
volume = {2019-Septe},
year = {2019}
}

Downloads: 0

{"_id":"cPSzLCDxm3uQWSwvq","bibbaseid":"wu-lucas-lopeztapia-wang-kim-molina-katsaggelos-semanticpriorbasedgenerativeadversarialnetworkforvideosuperresolution-2019","authorIDs":[],"author_short":["Wu, X.","Lucas, A.","Lopez-Tapia, S.","Wang, X.","Kim, Y. H.","Molina, R.","Katsaggelos, A. K."],"bibdata":{"bibtype":"inproceedings","type":"inproceedings","abstract":"Semantic information is widely used in the deep learning literature to improve the performance of visual media processing. In this work, we propose a semantic prior based Generative Adversarial Network (GAN) model for video super-resolution. The model fully utilizes various texture styles from different semantic categories of video-frame patches, contributing to more accurate and efficient learning for the generator. Based on the GAN framework, we introduce the semantic prior by making use of the spatial feature transform during the learning process of the generator. The patch-wise semantic prior is extracted on the whole video frame by a semantic segmentation network. A hybrid loss function is designed to guide the learning performance. Experimental results show that our proposed model is advantageous in sharpening video frames, reducing noise and artifacts, and recovering realistic textures.","author":[{"propositions":[],"lastnames":["Wu"],"firstnames":["Xinyi"],"suffixes":[]},{"propositions":[],"lastnames":["Lucas"],"firstnames":["Alice"],"suffixes":[]},{"propositions":[],"lastnames":["Lopez-Tapia"],"firstnames":["Santiago"],"suffixes":[]},{"propositions":[],"lastnames":["Wang"],"firstnames":["Xijun"],"suffixes":[]},{"propositions":[],"lastnames":["Kim"],"firstnames":["Yul","Hee"],"suffixes":[]},{"propositions":[],"lastnames":["Molina"],"firstnames":["Rafael"],"suffixes":[]},{"propositions":[],"lastnames":["Katsaggelos"],"firstnames":["Aggelos","K."],"suffixes":[]}],"booktitle":"2019 27th European Signal Processing Conference (EUSIPCO)","doi":"10.23919/EUSIPCO.2019.8902987","isbn":"978-9-0827-9703-9","issn":"22195491","keywords":"Generative Adversarial Networks,Hybrid loss function,Semantic Segmentation,Spatial Feature Transform,Video Super-Resolution","month":"sep","pages":"1–5","publisher":"IEEE","title":"Semantic Prior Based Generative Adversarial Network for Video Super-Resolution","url":"https://ieeexplore.ieee.org/document/8902987/","volume":"2019-Septe","year":"2019","bibtex":"@inproceedings{Xinyi2019,\nabstract = {Semantic information is widely used in the deep learning literature to improve the performance of visual media processing. In this work, we propose a semantic prior based Generative Adversarial Network (GAN) model for video super-resolution. The model fully utilizes various texture styles from different semantic categories of video-frame patches, contributing to more accurate and efficient learning for the generator. Based on the GAN framework, we introduce the semantic prior by making use of the spatial feature transform during the learning process of the generator. The patch-wise semantic prior is extracted on the whole video frame by a semantic segmentation network. A hybrid loss function is designed to guide the learning performance. Experimental results show that our proposed model is advantageous in sharpening video frames, reducing noise and artifacts, and recovering realistic textures.},\nauthor = {Wu, Xinyi and Lucas, Alice and Lopez-Tapia, Santiago and Wang, Xijun and Kim, Yul Hee and Molina, Rafael and Katsaggelos, Aggelos K.},\nbooktitle = {2019 27th European Signal Processing Conference (EUSIPCO)},\ndoi = {10.23919/EUSIPCO.2019.8902987},\nisbn = {978-9-0827-9703-9},\nissn = {22195491},\nkeywords = {Generative Adversarial Networks,Hybrid loss function,Semantic Segmentation,Spatial Feature Transform,Video Super-Resolution},\nmonth = {sep},\npages = {1--5},\npublisher = {IEEE},\ntitle = {{Semantic Prior Based Generative Adversarial Network for Video Super-Resolution}},\nurl = {https://ieeexplore.ieee.org/document/8902987/},\nvolume = {2019-Septe},\nyear = {2019}\n}\n","author_short":["Wu, X.","Lucas, A.","Lopez-Tapia, S.","Wang, X.","Kim, Y. H.","Molina, R.","Katsaggelos, A. K."],"key":"Xinyi2019","id":"Xinyi2019","bibbaseid":"wu-lucas-lopeztapia-wang-kim-molina-katsaggelos-semanticpriorbasedgenerativeadversarialnetworkforvideosuperresolution-2019","role":"author","urls":{"Paper":"https://ieeexplore.ieee.org/document/8902987/"},"keyword":["Generative Adversarial Networks","Hybrid loss function","Semantic Segmentation","Spatial Feature Transform","Video Super-Resolution"],"metadata":{"authorlinks":{}},"downloads":0},"bibtype":"inproceedings","biburl":"https://sites.northwestern.edu/ivpl/files/2023/06/IVPL_Updated_publications-1.bib","creationDate":"2021-02-11T19:15:22.081Z","downloads":0,"keywords":["generative adversarial networks","hybrid loss function","semantic segmentation","spatial feature transform","video super-resolution"],"search_terms":["semantic","prior","based","generative","adversarial","network","video","super","resolution","wu","lucas","lopez-tapia","wang","kim","molina","katsaggelos"],"title":"Semantic Prior Based Generative Adversarial Network for Video Super-Resolution","year":2019,"dataSources":["NqWTiMfRR56v86wRs","r6oz3cMyC99QfiuHW","ya2CyA73rpZseyrZ8","ePKPjG8C6yvpk4mEK","E6Bth2QB5BYjBMZE7","nbnEjsN7MJhurAK9x","PNQZj6FjzoxxJk4Yi","7FpDWDGJ4KgpDiGfB","bod9ms4MQJHuJgPpp","QR9t5P2cLdJuzhfzK","D8k2SxfC5dKNRFgro","7Dwzbxq93HWrJEhT6","qhF8zxmGcJfvtdeAg","fvDEHD49E2ZRwE3fb","H7crv8NWhZup4d4by","DHqokWsryttGh7pJE","vRJd4wNg9HpoZSMHD","sYxQ6pxFgA59JRhxi","w2WahSbYrbcCKBDsC","XasdXLL99y5rygCmq","3gkSihZQRfAD2KBo3","t5XMbyZbtPBo4wBGS","bEpHM2CtrwW2qE8FP","teJzFLHexaz5AQW5z"]}