Face2Face: Real-Time Face Capture and Reenactment of RGB Videos. Thies, J., Zollhofer, M., Stamminger, M., Theobalt, C., & NieBner, M. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2387–2395, Las Vegas, NV, USA, June, 2016. IEEE. ![Face2Face: Real-Time Face Capture and Reenactment of RGB Videos [link]](https://bibbase.org/img/filetypes/link.svg "link")
Paper doi abstract bibtex We present a novel approach for real-time facial reenactment of a monocular target video sequence (e.g., Youtube video). The source sequence is also a monocular video stream, captured live with a commodity webcam. Our goal is to animate the facial expressions of the target video by a source actor and re-render the manipulated output video in a photo-realistic fashion. To this end, we first address the under-constrained problem of facial identity recovery from monocular video by non-rigid model-based bundling. At run time, we track facial expressions of both source and target video using a dense photometric consistency measure. Reenactment is then achieved by fast and efficient deformation transfer between source and target. The mouth interior that best matches the re-targeted expression is retrieved from the target sequence and warped to produce an accurate fit. Finally, we convincingly re-render the synthesized target face on top of the corresponding video stream such that it seamlessly blends with the real-world illumination. We demonstrate our method in a live setup, where Youtube videos are reenacted in real time.
@inproceedings{thies_face2face:_2016,
address = {Las Vegas, NV, USA},
title = {{Face2Face}: {Real}-{Time} {Face} {Capture} and {Reenactment} of {RGB} {Videos}},
isbn = {978-1-4673-8851-1},
shorttitle = {{Face2Face}},
url = {http://ieeexplore.ieee.org/document/7780631/},
doi = {10.1109/CVPR.2016.262},
abstract = {We present a novel approach for real-time facial reenactment of a monocular target video sequence (e.g., Youtube video). The source sequence is also a monocular video stream, captured live with a commodity webcam. Our goal is to animate the facial expressions of the target video by a source actor and re-render the manipulated output video in a photo-realistic fashion. To this end, we first address the under-constrained problem of facial identity recovery from monocular video by non-rigid model-based bundling. At run time, we track facial expressions of both source and target video using a dense photometric consistency measure. Reenactment is then achieved by fast and efficient deformation transfer between source and target. The mouth interior that best matches the re-targeted expression is retrieved from the target sequence and warped to produce an accurate fit. Finally, we convincingly re-render the synthesized target face on top of the corresponding video stream such that it seamlessly blends with the real-world illumination. We demonstrate our method in a live setup, where Youtube videos are reenacted in real time.},
language = {en},
urldate = {2018-10-02},
booktitle = {2016 {IEEE} {Conference} on {Computer} {Vision} and {Pattern} {Recognition} ({CVPR})},
publisher = {IEEE},
author = {Thies, Justus and Zollhofer, Michael and Stamminger, Marc and Theobalt, Christian and NieBner, Matthias},
month = jun,
year = {2016},
keywords = {graphics},
pages = {2387--2395}
}
Downloads: 0
{"_id":"eQjJwMG3Fryswdqmf","bibbaseid":"thies-zollhofer-stamminger-theobalt-niebner-face2facerealtimefacecaptureandreenactmentofrgbvideos-2016","authorIDs":[],"author_short":["Thies, J.","Zollhofer, M.","Stamminger, M.","Theobalt, C.","NieBner, M."],"bibdata":{"bibtype":"inproceedings","type":"inproceedings","address":"Las Vegas, NV, USA","title":"Face2Face: Real-Time Face Capture and Reenactment of RGB Videos","isbn":"978-1-4673-8851-1","shorttitle":"Face2Face","url":"http://ieeexplore.ieee.org/document/7780631/","doi":"10.1109/CVPR.2016.262","abstract":"We present a novel approach for real-time facial reenactment of a monocular target video sequence (e.g., Youtube video). The source sequence is also a monocular video stream, captured live with a commodity webcam. Our goal is to animate the facial expressions of the target video by a source actor and re-render the manipulated output video in a photo-realistic fashion. To this end, we first address the under-constrained problem of facial identity recovery from monocular video by non-rigid model-based bundling. At run time, we track facial expressions of both source and target video using a dense photometric consistency measure. Reenactment is then achieved by fast and efficient deformation transfer between source and target. The mouth interior that best matches the re-targeted expression is retrieved from the target sequence and warped to produce an accurate fit. Finally, we convincingly re-render the synthesized target face on top of the corresponding video stream such that it seamlessly blends with the real-world illumination. We demonstrate our method in a live setup, where Youtube videos are reenacted in real time.","language":"en","urldate":"2018-10-02","booktitle":"2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","publisher":"IEEE","author":[{"propositions":[],"lastnames":["Thies"],"firstnames":["Justus"],"suffixes":[]},{"propositions":[],"lastnames":["Zollhofer"],"firstnames":["Michael"],"suffixes":[]},{"propositions":[],"lastnames":["Stamminger"],"firstnames":["Marc"],"suffixes":[]},{"propositions":[],"lastnames":["Theobalt"],"firstnames":["Christian"],"suffixes":[]},{"propositions":[],"lastnames":["NieBner"],"firstnames":["Matthias"],"suffixes":[]}],"month":"June","year":"2016","keywords":"graphics","pages":"2387–2395","bibtex":"@inproceedings{thies_face2face:_2016,\n\taddress = {Las Vegas, NV, USA},\n\ttitle = {{Face2Face}: {Real}-{Time} {Face} {Capture} and {Reenactment} of {RGB} {Videos}},\n\tisbn = {978-1-4673-8851-1},\n\tshorttitle = {{Face2Face}},\n\turl = {http://ieeexplore.ieee.org/document/7780631/},\n\tdoi = {10.1109/CVPR.2016.262},\n\tabstract = {We present a novel approach for real-time facial reenactment of a monocular target video sequence (e.g., Youtube video). The source sequence is also a monocular video stream, captured live with a commodity webcam. Our goal is to animate the facial expressions of the target video by a source actor and re-render the manipulated output video in a photo-realistic fashion. To this end, we first address the under-constrained problem of facial identity recovery from monocular video by non-rigid model-based bundling. At run time, we track facial expressions of both source and target video using a dense photometric consistency measure. Reenactment is then achieved by fast and efficient deformation transfer between source and target. The mouth interior that best matches the re-targeted expression is retrieved from the target sequence and warped to produce an accurate fit. Finally, we convincingly re-render the synthesized target face on top of the corresponding video stream such that it seamlessly blends with the real-world illumination. We demonstrate our method in a live setup, where Youtube videos are reenacted in real time.},\n\tlanguage = {en},\n\turldate = {2018-10-02},\n\tbooktitle = {2016 {IEEE} {Conference} on {Computer} {Vision} and {Pattern} {Recognition} ({CVPR})},\n\tpublisher = {IEEE},\n\tauthor = {Thies, Justus and Zollhofer, Michael and Stamminger, Marc and Theobalt, Christian and NieBner, Matthias},\n\tmonth = jun,\n\tyear = {2016},\n\tkeywords = {graphics},\n\tpages = {2387--2395}\n}\n\n","author_short":["Thies, J.","Zollhofer, M.","Stamminger, M.","Theobalt, C.","NieBner, M."],"key":"thies_face2face:_2016","id":"thies_face2face:_2016","bibbaseid":"thies-zollhofer-stamminger-theobalt-niebner-face2facerealtimefacecaptureandreenactmentofrgbvideos-2016","role":"author","urls":{"Paper":"http://ieeexplore.ieee.org/document/7780631/"},"keyword":["graphics"],"downloads":0,"html":""},"bibtype":"inproceedings","biburl":"https://bibbase.org/zotero/21h","creationDate":"2020-05-22T18:11:30.464Z","downloads":0,"keywords":["av","graphics"],"search_terms":["face2face","real","time","face","capture","reenactment","rgb","videos","thies","zollhofer","stamminger","theobalt","niebner"],"title":"Face2Face: Real-Time Face Capture and Reenactment of RGB Videos","year":2016,"dataSources":["NpxDwkFvCjNmGYFSj"]}