Guided Event Filtering: Synergy between Intensity Images and Neuromorphic Events for High Performance Imaging. Duan, P., Wang, Z., Shi, B., Cossairt, O., Huang, T., & Katsaggelos, A. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(11):1–1, 2021. ![Guided Event Filtering: Synergy between Intensity Images and Neuromorphic Events for High Performance Imaging [link]](https://bibbase.org/img/filetypes/link.svg "link")
Paper doi abstract bibtex Many visual and robotics tasks in real-world scenarios rely on robust handling of high speed motion and high dynamic range (HDR) with effectively high spatial resolution and low noise. Such stringent requirements, however, cannot be directly satisfied by a single imager or imaging modality, rather by multi-modal sensors with complementary advantages. In this paper, we address high performance imaging by exploring the synergy between traditional frame-based sensors with high spatial resolution and low sensor noise, and emerging event-based sensors with high speed and high dynamic range. We introduce a novel computational framework, termed Guided Event Filtering (GEF), to process these two streams of input data and output a stream of super-resolved yet noise-reduced events. To generate high quality events, GEF first registers the captured noisy events onto the guidance image plane according to our flow model. it then performs joint image filtering that inherits the mutual structure from both inputs. Lastly, GEF re-distributes the filtered event frame in the space-time volume while preserving the statistical characteristics of the original events. When the guidance images under-perform, GEF incorporates an event self-guiding mechanism that resorts to neighbor events for guidance. We demonstrate the benefits of GEF by applying the output high quality events to existing event-based algorithms across diverse application categories, including high speed object tracking, depth estimation, high frame-rate video synthesis, and super resolution/HDR/color image restoration.
@article{Peiqi2021,
abstract = {Many visual and robotics tasks in real-world scenarios rely on robust handling of high speed motion and high dynamic range (HDR) with effectively high spatial resolution and low noise. Such stringent requirements, however, cannot be directly satisfied by a single imager or imaging modality, rather by multi-modal sensors with complementary advantages. In this paper, we address high performance imaging by exploring the synergy between traditional frame-based sensors with high spatial resolution and low sensor noise, and emerging event-based sensors with high speed and high dynamic range. We introduce a novel computational framework, termed Guided Event Filtering (GEF), to process these two streams of input data and output a stream of super-resolved yet noise-reduced events. To generate high quality events, GEF first registers the captured noisy events onto the guidance image plane according to our flow model. it then performs joint image filtering that inherits the mutual structure from both inputs. Lastly, GEF re-distributes the filtered event frame in the space-time volume while preserving the statistical characteristics of the original events. When the guidance images under-perform, GEF incorporates an event self-guiding mechanism that resorts to neighbor events for guidance. We demonstrate the benefits of GEF by applying the output high quality events to existing event-based algorithms across diverse application categories, including high speed object tracking, depth estimation, high frame-rate video synthesis, and super resolution/HDR/color image restoration.},
author = {Duan, Peiqi and Wang, Zihao and Shi, Boxin and Cossairt, Oliver and Huang, Tiejun and Katsaggelos, Aggelos},
doi = {10.1109/TPAMI.2021.3113344},
issn = {0162-8828},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
keywords = {Computational hybrid cameras,event-based imaging and vision,joint filtering},
number = {11},
pages = {1--1},
pmid = {34543190},
title = {{Guided Event Filtering: Synergy between Intensity Images and Neuromorphic Events for High Performance Imaging}},
url = {https://ieeexplore.ieee.org/document/9541050/},
volume = {44},
year = {2021}
}
Downloads: 0
{"_id":"LQwpHyexBmgJypHHj","bibbaseid":"duan-wang-shi-cossairt-huang-katsaggelos-guidedeventfilteringsynergybetweenintensityimagesandneuromorphiceventsforhighperformanceimaging-2021","author_short":["Duan, P.","Wang, Z.","Shi, B.","Cossairt, O.","Huang, T.","Katsaggelos, A."],"bibdata":{"bibtype":"article","type":"article","abstract":"Many visual and robotics tasks in real-world scenarios rely on robust handling of high speed motion and high dynamic range (HDR) with effectively high spatial resolution and low noise. Such stringent requirements, however, cannot be directly satisfied by a single imager or imaging modality, rather by multi-modal sensors with complementary advantages. In this paper, we address high performance imaging by exploring the synergy between traditional frame-based sensors with high spatial resolution and low sensor noise, and emerging event-based sensors with high speed and high dynamic range. We introduce a novel computational framework, termed Guided Event Filtering (GEF), to process these two streams of input data and output a stream of super-resolved yet noise-reduced events. To generate high quality events, GEF first registers the captured noisy events onto the guidance image plane according to our flow model. it then performs joint image filtering that inherits the mutual structure from both inputs. Lastly, GEF re-distributes the filtered event frame in the space-time volume while preserving the statistical characteristics of the original events. When the guidance images under-perform, GEF incorporates an event self-guiding mechanism that resorts to neighbor events for guidance. We demonstrate the benefits of GEF by applying the output high quality events to existing event-based algorithms across diverse application categories, including high speed object tracking, depth estimation, high frame-rate video synthesis, and super resolution/HDR/color image restoration.","author":[{"propositions":[],"lastnames":["Duan"],"firstnames":["Peiqi"],"suffixes":[]},{"propositions":[],"lastnames":["Wang"],"firstnames":["Zihao"],"suffixes":[]},{"propositions":[],"lastnames":["Shi"],"firstnames":["Boxin"],"suffixes":[]},{"propositions":[],"lastnames":["Cossairt"],"firstnames":["Oliver"],"suffixes":[]},{"propositions":[],"lastnames":["Huang"],"firstnames":["Tiejun"],"suffixes":[]},{"propositions":[],"lastnames":["Katsaggelos"],"firstnames":["Aggelos"],"suffixes":[]}],"doi":"10.1109/TPAMI.2021.3113344","issn":"0162-8828","journal":"IEEE Transactions on Pattern Analysis and Machine Intelligence","keywords":"Computational hybrid cameras,event-based imaging and vision,joint filtering","number":"11","pages":"1–1","pmid":"34543190","title":"Guided Event Filtering: Synergy between Intensity Images and Neuromorphic Events for High Performance Imaging","url":"https://ieeexplore.ieee.org/document/9541050/","volume":"44","year":"2021","bibtex":"@article{Peiqi2021,\nabstract = {Many visual and robotics tasks in real-world scenarios rely on robust handling of high speed motion and high dynamic range (HDR) with effectively high spatial resolution and low noise. Such stringent requirements, however, cannot be directly satisfied by a single imager or imaging modality, rather by multi-modal sensors with complementary advantages. In this paper, we address high performance imaging by exploring the synergy between traditional frame-based sensors with high spatial resolution and low sensor noise, and emerging event-based sensors with high speed and high dynamic range. We introduce a novel computational framework, termed Guided Event Filtering (GEF), to process these two streams of input data and output a stream of super-resolved yet noise-reduced events. To generate high quality events, GEF first registers the captured noisy events onto the guidance image plane according to our flow model. it then performs joint image filtering that inherits the mutual structure from both inputs. Lastly, GEF re-distributes the filtered event frame in the space-time volume while preserving the statistical characteristics of the original events. When the guidance images under-perform, GEF incorporates an event self-guiding mechanism that resorts to neighbor events for guidance. We demonstrate the benefits of GEF by applying the output high quality events to existing event-based algorithms across diverse application categories, including high speed object tracking, depth estimation, high frame-rate video synthesis, and super resolution/HDR/color image restoration.},\nauthor = {Duan, Peiqi and Wang, Zihao and Shi, Boxin and Cossairt, Oliver and Huang, Tiejun and Katsaggelos, Aggelos},\ndoi = {10.1109/TPAMI.2021.3113344},\nissn = {0162-8828},\njournal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},\nkeywords = {Computational hybrid cameras,event-based imaging and vision,joint filtering},\nnumber = {11},\npages = {1--1},\npmid = {34543190},\ntitle = {{Guided Event Filtering: Synergy between Intensity Images and Neuromorphic Events for High Performance Imaging}},\nurl = {https://ieeexplore.ieee.org/document/9541050/},\nvolume = {44},\nyear = {2021}\n}\n","author_short":["Duan, P.","Wang, Z.","Shi, B.","Cossairt, O.","Huang, T.","Katsaggelos, A."],"key":"Peiqi2021","id":"Peiqi2021","bibbaseid":"duan-wang-shi-cossairt-huang-katsaggelos-guidedeventfilteringsynergybetweenintensityimagesandneuromorphiceventsforhighperformanceimaging-2021","role":"author","urls":{"Paper":"https://ieeexplore.ieee.org/document/9541050/"},"keyword":["Computational hybrid cameras","event-based imaging and vision","joint filtering"],"metadata":{"authorlinks":{}}},"bibtype":"article","biburl":"https://sites.northwestern.edu/ivpl/files/2023/06/IVPL_Updated_publications-1.bib","dataSources":["ePKPjG8C6yvpk4mEK","ya2CyA73rpZseyrZ8","zFPgsTDAW8aDnb5iN","E6Bth2QB5BYjBMZE7","nbnEjsN7MJhurAK9x","PNQZj6FjzoxxJk4Yi","7FpDWDGJ4KgpDiGfB","bod9ms4MQJHuJgPpp","QR9t5P2cLdJuzhfzK","D8k2SxfC5dKNRFgro","7Dwzbxq93HWrJEhT6","qhF8zxmGcJfvtdeAg","fvDEHD49E2ZRwE3fb","H7crv8NWhZup4d4by","DHqokWsryttGh7pJE","vRJd4wNg9HpoZSMHD","sYxQ6pxFgA59JRhxi","w2WahSbYrbcCKBDsC","XasdXLL99y5rygCmq","3gkSihZQRfAD2KBo3","t5XMbyZbtPBo4wBGS","bEpHM2CtrwW2qE8FP","teJzFLHexaz5AQW5z"],"keywords":["computational hybrid cameras","event-based imaging and vision","joint filtering"],"search_terms":["guided","event","filtering","synergy","between","intensity","images","neuromorphic","events","high","performance","imaging","duan","wang","shi","cossairt","huang","katsaggelos"],"title":"Guided Event Filtering: Synergy between Intensity Images and Neuromorphic Events for High Performance Imaging","year":2021}