Online kernel principal component analysis: a reduced-order model. Honeine, P. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(9):1814 - 1826, September, 2012. Link Paper doi abstract bibtex Kernel principal component analysis (kernel-PCA) is an elegant nonlinear extension of one of the most used data analysis and dimensionality reduction techniques, the principal component analysis. In this paper, we propose an online algorithm for kernel-PCA. To this end, we examine a kernel-based version of Oja's rule, initially put forward to extract a linear principal axe. As with most kernel-based machines, the model order equals the number of available observations. To provide an online scheme, we propose to control the model order. We discuss theoretical results, such as an upper bound on the error of approximating the principal functions with the reduced-order model. We derive a recursive algorithm to discover the first principal axis, and extend it to multiple axes. Experimental results demonstrate the effectiveness of the proposed approach, both on synthetic data set and on images of handwritten digits, with comparison to classical kernel-PCA and iterative kernel-PCA.
@ARTICLE{12.tpami,
author = "Paul Honeine",
title = "Online kernel principal component analysis: a reduced-order model",
journal = "IEEE Transactions on Pattern Analysis and Machine Intelligence",
year = "2012",
volume = "34",
number = "9",
pages = "1814 - 1826",
month = sep,
url_link= "https://ieeexplore.ieee.org/document/6112772",
doi = "10.1109/TPAMI.2011.270",
url_paper = "http://www.honeine.fr/paul/publi/12.tpami.onlineKPCA.pdf",
keywords = "non-stationarity, machine learning, pre-image problem, sparsity, adaptive filtering, data analysis, function approximation, principal component analysis, reduced order systems, online kernel principal component analysis, reduced-order model, data analysis, dimensionality reduction techniques, online algorithm, Oja rule, linear principal axe extraction, kernel-based machines, principal function approximation, synthetic data set, handwritten digit image, classical kernel-PCA, iterative kernel-PCA, Kernel, Principal component analysis, Eigenvalues and eigenfunctions, Dictionaries, Algorithm design and analysis, Data models, Training data, Principal component analysis, online algorithm, machine learning, reproducing kernel, Oja's rule, recursive algorithm.",
abstract={Kernel principal component analysis (kernel-PCA) is an elegant nonlinear extension of one of the most used data analysis and dimensionality reduction techniques, the principal component analysis. In this paper, we propose an online algorithm for kernel-PCA. To this end, we examine a kernel-based version of Oja's rule, initially put forward to extract a linear principal axe. As with most kernel-based machines, the model order equals the number of available observations. To provide an online scheme, we propose to control the model order. We discuss theoretical results, such as an upper bound on the error of approximating the principal functions with the reduced-order model. We derive a recursive algorithm to discover the first principal axis, and extend it to multiple axes. Experimental results demonstrate the effectiveness of the proposed approach, both on synthetic data set and on images of handwritten digits, with comparison to classical kernel-PCA and iterative kernel-PCA.},
}
Downloads: 0
{"_id":"ninJBDoLX65qKRJow","bibbaseid":"honeine-onlinekernelprincipalcomponentanalysisareducedordermodel-2012","author_short":["Honeine, P."],"bibdata":{"bibtype":"article","type":"article","author":[{"firstnames":["Paul"],"propositions":[],"lastnames":["Honeine"],"suffixes":[]}],"title":"Online kernel principal component analysis: a reduced-order model","journal":"IEEE Transactions on Pattern Analysis and Machine Intelligence","year":"2012","volume":"34","number":"9","pages":"1814 - 1826","month":"September","url_link":"https://ieeexplore.ieee.org/document/6112772","doi":"10.1109/TPAMI.2011.270","url_paper":"http://www.honeine.fr/paul/publi/12.tpami.onlineKPCA.pdf","keywords":"non-stationarity, machine learning, pre-image problem, sparsity, adaptive filtering, data analysis, function approximation, principal component analysis, reduced order systems, online kernel principal component analysis, reduced-order model, data analysis, dimensionality reduction techniques, online algorithm, Oja rule, linear principal axe extraction, kernel-based machines, principal function approximation, synthetic data set, handwritten digit image, classical kernel-PCA, iterative kernel-PCA, Kernel, Principal component analysis, Eigenvalues and eigenfunctions, Dictionaries, Algorithm design and analysis, Data models, Training data, Principal component analysis, online algorithm, machine learning, reproducing kernel, Oja's rule, recursive algorithm.","abstract":"Kernel principal component analysis (kernel-PCA) is an elegant nonlinear extension of one of the most used data analysis and dimensionality reduction techniques, the principal component analysis. In this paper, we propose an online algorithm for kernel-PCA. To this end, we examine a kernel-based version of Oja's rule, initially put forward to extract a linear principal axe. As with most kernel-based machines, the model order equals the number of available observations. To provide an online scheme, we propose to control the model order. We discuss theoretical results, such as an upper bound on the error of approximating the principal functions with the reduced-order model. We derive a recursive algorithm to discover the first principal axis, and extend it to multiple axes. Experimental results demonstrate the effectiveness of the proposed approach, both on synthetic data set and on images of handwritten digits, with comparison to classical kernel-PCA and iterative kernel-PCA.","bibtex":"@ARTICLE{12.tpami,\n author = \"Paul Honeine\",\n title = \"Online kernel principal component analysis: a reduced-order model\",\n journal = \"IEEE Transactions on Pattern Analysis and Machine Intelligence\",\n year = \"2012\",\n volume = \"34\",\n number = \"9\",\n pages = \"1814 - 1826\",\n month = sep,\n url_link= \"https://ieeexplore.ieee.org/document/6112772\",\n doi = \"10.1109/TPAMI.2011.270\",\n url_paper = \"http://www.honeine.fr/paul/publi/12.tpami.onlineKPCA.pdf\",\n keywords = \"non-stationarity, machine learning, pre-image problem, sparsity, adaptive filtering, data analysis, function approximation, principal component analysis, reduced order systems, online kernel principal component analysis, reduced-order model, data analysis, dimensionality reduction techniques, online algorithm, Oja rule, linear principal axe extraction, kernel-based machines, principal function approximation, synthetic data set, handwritten digit image, classical kernel-PCA, iterative kernel-PCA, Kernel, Principal component analysis, Eigenvalues and eigenfunctions, Dictionaries, Algorithm design and analysis, Data models, Training data, Principal component analysis, online algorithm, machine learning, reproducing kernel, Oja's rule, recursive algorithm.\",\n abstract={Kernel principal component analysis (kernel-PCA) is an elegant nonlinear extension of one of the most used data analysis and dimensionality reduction techniques, the principal component analysis. In this paper, we propose an online algorithm for kernel-PCA. To this end, we examine a kernel-based version of Oja's rule, initially put forward to extract a linear principal axe. As with most kernel-based machines, the model order equals the number of available observations. To provide an online scheme, we propose to control the model order. We discuss theoretical results, such as an upper bound on the error of approximating the principal functions with the reduced-order model. We derive a recursive algorithm to discover the first principal axis, and extend it to multiple axes. Experimental results demonstrate the effectiveness of the proposed approach, both on synthetic data set and on images of handwritten digits, with comparison to classical kernel-PCA and iterative kernel-PCA.}, \n}\n","author_short":["Honeine, P."],"key":"12.tpami","id":"12.tpami","bibbaseid":"honeine-onlinekernelprincipalcomponentanalysisareducedordermodel-2012","role":"author","urls":{" link":"https://ieeexplore.ieee.org/document/6112772"," paper":"http://www.honeine.fr/paul/publi/12.tpami.onlineKPCA.pdf"},"keyword":["non-stationarity","machine learning","pre-image problem","sparsity","adaptive filtering","data analysis","function approximation","principal component analysis","reduced order systems","online kernel principal component analysis","reduced-order model","data analysis","dimensionality reduction techniques","online algorithm","Oja rule","linear principal axe extraction","kernel-based machines","principal function approximation","synthetic data set","handwritten digit image","classical kernel-PCA","iterative kernel-PCA","Kernel","Principal component analysis","Eigenvalues and eigenfunctions","Dictionaries","Algorithm design and analysis","Data models","Training data","Principal component analysis","online algorithm","machine learning","reproducing kernel","Oja's rule","recursive algorithm."],"metadata":{"authorlinks":{}},"html":""},"bibtype":"article","biburl":"http://honeine.fr/paul/biblio_ph.bib","dataSources":["DsERGQxgYm5hGq3CY"],"keywords":["non-stationarity","machine learning","pre-image problem","sparsity","adaptive filtering","data analysis","function approximation","principal component analysis","reduced order systems","online kernel principal component analysis","reduced-order model","data analysis","dimensionality reduction techniques","online algorithm","oja rule","linear principal axe extraction","kernel-based machines","principal function approximation","synthetic data set","handwritten digit image","classical kernel-pca","iterative kernel-pca","kernel","principal component analysis","eigenvalues and eigenfunctions","dictionaries","algorithm design and analysis","data models","training data","principal component analysis","online algorithm","machine learning","reproducing kernel","oja's rule","recursive algorithm."],"search_terms":["online","kernel","principal","component","analysis","reduced","order","model","honeine"],"title":"Online kernel principal component analysis: a reduced-order model","year":2012}