CompactNets: Compact Hierarchical Compositional Networks for Visual Recognition. Lobel, H., Vidal, R., & Soto, A. Computer Vision and Image Understanding, 191:102841, 2020. Paper doi abstract bibtex 2 downloads CNN-based models currently provide state-of-the-art performance in image categorization tasks. While these methods are powerful in terms of representational capacity, they are generally not conceived with explicit means to control complexity. This might lead to scenarios where resources are used in a non-optimal manner, increasing the number of unspecialized or repeated neurons, and overfitting to data. In this work we propose CompactNets, a new approach to visual recognition that learns a hierarchy of shared, discriminative, specialized, and compact representations. CompactNets naturally capture the notion of compositional compactness, a characterization of complexity in compositional models, consisting on using the smallest number of patterns to build a suitable visual representation. We employ a structural regularizer with group-sparse terms in the objective function, that induces on each layer, an efficient and effective use of elements from the layer below. In particular, this allows groups of top-level features to be specialized based on category information. We evaluate CompactNets on the ILSVRC12 dataset, obtaining compact representations and competitive performance, using an order of magnitude less parameters than common CNN-based approaches. We show that CompactNets are able to outperform other group-sparse-based approaches, in terms of performance and compactness. Finally, transfer-learning experiments on small-scale datasets demonstrate high generalization power, providing remarkable categorization performance with respect to alternative approaches.
@Article{ lobel2020102841,
title = {CompactNets: Compact Hierarchical Compositional Networks
for Visual Recognition},
journal = {Computer Vision and Image Understanding},
volume = {191},
pages = {102841},
year = {2020},
issn = {1077-3142},
doi = {https://doi.org/10.1016/j.cviu.2019.102841},
url = {https://www.sciencedirect.com/science/article/pii/S1077314218301905},
author = {Hans Lobel and René Vidal and Alvaro Soto},
keywords = {Deep learning, Regularization, Group sparsity, Image
categorization},
abstract = {CNN-based models currently provide state-of-the-art
performance in image categorization tasks. While these
methods are powerful in terms of representational capacity,
they are generally not conceived with explicit means to
control complexity. This might lead to scenarios where
resources are used in a non-optimal manner, increasing the
number of unspecialized or repeated neurons, and
overfitting to data. In this work we propose CompactNets, a
new approach to visual recognition that learns a hierarchy
of shared, discriminative, specialized, and compact
representations. CompactNets naturally capture the notion
of compositional compactness, a characterization of
complexity in compositional models, consisting on using the
smallest number of patterns to build a suitable visual
representation. We employ a structural regularizer with
group-sparse terms in the objective function, that induces
on each layer, an efficient and effective use of elements
from the layer below. In particular, this allows groups of
top-level features to be specialized based on category
information. We evaluate CompactNets on the ILSVRC12
dataset, obtaining compact representations and competitive
performance, using an order of magnitude less parameters
than common CNN-based approaches. We show that CompactNets
are able to outperform other group-sparse-based approaches,
in terms of performance and compactness. Finally,
transfer-learning experiments on small-scale datasets
demonstrate high generalization power, providing remarkable
categorization performance with respect to alternative
approaches.}
}
Downloads: 2
{"_id":"o3yXPyMysm6YnresZ","bibbaseid":"lobel-vidal-soto-compactnetscompacthierarchicalcompositionalnetworksforvisualrecognition-2020","authorIDs":["GbYBJvxugXMriQwbi","LpqQBhFH3PxepH9KY","SpKJ5YujbHKZnHc4v","WfZbctNQYDBaiYW6n","Xk2Q5qedS5MFHvjEW","dGRBfr8zhMmbwK6eP","g6iKCQCFnJgKYYHaP","h2hTcQYuf2PB3oF8t","jAtuJBcGhng4Lq2Nd","uC8ATA8AfngWpYLBq","zCbPxKnQGgDHiHMWn"],"author_short":["Lobel, H.","Vidal, R.","Soto, A."],"bibdata":{"bibtype":"article","type":"article","title":"CompactNets: Compact Hierarchical Compositional Networks for Visual Recognition","journal":"Computer Vision and Image Understanding","volume":"191","pages":"102841","year":"2020","issn":"1077-3142","doi":"https://doi.org/10.1016/j.cviu.2019.102841","url":"https://www.sciencedirect.com/science/article/pii/S1077314218301905","author":[{"firstnames":["Hans"],"propositions":[],"lastnames":["Lobel"],"suffixes":[]},{"firstnames":["René"],"propositions":[],"lastnames":["Vidal"],"suffixes":[]},{"firstnames":["Alvaro"],"propositions":[],"lastnames":["Soto"],"suffixes":[]}],"keywords":"Deep learning, Regularization, Group sparsity, Image categorization","abstract":"CNN-based models currently provide state-of-the-art performance in image categorization tasks. While these methods are powerful in terms of representational capacity, they are generally not conceived with explicit means to control complexity. This might lead to scenarios where resources are used in a non-optimal manner, increasing the number of unspecialized or repeated neurons, and overfitting to data. In this work we propose CompactNets, a new approach to visual recognition that learns a hierarchy of shared, discriminative, specialized, and compact representations. CompactNets naturally capture the notion of compositional compactness, a characterization of complexity in compositional models, consisting on using the smallest number of patterns to build a suitable visual representation. We employ a structural regularizer with group-sparse terms in the objective function, that induces on each layer, an efficient and effective use of elements from the layer below. In particular, this allows groups of top-level features to be specialized based on category information. We evaluate CompactNets on the ILSVRC12 dataset, obtaining compact representations and competitive performance, using an order of magnitude less parameters than common CNN-based approaches. We show that CompactNets are able to outperform other group-sparse-based approaches, in terms of performance and compactness. Finally, transfer-learning experiments on small-scale datasets demonstrate high generalization power, providing remarkable categorization performance with respect to alternative approaches.","bibtex":"@Article{\t lobel2020102841,\n title\t\t= {CompactNets: Compact Hierarchical Compositional Networks\n\t\t for Visual Recognition},\n journal\t= {Computer Vision and Image Understanding},\n volume\t= {191},\n pages\t\t= {102841},\n year\t\t= {2020},\n issn\t\t= {1077-3142},\n doi\t\t= {https://doi.org/10.1016/j.cviu.2019.102841},\n url\t\t= {https://www.sciencedirect.com/science/article/pii/S1077314218301905},\n author\t= {Hans Lobel and René Vidal and Alvaro Soto},\n keywords\t= {Deep learning, Regularization, Group sparsity, Image\n\t\t categorization},\n abstract\t= {CNN-based models currently provide state-of-the-art\n\t\t performance in image categorization tasks. While these\n\t\t methods are powerful in terms of representational capacity,\n\t\t they are generally not conceived with explicit means to\n\t\t control complexity. This might lead to scenarios where\n\t\t resources are used in a non-optimal manner, increasing the\n\t\t number of unspecialized or repeated neurons, and\n\t\t overfitting to data. In this work we propose CompactNets, a\n\t\t new approach to visual recognition that learns a hierarchy\n\t\t of shared, discriminative, specialized, and compact\n\t\t representations. CompactNets naturally capture the notion\n\t\t of compositional compactness, a characterization of\n\t\t complexity in compositional models, consisting on using the\n\t\t smallest number of patterns to build a suitable visual\n\t\t representation. We employ a structural regularizer with\n\t\t group-sparse terms in the objective function, that induces\n\t\t on each layer, an efficient and effective use of elements\n\t\t from the layer below. In particular, this allows groups of\n\t\t top-level features to be specialized based on category\n\t\t information. We evaluate CompactNets on the ILSVRC12\n\t\t dataset, obtaining compact representations and competitive\n\t\t performance, using an order of magnitude less parameters\n\t\t than common CNN-based approaches. We show that CompactNets\n\t\t are able to outperform other group-sparse-based approaches,\n\t\t in terms of performance and compactness. Finally,\n\t\t transfer-learning experiments on small-scale datasets\n\t\t demonstrate high generalization power, providing remarkable\n\t\t categorization performance with respect to alternative\n\t\t approaches.}\n}\n\n","author_short":["Lobel, H.","Vidal, R.","Soto, A."],"key":"lobel2020102841","id":"lobel2020102841","bibbaseid":"lobel-vidal-soto-compactnetscompacthierarchicalcompositionalnetworksforvisualrecognition-2020","role":"author","urls":{"Paper":"https://www.sciencedirect.com/science/article/pii/S1077314218301905"},"keyword":["Deep learning","Regularization","Group sparsity","Image categorization"],"metadata":{"authorlinks":{"soto, a":"https://asoto.ing.puc.cl/publications/"}},"downloads":2},"bibtype":"article","biburl":"https://raw.githubusercontent.com/ialab-puc/ialab.ing.puc.cl/master/pubs.bib","creationDate":"2020-04-27T04:09:41.472Z","downloads":2,"keywords":["deep learning","regularization","group sparsity","image categorization"],"search_terms":["compactnets","compact","hierarchical","compositional","networks","visual","recognition","lobel","vidal","soto"],"title":"CompactNets: Compact Hierarchical Compositional Networks for Visual Recognition","year":2020,"dataSources":["3YPRCmmijLqF4qHXd","sg6yZ29Z2xB5xP79R","m8qFBfFbjk9qWjcmJ","QjT2DEZoWmQYxjHXS"]}