Counting Apples and Oranges With Deep Learning: A Data-Driven Approach. Chen, S. W., Shivakumar, S. S., Dcunha, S., Das, J., Okon, E., Qu, C., Taylor, C. J., & Kumar, V. IEEE Robotics and Automation Letters, 2(2):781–788, April, 2017.
doi  abstract   bibtex   
This paper describes a fruit counting pipeline based on deep learning that accurately counts fruit in unstructured environments. Obtaining reliable fruit counts is challenging because of variations in appearance due to illumination changes and occlusions from foliage and neighboring fruits. We propose a novel approach that uses deep learning to map from input images to total fruit counts. The pipeline utilizes a custom crowdsourcing platform to quickly label large data sets. A blob detector based on a fully convolutional network extracts candidate regions in the images. A counting algorithm based on a second convolutional network then estimates the number of fruits in each region. Finally, a linear regression model maps that fruit count estimate to a final fruit count. We analyze the performance of the pipeline on two distinct data sets of oranges in daylight, and green apples at night, utilizing human generated labels as ground truth. We also show that the pipeline has a short training time and performs well with a limited data set size. Our method generalizes across both data sets and is able to perform well even on highly occluded fruits that are challenging for human labelers to annotate.
@article{chen_counting_2017,
	title = {Counting {Apples} and {Oranges} {With} {Deep} {Learning}: {A} {Data}-{Driven} {Approach}},
	volume = {2},
	shorttitle = {Counting {Apples} and {Oranges} {With} {Deep} {Learning}},
	doi = {10.1109/LRA.2017.2651944},
	abstract = {This paper describes a fruit counting pipeline based on deep learning that accurately counts fruit in unstructured environments. Obtaining reliable fruit counts is challenging because of variations in appearance due to illumination changes and occlusions from foliage and neighboring fruits. We propose a novel approach that uses deep learning to map from input images to total fruit counts. The pipeline utilizes a custom crowdsourcing platform to quickly label large data sets. A blob detector based on a fully convolutional network extracts candidate regions in the images. A counting algorithm based on a second convolutional network then estimates the number of fruits in each region. Finally, a linear regression model maps that fruit count estimate to a final fruit count. We analyze the performance of the pipeline on two distinct data sets of oranges in daylight, and green apples at night, utilizing human generated labels as ground truth. We also show that the pipeline has a short training time and performs well with a limited data set size. Our method generalizes across both data sets and is able to perform well even on highly occluded fruits that are challenging for human labelers to annotate.},
	number = {2},
	journal = {IEEE Robotics and Automation Letters},
	author = {Chen, S. W. and Shivakumar, S. S. and Dcunha, S. and Das, J. and Okon, E. and Qu, C. and Taylor, C. J. and Kumar, V.},
	month = apr,
	year = {2017},
	keywords = {Agricultural automation, Image segmentation, Labeling, Lighting, Machine learning, Neural networks, Pipelines, Training, agricultural automation, agricultural products, apple counting, blob detector, candidate region extraction, categorization, counting algorithm, crowdsourcing, crowdsourcing platform, data-driven approach, feature extraction, feedforward neural nets, fruit counting pipeline, fully convolutional network, green apples, human generated label utilization, illumination changes, large data sets, learning (artificial intelligence), linear regression model, object detection, occlusions, orange counting, performance analysis, performance evaluation, regression analysis, segmentation, visual learning},
	pages = {781--788}
}
Downloads: 0