Odd Leaf Out. Hansen, D., Lewis, D., Rotman, D., Jacobs, D., & Biswas, A. In PASSAT/SocialCom 2011, Privacy, Security, Risk and Trust (PASSAT), 2011 IEEE Third International Conference on and 2011 IEEE Third International Conference on Social Computing (SocialCom), pages 87-94. IEEE.
Odd Leaf Out [pdf]Paper  Odd Leaf Out [link]Website  abstract   bibtex   
A growing number of projects are solving complex computational and scientific tasks by soliciting human feedback through games. Many games with a purpose focus on generating textual tags for images. In contrast, we introduce a new game, Odd Leaf Out, which provides players with an enjoyable and educational game that serves the purpose of identifying misclassification errors in a large database of labeled leaf images. The game uses a novel mechanism to solicit useful information from players' incorrect answers. A study of 165 players showed that game data can be used to identify mislabeled leaves much more quickly than would have been possible using a computer vision algorithm alone. Domain novices and experts were equally good at identifying mislabeled images, although domain experts enjoyed the game more. We discuss the successes and challenges of this new game, which can be applied to other domains with labeled image datasets.
@inProceedings{
 title = {Odd Leaf Out},
 type = {inProceedings},
 identifiers = {[object Object]},
 keywords = {-games with a purpose,arijit biswas,computer vision,david jacobs,department of computer science,detection,error,leaf identification},
 pages = {87-94},
 websites = {http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=6113099},
 publisher = {IEEE},
 id = {f6927365-fe38-3a71-832f-c23b25c01906},
 created = {2013-08-02T11:04:42.000Z},
 file_attached = {true},
 profile_id = {2b906273-dc25-3aff-9eef-94e37f452149},
 group_id = {ab695928-535d-3373-a630-70913ea6b675},
 last_modified = {2013-09-05T16:54:11.000Z},
 tags = {challenges,data collection,education and learning,gamification},
 read = {true},
 starred = {false},
 authored = {false},
 confirmed = {true},
 hidden = {false},
 citation_key = {Hansenb},
 abstract = {A growing number of projects are solving complex computational and scientific tasks by soliciting human feedback through games. Many games with a purpose focus on generating textual tags for images. In contrast, we introduce a new game, Odd Leaf Out, which provides players with an enjoyable and educational game that serves the purpose of identifying misclassification errors in a large database of labeled leaf images. The game uses a novel mechanism to solicit useful information from players' incorrect answers. A study of 165 players showed that game data can be used to identify mislabeled leaves much more quickly than would have been possible using a computer vision algorithm alone. Domain novices and experts were equally good at identifying mislabeled images, although domain experts enjoyed the game more. We discuss the successes and challenges of this new game, which can be applied to other domains with labeled image datasets.},
 bibtype = {inProceedings},
 author = {Hansen, Derek and Lewis, Darcy and Rotman, Dana and Jacobs, David and Biswas, Arijit},
 booktitle = {PASSAT/SocialCom 2011, Privacy, Security, Risk and Trust (PASSAT), 2011 IEEE Third International Conference on and 2011 IEEE Third International Conference on Social Computing (SocialCom)}
}
Downloads: 0