Caltech UCSD Binary Annotation Model (CUBAM) Toolbox
This is a toolbox written in Python and C++ and implements the models used in the publications below.
Welinder P., Branson S., Belongie S., Perona, P. “The Multidimensional Wisdom of Crowds.” Conference on Neural Information Processing Systems (NIPS) 2010. pdf
Welinder, P., Perona, P. Online crowdsourcing: rating annotators and obtaining cost-effective labels. Workshop on Advancing Computer Vision with Humans in the Loop at CVPR. 2010. pdf