Paffenroth, Randy Clinton
Whitehill, Jacob Richard
This Major Qualifying Project introduces a novel crowdsourcing consensus model and inference algorithm which we call PICA (Permutation-Invariant Crowdsourcing Aggregation) that is designed to recover the ground-truth labels of a dataset while being invariant to the class permutations enacted by the different annotators. The PICA model is constructed by endowing each annotator with a doubly-stochastic matrix (DSM), which models the probabilities that an annotator will perceive one class and transcribe it into another. We conduct simulations and experiments to show the advantage of PICA over similar models for three different clustering/labeling tasks, including aggregating dense image segmentations and clustering text passages. Our work was published in HCOMP 2018.
Worcester Polytechnic Institute
Major Qualifying Project