This Major Qualifying Project introduces a novel crowdsourcing consensus model and inference algorithm -which we call PICA (Permutation-Invariant Crowdsourcing Aggregation) - that is designed to recover the ground-truth labels of a dataset while being invariant to the class permutations enacted by the different annotators. The PICA model is constructed by endowing each annotator with a doubly-stochastic matrix (DSM), which models the probabilities that an annotator will perceive one class and transcribe it into another. We conduct simulations and experiments to show the advantage of PICA over similar models for three different clustering/labeling tasks, including aggregating dense image segmentations and clustering text passages. Our work was published in HCOMP 2018.

• This report represents the work of one or more WPI undergraduate students submitted to the faculty as evidence of completion of a degree requirement. WPI routinely publishes these reports on its website without editorial or peer review.

Creator

• Giancola, Michael Joseph

Publisher

• Worcester Polytechnic Institute

Identifier

• E-project-042218-194555

Award

• Provost's MQP Award

Advisor

• Whitehill, Jacob Richard
• Paffenroth, Randy Clinton

Year

• 2018

Date created

• 2018-04-22

Resource type

• Major Qualifying Project

Major

• Interdisciplinary

Rights statement

• In Copyright

Last modified

• 2023-01-19