Google Research

Crowdsourcing a Gold Standard for Medical Relation Extraction with CrowdTruth

Proceedings of the 2016 Collective Intelligence Conference


In this paper, we make the following contributions: (1) a comparison of the quality and efficacy of annotations for medical relation extraction provided by both crowd and medical experts, showing that crowd annotations are equivalent to those of experts, with appropriate processing; (2) an openly available dataset of 900 English sentences for medical relation extraction, centering primarily on the cause relation, that have been processed with disagreement analysis and by experts.

Learn more about how we do research

We maintain a portfolio of research projects, providing individuals and teams the freedom to emphasize specific types of work