Popular GitHub Repositories for Crowdsourcing

Everything you need about Active Learning (AL).

AlpacaTag: An Active Learning-based Crowd Annotation Framework for Sequence Tagging (ACL 2019 Demo)

An On-Chain Open-Source Platform for Rapid AI Model Productization Using Decentralized Resources with Flexibility and Scalability

[CVPR 2017] AMT chat interface code used to collect the Visual Dialog dataset

A neural network layer that enables training of deep neural networks directly from crowdsourced labels (e.g. from Amazon Mechanical Turk) or, more generally, labels from multiple annotators with different biases and levels of expertise.

OVALChat is a customizable Web app aimed at conducting user studies with chatbots

This repository provides data and code for "Vox Populi, Vox DIY: Benchmark Dataset for Crowdsourced Audio Transcription" paper.

Create and run interactive behavioral experiments on the web, including VR studies, for free!

Aggregation framework for annotating datasets in computer vision tasks (detection, segmentation, video captioning etc.)

NLP crowdsourcing platform for word-level annotations

Code for the KDD-2023 paper: Neural-Hidden-CRF: A Robust Weakly-Supervised Sequence Labeler

[ACL 21] Crowdsourcing Learning as Domain Adaptation: A Case Study on Named Entity Recognition

(NAACL 2019) Modelling Instance-Level Annotator Reliability for Natural Language Labelling Tasks

Build lightweight knowledge graphs from text in minutes. Annotate entities, link facts, and visualize relationships — all in one Streamlit app, with Google Sheets integration.

A curated list of awesome Weak-Supervision-Sequence-Labeling (WSSL) papers, methods & resources.

Using outlier detection and entity recognition to improve a crowdsourcing biocuration system

A collaborative Streamlit-powered annotation tool for labeling YouTube comments based on community rules, with real-time Google Sheets synchronization and multi-user tracking.