Knowledge Center

Content Moderation Dataset FAQs

Suggest Edits

Guidelines for custom Labels

The AI labels we are providing are made possible through solid research of giants upon whose shoulders we stand:

Our initial aim is to conveniently package for you the labels that target the largest number of standard use-cases. With that said, if after having gone through the list of labels you are left wanting, know that we are working towards being able to support you in building your own labels, and training models that can provide you with predictions that cater to your specific needs.

One recurrent question is: how many instances do I need to label before being able to train a model with decent accuracy?

  • The answer to this question will depend on the complexity of the task, the quality and diversity of the instances, along with the quality of the ground-truth labels. Furthermore, building the dataset does not have to be independent from the training. Meaning that you can iteratively train new versions of your models as you increase your number of labeled instances. Sometimes this can enable a positive loop where current versions of the models make predictions for you to correct instead of having to label from scratch. The downside here is that early bias might induce further bias.
  • Here are a few figures that might help you make a rough estimation of the quantity required:

Another recurrent question is: how to ensure ground-truth quality throughout the inherently subjective task of interpreting social posts?

  • CardiffNLP's approach was to include in their labeling instructions the following rules:
    • The same person might only label the same set of labels once. Redundant submissions are discarded.
    • For the entries of someone to be accepted, they must have correctly labeled 3 out of the 3 randomly sampled hidden tests that are part of the instances they are presented with to label.
    • While empty submissions are not allowed and each instance is required to be annotated with at least one label for multi-label tasks, it is also encouraged from anyone participating in the labeling to share feedback about any instance they are uncertain about.
  • The Six Attributes of Unhealthy Conversations work by Google Research, University of Oxford, University of South Carolina, and others, established more intricate guidelines to ensure the quality of their labels:
    • With regards to the subjective nature of their labels, they address it as such:

      The inherent subtlety, subjectivity, and frequent ambiguity of the attributes covered in this dataset make crowdsourcing quality attribute labels an unavoidably difficult process. Typically the goal in an annotation task would simply be to maximise agreement between the multiple annotators of each comment. However, when the annotation task is inherently subjective and meaningful difference of opinion is itself valuable data, the goal becomes instead to maximise common understanding of the task across annotators. This entails tailoring the phrasing of the questions put to annotators, so as to create as common an understanding as possible of what each question is really asking. This way, disagreement between annotators reflected in the dataset will represent different reasonable readings of the same comment which are themselves important to capture.

    • With regards to prompting for correctness from the annotators, they define an approach that relies on establishing "trustworthiness scores":

      Our primary quality control mechanism was to collate a set of ‘test comments’, for which we had manually established the correct answers. Annotators encountered one test comment per batch of seven comments they reviewed, without knowing which of the seven was the test comment, and their running accuracy on these test comments was defined as their ‘trustworthiness score’. The task required that annotators maintain a trustworthiness score of more than 78%. If an annotator dropped below this level, they were removed from the annotator pool for this task, and all of their prior annotations were discarded. The removed ‘bad’ annotator judgements were replaced by newly collected trusted judgements as necessary.

  • Be prepared to work with limitations and constraints for which no workaround is seemingly plausible:
    • Social posts contain transliterations, abbreviations, interjections, language for which no tokenizer is equipped with the right vocabulary, small talk that provides little to no context from which to infer remotely accurate predictions, terms and expressions that rise in popularity as quickly as they are forgotten, and overall human written expression at its most deliberate and raw and unrestricted form.
    • Aim to reflect the size range of your target inputs:
      • 19.6% of instances in the emotion dataset used by CardiffNLP fall between 118 and 134 number of characters. It is the largest proportion, followed by 13.8% ranging between 134 and 150 characters. In contrast, the largest proportion of posts on Farcaster is less than 20 characters long.
      • The Human ChatGPT Comparison Corpus (HC3) from Hello-SimpleAI strays the furthest in terms of the number of characters. It clearly did not prompt neither the human answers nor the ChatGPT ones for brevity. In fact, the sustained verbose eloquence is a strong feature indicative of LLM-generated content, which introduces added complexity in detecting it within the often short social posts.

Updated 6 months ago

What’s Next