A Shared Task for the Digital Humanities Chapter 2: Evaluating Annotation Guidelines

In this section, we will discuss our idea of guideline evaluation and the underlying considerations.Evaluating annotation guidelines in this way is a fairly new endeavor, and we have developed the High Chair Accessory evaluation setup from the ground up.Although we do not claim our Multiminerals choices to be universally valid or applicable, we believe that this approach to guideline evaluation is relevant for similar settings and can be adapted to projects that might have other preferences and priorities.