ACE08 Annotation Tasks
ACE 2008 annotation tasks include:- Local (within-document) EDR (Entity Detection and Recognition) and RDR (Relation Detection and Recognition) for English and Arabic.
- Global (cross-document) EDR and RDR for English and Arabic.
Local EDR and RDR
In 2008 the ACE Program will include entity and relation annotation for Arabic and English on entity and relation.
Entity
The current ACE task identifies five types of entities: Person, Organization, Location, Facility, and Geo-Political Entity (GPEs). Each type is further divided into subtypes (for instance, Person subtypes include Individual, Group and Indefinite). Annotators tag all mentions of each entity within a document, whether named, nominal or pronominal. For every mention, the annotator identifies the maximal extent of the string that represents the entity and labels the head of each mention. Nested mentions are also captured. Each entity is classified according to its type and subtype. Each entity mention is further tagged according to its class such as specific, generic, attributive, negatively quantified or underspecified. Annotators also review the entire document to group mentions of the same entity together; they also label cases of metonymy, where the name of one entity is used to refer to another entity (or entities) related to it.
Relation
The goal of the Relation task is to detect and characterize Relations of the targeted Types between entities. Relations are ordered pairs of entities. Annotators label the type and subtype for each relation, along with its syntactic class and syntactic extent. Relations are also tagged for modality and tense.
Global EDR and RDR XDOC
The 2008 ACE XDOC evaluation corpus will be on the order of 10,000 documents per language which will capture the following entity mention variations phenomena and their relations:
- Aliases (for instance, Ilich Ramirez Sanchez might be selected if it is known that Carlos the Jackal is also mentioned in the corpus)
- Orthographic variation(for instance, Mu'ammar Al-Qadhafi might be selected if it is known that Muammar al-Gaddafi also occurs)
- Confusable entities (for instance, Michael Jordan might be selected if it is known that the corpus contains mentions of "Michael Jordan the US basketball player" and mentions of "Michael Jordan the English football player"
For XDOC evaluation, LDC will do the following:
- within-in doc ACE annotation on 400 files out of the 10,000 documents for both English and Arabic
- xdoc annotation on 50 of the target entities and their relations
- xdoc annotation on the second argment which involved in the 50 target entities' relations on the 400 within-doc annotated files
ACE08 Annotation Guidelines
This page contains links to the latest version of the ACE08 annotation guidelines. Additional minor updates are possible as we refine instructions for distinguishing NAM vs. NOM entities in the context of the cross document task. Subsequent updates will be announced to the ace_list.English Guidelines
English-Entities-Guidelines_v6.6.pdf (June 13, 2008)English-Relations-Guidelines_v6.2.pdf (April 28, 2008)
English Global Entity and Relation Coreference Guidelines are forthcoming.
Arabic Guidelines
Arabic-Entities-Guidelines_v7.4.2.pdf (June 13, 2008)Arabic-Relations-Guidelines_v6.5.pdf (March 4, 2008)
Arabic Global Entity and Relation Coreference Guidelines are forthcoming.