LDC's ACE 2005 Annotation Toolkit
A copy of the 2005 ACE Annotation tool can now be downloaded for Windows platforms.
Installation
The installation of the ACE toolkit is quite easy:
- Unzip the archive (
aceTool2005_Windows_v1.0.zipunzips as a folder namedaceTool2005_windows); - Double click the .bat icon for the relevant tool (by language):
aceTool2005-Arabic.bataceTool2005-Chinese.bataceTool2005-English.bat
Using aceTool2005
A copy of the reference manual for aceTool2005 can be found here:
Basic Annotation Components
Entity Module
When an annotator uses this module, they first select the extent of an entity mention. They are then prompted to select the head extent and mention type of the mention. Next, they must select a row in the "entity table" for the mention. Rows in the entity table correspond to composite entities; one column in the table lists all of the referential (SPC, GEN, USP, etc) mentions of each entity while another column lists all of the attributive (ATR) mentions of each entity. Within columns, mentions are color-coded and separated into rows by mention type.
When an annotator selects a new row in the entity table, they must then select a type, subtype, and class for the entity as a whole. If a mention is being added to an entity with a type that requires mention roles (GPE, for now), then the annotator is next prompted to select a role for the current mention.
The entity module also allows annotators to "merge" to entities together in the table view.
For QC purposes, the entity module also has functionality that allows users to search for an annotation by ID. This allows users to find annotations that were identified as problematic by one of our QC scripts. In addition, annotators can run the "aceCheck" script inside the tool, allowing them to identify and fix logical and formatting errors in their annotations while they are still working on the file.
Value Module
The text in the value module will allow the user to see all of the currently annotated entities AND values. Mentions are color-coded by entity- or value-type. When an annotator uses this module, they first select the extent of a value mention. They are then prompted to select the type and subtype of the value. Values have no coreference and no mention-level attributes.
Like the entity module, the value module allows users to search for annotations by ID for the purposes of QC. It is also possible to run the "aceCheck" script directly from this module.
Relation Module
The text in the relation module will allow the user to see all of the currently annotated relations, values, AND entities. Mentions are color-coded by type. When an annotator uses this module, they first select the extent of a relation mention. They are then prompted to select the syntactic condition associated with the mention. Next, they must select a row in the "relation table" for the mention. Rows in the relation table correspond to relations; a column in the table lists all of the relation mention participants, grouped by their role in the relation, while another column in the table lists all of the relation mentions.
If the annotator adds a mention to a blank row (which corresponds to creating a new relation), they are prompted to select the type and subtype of the relation.
The annotator is next presented with a pop-up box for the purpose of assigning participants to the relation. Participants must by entity (or value?) mentions that fall within the extent of the relation mention.
Like the other modules, the relation module allows for searching by annotation ID as well as annotation checks through "aceCheck".
Event Module
The text in the event module will allow the user to see all of the currently annotated events, values, and entities. Mentions are color-coded by type. When an annotator uses this module, they first select the extent of an event mention. They are then prompted to select the anchor extent for the mention. Next, they must select a row in the "event table" for the mention. Rows in the event table correspond to coreferenced events; one column in the table lists all of the event mention participants, grouped by their role in the event, while another column lists all of the event mention anchors.
If the annotator adds a mention to a blank row (which corresponds to creating a new event), they are prompted to select the type and subtype of the event.
The annotator is next presented with a pop-up box for the purpose of assigning participants to the event. Participants must be entity or value mentions that fall within the extent of the event mention. Like the other modules, the event module allows for searching by annotation ID as well as annotation checks through "aceCheck".
Adjudication Modules
Each of the above modules will have a corresponding adjudication module. The adjudication module essentially consists of two copies of the corresponding annotation module in the same window. All of the functionality of the annotation module is preserved. In addition, it is possible to "hide" all annotations that are exactly the same. This allows the annotator to focus on discrepancies between two versions of the same file. As discrepancies are resolved, they are hidden from view; one all of the annotations match in the two versions of the file, the annotator can move on to the next task.
Validation and Quality Control Scripts
aceValidate
This script is run by annotators before "checking in" a completed file to the Annotation Workflow System. The script checks for common annotation and format errors and prompts the user to correct them.
aceCheck
This script checks for common errors in annotation and formatting which must be resolved before ag files may be converted to apf. (Superceded by aceValidate?)
aceCorrect
This script detects and corrects errors in existing .ag.xml files in preparation for delivery.
ag2apf
This script converts error-free files from their LDC-native ag format to apf (ACE Program Format). The ag2apf script also runs some additional QC checks for things that might cause conversion problems.
aceModify
The aceModify script works off of a tabular view of annotations, allowing annotations to be deleted or changed based on changes to the tabular view. This makes it easy to check for and correct things like reversed roles in a relation without having to open a file in the annotation tool.
aceView
The aceView script allows users to specify the kind of annotations they're interested in along with any number of attribute-value pairs. The script will then display a given sgm file with all of the annotations that match the specified criteria identified with curly brackets ({}). This tool allows annotators and managers to search for examples for guideline and training purposes, as well as giving a way to scan through annotations for QC purposes.
acePrint
The acePrint script generates a tabular view of each of the annotations in a document. It includes the extent and each of the features of the annotation. A sorted view of this output is used for QC; an annotator or manager can scan through a list of annotations of the same type looking for obvious outliers that should be double-checked or corrected.
FixBrokenAnnotations
need description
createRelease
This consists of utilties to automatically grab the latest ACE annotation files, run all validation steps, create corpus statistics and package the data for release.
General Infrastructure
AWS (Annotation Workflow System)
AWS is an automated workflow system that assigns files and tasks to annotators according to their managers' specifications. The system allows for multiple workflows (so that some files can be first- and second-passed, while others are dually first-passed and then adjudicated). It also handles the assignment of dual files, so that dual annotation is a completely blind process -- neither annotators nor their managers know which files have been selected for dual annotation. In addition, AWS handles the moving and copying of files, which reduces "save" and "copy" errors.