Annotation Guide

Annotation Instructions

Your task is to review a set of stories from a variety of news sources, and to label (annotate) each one as relevant or not to certain selected topics. The news sources are: Voice of America, VOA Today & The World Today; Public Radio International, The World; ABC, World News Tonight; CNN Headline News; Associated Press World Services; New York Times. All of the stories are collected between 1/4/98 and 6/30/98.

In total there will be 100 topics relating to news events, listed in packs of 21. These topics are selected by the LDC, and will refer to current news stories.

Most stories will match none of the topic definitions; some will match one; a very few will match more than one. Your task is to apply all relevant topics to each story. You will identify the story as NO (this is achieved by simply submitting the story), YES, BRIEF or REJECT. Use the YES label for stories that discuss the topic in a substantial way. Use the BRIEF label for stories that make only a passing reference to the topic. To determine the proper label, use the following criteria:

Please REJECT a story if:

Use Comments generously when rejecting files. Also use Comments to specify an "oddity" about an article. For instance, if the segmentation seems odd, but could be correct, label it or not as appropriate and comment on it.

The listing for each topic includes helpful, but incomplete information about the derivative event. Dates or date ranges are included where they are relevant. A link to a sample story is also available. Use this story as a reference only, an example of the language or vocabulary used in discussing this topic. You may take advantage of the headline information associated with each article, but cannot rely on that to be complete. Label-able topics may be mentioned in the middle of long articles.

There is more specific information related to topic definition below. If you have questions about how to label a specific story, refer to Stephanie SVtrassel, Kara Rennert, and Nii Martey.



Topics

"Topic" is defined in a special way specifically for TDT research. For the purposes of this project, topics refer to specific events or activities, such as the crash of a China Airlines airplane in Taipei, Taiwan on February 16, 1998, and encompass all facts, events and activities that are directly related to them. Here is the definition of topic and a few other essential terms, as used in TDT research:



Rules of Interpretation

Topics generally fall into a few general categories. As an aid to topic labeling consistency and to make the process efficient and accurate, please use the following guidelines in expanding from an event (the list provided in your labeling interface) to a topic.
1. Elections:
Examples - New people in office, new public officials, change in governments or parliaments (in other countries), voter scandals.
The event might be the confirmation of a new person into office, the activity around voting in a particular place and time, the opposing parties' or peoples' campaigns, or the election results. The topic would be the entire process, nominations, campaigns, elections, voting, ceremonies of inauguration.

2. Scandals/Hearings:
Examples - Monica Lewinsky, Kenneth Starr's investigations.
The event could be the investigation, independent counsels assigned to a new case, the discovery of a potential scandal, the subpoena of political figures. The topic would include all pieces of the scandal or the hearing including the allegations or the crime, the hearings, the negotiations with lawyers, the trial (if there is one), and even media coverage.

3. Legal /Criminal Cases:
Examples - crimes, arrests, cases.
The event might be the crime, the arrest, the sentencing, the arraignment, the search for a suspect. The topic is the whole package; crime, investigation, searches, victims, witnesses, trial, counsel, sentencing, punishment and other similarly related things.

4. Natural Disasters:
Examples - tornado, snow and ice storms, floods, droughts, mud-slide, volcanic eruptions.
The event would include causal activity (El Nino, in many cases this year) and direct consequences. The topic would also include; the declaration of a Federal Disaster Area, victims and losses, rebuilding, any predictions that were made, evacuation and relief efforts.

5. Accidents:
Examples - plane- car- train crash, bridge collapse, accidental shootings, boats sinking.
The event would be causal activities and unavoidable consequences like death tolls, injuries, loss of property. The topic includes mourners pursuit of legal action, investigations, issues with responsible parties (like drug and alcohol tests for drivers etc.)

6. Ongoing violence or war:
Examples - terrorism in Algeria, crisis in Iraq, the Israeli/Palestinian conflict.
In these cases the event might be a single act of violence, a series of attacks based on a single issue or a retaliatory act. The topic would expand to include all violence related to the same people, place, issue and time frame. These are the hardest to define, since war is often so complex and multi-layered. Consequences or causes often include (and would therefore be topic relevant) preparations for fighting, technology, weapons, negotiations, casualties, politics, underlying issues.

7. Science and Discovery News:
Examples - John Glenn being sent back into space, archaeological discoveries.
The event is the discovery or the decision or the breakthrough The topic, then, would include the technology developed to make this event happen, the researchers/scientists involved in the process, the impact on every day life, all history and research that was involved in the discovery.

8. Finances:
Examples - Asian economy, major corporate mergers.
The topic here could include information about job losses, impacts on businesses in other countries, IMF involvement and sometimes bail out, NYSE reactions (heavy trading BECAUSE Tokyo closed incredibly low). Again, anything that can be defined as a CAUSE of the event or a direct consequence of the event are topic-relevant.

9. New Laws :
Examples - Proposed Amendments, new legislation passed.
While the event may be the vote to pass a proposed amendment, or the proposal for new legislation, the topic includes the proposal, the lobbying or campaigning, the votes (either public voting or House or Senate voting etc.), consequences of the new legislation like protesting or court cases testing it's constitutionality.

10. Sports News :
Examples - Olympics, Super Bowl, Figure Skating Championships, Tournaments.
The event is probably a particular competition or game, and the topic includes the training for the game or competition, announcements of (medal) winners or losers, injuries during the game or competition, stories about athletes or teams involved and their preparations and stories about victory celebrations.

11. MISC. News :
Examples - Dr. Spock's Death, Madeleine Albright's trip to Canada, David Satcher's confirmation.
These events are not easily categorized but might trigger many stories about the event. In these cases, keep in mind that we are defining topic as the seminal event and all directly related events and activities. (include here causes and consequences) If the event is the death of someone, the causes (illness) and the consequences (memorial services) will all be on topic. A diplomatic trip topic would include plans made for the trip, results of the trip (a GREAT relationship with Canada??) would be on topic.



Administrative Tracking

Please choose a file, from the source you have been assigned, and WRITE DOWN the file name. We need to keep careful records, both programmatic and on paper. While you are working within a file, you will be able to back up and make changes to your labels. Once you log out, the "rejects" you have identified will be labeled as such forever in the interface, so please be especially careful when labeling these.




Last modified: Mon Apr 5 16:39:41 1999