(360) previous ~ index ~ next
To: Jonathan Fiscus <jonathan.fiscus@nist.gov>
From: Stephanie Strassel <strassel@ldc.upenn.edu>
Subject: Re: Ontopic both YES and BRIEF
Date: Wed, 19 Jun 2002 10:01:43 -0400
To clarify - at the most recent TDT PI meeting, LDC proposed abandoning
the BRIEF category altogether for future annotation. Annotators dislike
BRIEF and find it difficult to apply consistently; they end up spending
valuable time arguing about whether a story is BRIEF or YES and
searching for passing references to topics that count as BRIEF.
Instead, we proposed treating stories that make only a passing reference
to a topic (i.e., naming a topic without providing any information about
it) as NO, and treating stories that discuss a topic as YES, regardless
of how much of the story discusses the topic. The research community
accepted this proposal at the PI meeting so we've adopted it as part of
our upcoming annotation effort.
Therefore, any revised treatment of BRIEF will apply only to previously
annotated corpora. Please keep in mind that our standard for BRIEF vs.
YES in the past has been (from the TDT annotation guidelines):
-YES if at least 10% of the story is devoted to that topic.
-BRIEF if less than 10% of the story is devoted to that topic. There is
no such thing as too brief a reference. Any mention is worth noting with
BRIEF.
Stephanie
Jonathan Fiscus wrote:
>
> Folks,
>
> At the last TDT meeting, we agreed that stories marked with BRIEF would
> now be considered ontopic in addition to stories marked YES. The
> question I have is with regard to the tracking task: Should stories
> marked as BRIEF be permitted to be training stories?
>
> I'm assuming the answer is yes. Any dissent?
>
> Jon
> -------------------------------------------------------------
> To unsubscribe from tdt-distrib, email majordomo@ldc.upenn.edu
> with "unsubscribe tdt-distrib" in the body of the message.
--
Stephanie Strassel
Assoc. Director, Annotation * Phone: 215.898.9681
Linguistic Data Consortium * Fax: 215.573.2175
3615 Market Street * strassel@ldc.upenn.edu
Philadelphia, PA 19104-2608 * http://www.ldc.upenn.edu
-------------------------------------------------------------
To unsubscribe from tdt-distrib, email majordomo@ldc.upenn.edu
with "unsubscribe tdt-distrib" in the body of the message.
(360) previous ~ index ~ next
Last updated Wed Jun 19 11:58:05 2002