(020) previous ~ index ~ next
To: Jon Yamron <Jon@dragonsys.com>
From: "George R. Doddington" <doddington@sri.com>
Subject: Re: A TDT question requiring some judgement? -Reply
Date: Sat, 04 Apr 1998 13:54:05 -0500
Here is the LDC instruction to the TDT2 labelers for determining
whether a story should be tagged for a topic:
When you see a reference to an event, please check YES (relevant)
or if less than 10% but more than two declarative independent
clauses of the article discusses the event, check BRIEF (relevant,
but only briefly mentioned).
Apparently, there is some underlying sentiment within the community
that the story must be factual, but that has never been discussed,
to the best of my knowledge. Furthermore, it may well be the case
that satirical stories have roughly the same level of value as do
factual reporting. In any case, distinguishing between the two
makes makes language understanding seem like child's play. I see
no reason to introduce such additional complexity into the TDT2
task definition at this point.
Now, as to whether the story discusses topic 2, the Monica Lewinsky
Case, I would say that it certainly qualifies as a "BRIEF", but that
it seems to fall short of the 10% criterion, unless you interpret
some very indirect innuendos as "discusses the event", such as
"a DOUBLE-CROSSING, LONG-TAILED RAT", and "As for a Valentine's
present, steer clear of ... dresses". I would suggest that this
level of indirection NOT be considered as "discussed the event".
While I'm at it, there are a couple of other comments that I'd like
to make, just for the hell of it:
1) What is the topic that this story discusses? I would offer two
possible candidates:
WHAT: President Clinton's Valentine's dinner
WHERE: Camp David, MD
WHEN: 14 February 1998 (Valentine's Day)
WHAT: Leak of briefing documents for Valentine's dinner
WHERE: Washington D.C.
WHEN: ~10-14 February 1998
I'm mentioning this because I sense that topics are sometimes
defined to be a precipitating (sub)event rather than the event
itself. I realize that this is a soft area and subject to debate,
interpretation and taste. But I would prefer to define the topics
to be more inclusive rather than limited by specifics. The risk in
doing this is that the topic may become difficult to delineate.
2) The following rewrite might make the tagging instructions
clearer and easier for determining how to tag a story with
respect to a defined TDT2 topic:
Tag the story "YES" if it discusses the topic. However, tag
the story "BRIEF" if the topic is mentioned only briefly or in
passing. (To be specific, BRIEF shall mean that less than 10%
of the story, or fewer than two independent clauses, are devoted
to the topic.)
I think that we can help ourselves by sticking to common terminology
(e.g., "story" rather than "article") and by ensuring that when we
use key terms (such as "story", "event" and "topic") that we mean
what we say (rather than occassional slip ups when, for example,
"story" or "event" is substituted for the intended "topic".)
--
George R. Doddington [at home]. Reply to doddington@sri.com,
or call 703/556-3434 [at home], or 301/688-0976 [at NSA]
(020) previous ~ index ~ next
Last updated Wed Sep 9 09:40:47 1998