(171) previous ~ index ~ next
To: "'doddington@nist.gov'" <doddington@nist.gov>,
From: "Strzalkowski, Tomek (CRD)" <strzalkowski@crd.ge.com>
Subject: RE: Topic tracking on partial source files
Date: Thu, 2 Sep 1999 12:58:02 -0400
George,
I have just recently returned from a trip. Is the dry run workshop
a two day meeting now (as suggested in your message)? My last info
is James' email from July where he said the workshop would run for
1 day only. Also, what are the latest deadlines for submission?
Are the dates on the TDT3 web site accurate (at least the dry run
workshop date is not).
--- Tomek
-----Original Message-----
From: George Doddington [mailto:doddington@nist.gov]
Sent: Thursday, September 02, 1999 12:39 PM
To: Ralf Brown
Cc: TDT distribution
Subject: Re: Topic tracking on partial source files
Ralf Brown wrote:
>
> The only issue I have with eval 2.7 isn't with the actual evaluation
> plan, but with Jon Fiscus' announcement that index files will not list
> the partial source files just after the Nt_max'th YES-labeled story
> for an event. In my opinion, that makes *all* tracking software using
> those index files non-compliant with the spec:
>
> Page 2, section 4.2
> "The tracking task is then to correctly classify ALL SUBSEQUENT STORIES
> as to whether or not they discuss the target topic."
> [emphasis mine]
>
> Page 4, section 5.2 (right-hand column)
> [Training data is all stories up to last YES-labeled training story.]
> "The test set will comprise the remainder of the corpus that follows."
This was done to make running and evaluating the TDT system as
simple and straightforward as possible. We believe that there
is little to be gained from requiring that partial source files
be processed. We can discuss this at our meeting on 7-8 October.
If there is good reason and a strong desire to include these
partial source files, then we will put them back in.
--
George Doddington at NIST: doddington@nist.gov or 301/975-3261
(171) previous ~ index ~ next
Last updated Thu Sep 2 18:19:20 1999