(170) previous ~ index ~ next

---

To: Ralf Brown <Ralf_Brown@v.gp.cs.cmu.edu>
From: George Doddington <doddington@nist.gov>
Subject: Re: Topic tracking on partial source files
Date: Thu, 02 Sep 1999 12:39:15 -0400

---

Ralf Brown wrote:
>
> The only issue I have with eval 2.7 isn't with the actual evaluation
> plan, but with Jon Fiscus' announcement that index files will not list
> the partial source files just after the Nt_max'th YES-labeled story
> for an event. In my opinion, that makes *all* tracking software using
> those index files non-compliant with the spec:
>
> Page 2, section 4.2
> "The tracking task is then to correctly classify ALL SUBSEQUENT STORIES
> as to whether or not they discuss the target topic."
> [emphasis mine]
>
> Page 4, section 5.2 (right-hand column)
> [Training data is all stories up to last YES-labeled training story.]
> "The test set will comprise the remainder of the corpus that follows."

This was done to make running and evaluating the TDT system as
simple and straightforward as possible. We believe that there
is little to be gained from requiring that partial source files
be processed. We can discuss this at our meeting on 7-8 October.
If there is good reason and a strong desire to include these
partial source files, then we will put them back in.
--
George Doddington at NIST: doddington@nist.gov or 301/975-3261

---

(170) previous ~ index ~ next

Last updated Thu Sep 2 18:19:20 1999