(245) previous ~ index ~ next
To: Rich Schwartz <schwartz@bbn.com>
From: George Doddington <doddington@nist.gov>
Subject: Re: Evaluation plan
Date: Mon, 12 Jun 2000 09:34:50 -0700
Rich Schwartz wrote:
>
> Jon,
> It's hard to know how good different sites are at memorizing the
> answers (i.e., tuning). But as I understand it, there are additional
> topics defined now, isn't that right? In this case, I don't see what
> would be wrong with declaring the old 60 topics as dev test, and the new
> ones as eval test. It's pretty hard to see how training on the old topics
> is somehow cheating on the new ones, even though the corpus is the same.
> It would, at best be a second-order effect.
>
> --Rich
Surely you would agree that this depends on the task. Knowing 60 (common)
topics might be expected to have little effect on topic tracking, but very
major effect on topic detection.
--
George Doddington in Orinda, CA: doddington@nist.gov or 925/631-6628
(245) previous ~ index ~ next
Last updated Mon Jun 12 13:26:40 2000