(237) previous ~ index ~ next
To: Jon_Yamron@dragonsys.com
From: Rich Schwartz <schwartz@bbn.com>
Subject: Re: Evaluation plan
Date: Thu, 1 Jun 2000 19:46:19 -0400 (EDT)
Jon,
On Thu, 1 Jun 2000 Jon_Yamron@Dragonsys.com wrote:
> In short, if we want to get useful work done this year, I think we need an
> evaluation plan that allows us to use the 60 topics from Eval-99 for
> development.
I agree that having a good development test set is critical for
doing useful work. Otherwise we all just gambling.
Your suggestion of using the eval-99 topics as development and
then testing on a new set of 60 topics from the same set sounds like a
good compromise. Since we currently have no way to evaluate our systems
on the new topics (since we don't know what they are yet), there is no
way to cheat even though we may run our systems on the data many times.
--Rich
(237) previous ~ index ~ next
Last updated Mon Jun 12 13:26:39 2000