(287) previous ~ index ~ next
To: Steve Lowe <steve@dragonsys.com>
From: Jonathan Fiscus <jonathan.fiscus@nist.gov>
Subject: Re: Default evaluation conditions
Date: Tue, 22 Dec 1998 16:05:35 -0500
Steve,
I'm sorry if there is some confusion about the issue of contrastive
runs. I did talk to Charles before sending the Dec 14th note. However,
the January 6th deadline was of my design so that we would A) be able to
collect more contrastive test runs and B) be able to finish the scoring
runs by the January 8th deadline.
My understanding is that Charles would like to see as many officially
scored results as possible. That way the official account of the
evaluation which will be published will be able to address all facets of
the evaluation. We never intended to give you the impression that only
NIST should be running the eval software. In fact, we encourage this to
help us debug the data and evaluation software.
If you are unable to complete your contrastive runs by the 6th, I'd
encourage you to not rush these runs and just self-score them after I
publish the topic relevance table.
Regards
Jon
Steve Lowe wrote:
>
> Jon,
>
> Gee, Jon, I fear there has been a miscommunication here, and judging
> by the e-mail that has been going back and forth, I'm not the only one
> who was confused. Your message of Dec. 14th stated:
>
> "I spoke to Charles late last week and he was concerned that no
> one would submit results for the other than the default TDT
> evaluations."
>
> I think we here at Dragon all took this as indicating that you were
> requesting the submissions and designating January 6th as the deadline
> as a proxy for Charles. In particular, we assumed that the date
> reflected his need for results on other test conditions ASAP, and of
> course, everyone jumped.
>
> Your most recent message makes it appear that the only reason for
> submitting by the 6th is so that NIST can do the scoring instead of us
> doing it ourselves. Is this correct? Can you clarify what the story
> is here? Is there a reason we should break our necks for the January
> 6th deadline?
>
> Dragon had always planned to run contrastive tests in January, even
> before your Dec. 14th message. That message just gave us the
> impression that the sponsors wanted to see scored results much earlier
> than we had expected to submit them.
>
> Steve
--
Jonathan Fiscus Snailmail: Nat'l Inst. of Stds. and Tech.
NIST 100 Bureau Dr. Stop 8940
Phone: (301) 975-3182 Gaithersburg, MD 20899-8940
Email: jfiscus@nist.gov
(287) previous ~ index ~ next
Last updated Wed Feb 3 10:44:20 1999