(290) previous ~ index ~ next
To: clwayne@afterlife.ncsc.mil
From: Steve Lowe <steve@dragonsys.com>
Subject: Re: Default evaluation conditions
Date: Thu, 24 Dec 1998 11:24:55 -0500
Charles,
Thanks for joining in. In response to your point:
> I do not see what the problem is. Jon has accurately represented my views
> (in favor of meaningful contrastive experiments), has stated that he would
> like to have results by the 6th or 7th (to meet NIST's deadlines), but has
> provided an acceptable option (scoring your own results).
You have just supplied the answer I needed. YOU said "... provided an
acceptable option (scoring your own results)." Since you, as the
sponsor, consider later, self-scored contrasts acceptable, I'm fine.
It was not clear to me from reading Jon Fiscus's messages how strongly
or weakly you might want results in the NIST summaries, so I didn't
know how to prioritize the Jan. 6th deadline. The messages said we
could self-score, but there were references to an "official account",
and I didn't know what weight you put on that account.
We take your requests very seriously, Charles, and we try to
accomodate them. We are aware that you use these reports in
presentations to your customers, and I don't know how much it matters
in those presentations whether our results are in NIST's "official
account" versus appearing in our own paper in the Proceedings. For a
true deadline we have to take extra precautions to make sure we don't
miss it due to a data processing blunder or computer failure. This
means extra time and doing the preparations earlier, for example,
between the Dec. 21 eval and Christmas. If the date is, instead, a
desirable goal, we can make reasonable plans to submit by that date
and it is not a catastrophe if something goes wrong.
You have confirmed that you consider the 6th to be in that latter
category, and that's fine with us. We DO have submissions planned for
the 6th; we estimate that our probability of success is 0.93.
By the way, Jon Fiscus and I had a pleasant conversation on the
telephone yesterday (Wednesday). He indicated then (just as you say)
that you considered the early submissions desirable but not vital,
that mainly you just want to assure that contrasts were run. He also
explained why he and the NIST team would prefer not to run a second
scoring later in the month if it could be avoided; given that
self-scoring is an acceptable alternative, I now see no reason to ask
Jon to do such a scoring.
Well, I'm now satisfied with this resolution. Thanks to all who
responded.
Happy Holidays to You and Your Family and Friends,
Steve
(290) previous ~ index ~ next
Last updated Wed Feb 3 10:44:20 1999