(176) previous ~ index ~ next
To: Rich Schwartz <schwartz@bbn.com>
From: Jonathan Fiscus <jonathan.fiscus@nist.gov>
Subject: Re: Dev-test data
Date: Thu, 10 Sep 1998 13:50:13 -0400
Rich
I didn't see that you had submitted any thing, (I didn't get an email so
I didn't look). I'll get those scored and back to you presently.
Your right, it's important for participants to be able to self-score, (I
don't want to score a billion runs anymore that you guys want to package
them up!) I would like to be able to do some additional analysis here
though. It's important for me, as a software engineer, to be able to
work with as much data as possible so I can better support the community
and as a benchmark test administrator to fairly represent the community
when results are published.
Jon
Rich Schwartz wrote:
>
> Jon,
>
> Thanks for offering to score other conditions. But at the moment,
> it might be more useful if we could get the score you computed on the
> condition we each sent in order that we can compare it with the score that
> we get when we score it ourselves.
>
> Assuming the score is the same, then we would each be able to run
> as many conditions as we like in order to make a host of comparisons that
> we will each show, without having to bother packaging them up and without
> you having to score all of the conditions for everyone.
>
> Thanks,
> --Rich
> =========================================
>
> On Thu, 10 Sep 1998, Jonathan Fiscus wrote:
>
> > Folks,
> >
> > Now that mostly everyone has worked through the process of running the
> > evaluation and submitting results, would any of the participants like to
> > have additional runs scored and reported on at the workshop? NIST would
> > be very interested in receiving any additional data including in the
> > following areas:
> >
> > - Segmentation evaluations on manual transcriptions,
> > - Tracking evaluations for varying Nt values,
> > - Tracking evaluations for varying input conditions,
> > (i.e. asr vs man_ccap or man_fdch)
> > - Detection evaluations for for varying input conditions,
> > (i.e. asr vs man_ccap or man_fdch)
> >
> > Having this data would enable us to do more analysis prior to the
> > workshop and to experiment with presentation of benchmark results prior
> > to this fall.
> >
> > We could score the data and get the scores back to you and include them
> > in our presentation if the data is received by COB Monday. We could
> > accommodate mid day Tuesday if need be.
> >
> > Thanks
> > Jon
> >
> > --
> > Jon Fiscus
> > NIST
> > Email: jfiscus@nist.gov
> > Phone: (301) 975-3182
> >
--
Jon Fiscus
NIST
Email: jfiscus@nist.gov
Phone: (301) 975-3182
(176) previous ~ index ~ next
Last updated Fri Sep 11 13:52:54 1998