(033) previous ~ index ~ next
To: tdt-distrib@unagi.cis.upenn.edu
From: jonathan.fiscus@nist.gov
Subject: TDT - Phase 2 '98 Evaluation Scoring Tools
Date: Thu, 30 Apr 1998 15:33:04 -0400
Folks,
The beta version of the TDT2 '98 Evaluation tools are now ready for
testing by participants. The evaluation plan has also been updated to
reflect minor changes in the output file formats for each for the TDT
tasks and the addition of test indexes.
Presently, the LDC will release the dry-dry-run test corpus. The
corpus is a re-packaging of the April 7th delivery 'tdt-deliv-980407'.
The corpus has example system output files with which the evaluation
software can be tested.
We have started a web page to document NIST's efforts wrt. the TDT
evaluations. It's URL is 'http://www.nist.gov/speech/tdt_98.htm'.
The web page points to the evaluation software and resources
maintained by NIST.
Alternatively, you can access the TDT2 Evaluation Software at the URL
ftp://jaguar.ncsl.nist.gov/tdt98/TDT2eval_v0.1.tar.Z
and the updated evaluation plan at the URL:
ftp://jaguar.ncsl.nist.gov/tdt98/tdt2.eval.plan.98.v2.2.ps
..........
Our hope is that participants will download the evaluation software and
dry-dry-run corpus before the meeting next week. We would like to have
as much feedback about the software as possible during the meeting.
We are also looking for actual TDT system outputs from the dry-dry-run
corpus. If anyone does run with the data, please consider sending NIST
a copy so that we can continue coding and debugging the software.
............
There are at least two future enhancements to the software that work
will begin on immediately. First, DET curve generation will be added
to both the tracking and detection scoring modules. Second, the
segmentation scoring program is slow, we'll attempt to decrease it's
runtime.
Regards,
Jon Fiscus
--
Jon Fiscus
NIST
Email: jfiscus@nist.gov
Phone: (301) 975-3182
(033) previous ~ index ~ next
Last updated Wed Sep 9 09:40:47 1998