(159) previous ~ index ~ next
To: tdt-distrib <tdt-distrib@ldc.upenn.edu>
From: Jonathan Fiscus <jonathan.fiscus@nist.gov>
Subject: New Eval Software, v1.4, and dry-run index files
Date: Thu, 12 Aug 1999 12:15:32 -0400
Folks,
I've just put a new version of the evaluation software on our FTP site.
You can get it from the URL:
ftp://jaguar.ncsl.nist.gov/tdt3/TDT3eval_v1.4.tgz
Major changes in the release include:
- Evaluation support for the Story Link Detection (SLD) task.
- The FAQ included in the documentation
- Command line specification of alternative topic relevance tables
- Added Paul van Mulgret's precision/recall statistics to the
segmentation evaluation script.
- Computation of normalized cost functions, per the
evaluation specification.
- Code modification to permit the use of .as1 ASR transcripts.
- DET cloud graphs for detection and first story detection indicate
off-graph performance points.
- Modified the first story evaluation script to better manipulate
DET plots, and to read/write an answer key file.
Also, new index files are available for the October dry run. You can
find them at the URLs:
ftp://jaguar.ncsl.nist.gov/tdt3/oct_dryrun/indexes_oct_dry_run_asr_v1.tar.Z
ftp://jaguar.ncsl.nist.gov/tdt3/oct_dryrun/indexes_oct_dry_run_as1_v1.tar.Z
There are two complete sets of index files, the first uses the .asr ASR
transcripts, and the second uses the .as1 transcripts.
Both sets include Story Link Detection index files. Please note that
the story link index files may change prior to the dry run evaluation.
We are still in the process of defining the document selection process
for the TDT3 evaluation. If the finalized process differs from the
techniques used to generate the dry run index files, new ones will be
issued.
Jon
--
Jonathan Fiscus
National Inst. of Stds. and Tech.
100 Bureau Dr. Stop 8940
Gaithersburg, MD 20899-8940
Phone: (301) 975-3182
Email: jonathan.fiscus@nist.gov
(159) previous ~ index ~ next
Last updated Thu Aug 19 16:14:48 1999