(101) previous ~ index ~ next
To: tdt-distrib <tdt-distrib@ldc.upenn.edu>
From: Jonathan Fiscus <jonathan.fiscus@nist.gov>
Subject: TDT3 eval software and index files
Date: Fri, 21 May 1999 15:32:24 -0400
Folks,
Well... I finally have the first version of the TDT3 evaluation
software ready!!! There have been a considerable number of changes made
to the software which include:
Support for the multilingual evaluations
Topic Weighted DET Curves for Tracking
YDZ performance measures for Detection
For Detection Evaluation: the ability to score subsets of a test set
Bug fixes:
1) Erroneous mapping of hypothsized story boundaries to reference
story boundaries for the unknown story boundary conditions
- This does effect TDT2 evaluation scores very slightly.
2) Omission of possible test stories in the detection evaluation
- This does effect TDT2 evaluation scores slightly, detection
costs increase by 0.0002 on average.
I have also found problems with version 2 of the example index files, so
there's a new version of index files too.
The software and index files are available at the URLs:
ftp://jaguar.ncsl.nist.gov/tdt3/TDT3eval_v1.0.tgz
ftp://jaguar.ncsl.nist.gov/tdt3/tdt3_indexes_jan-mar_v3.tgz
ftp://jaguar.ncsl.nist.gov/tdt3/tdt3_indexes_jan-mar_v3/index.htm
So, In the interest of getting things rolling for the dry-run. What
else do people need besides:
a) The second half (May-June) of the Mandarin data
b) First Story Detection Evaluation software
(which, BTW, is next on my todo list)
- Jon
PS. I hear those engines rev'in
--
Jonathan Fiscus Snailmail: Nat'l Inst. of Stds. and Tech.
NIST 100 Bureau Dr. Stop 8940
Phone: (301) 975-3182 Gaithersburg, MD 20899-8940
Email: jfiscus@nist.gov
(101) previous ~ index ~ next
Last updated Mon Jun 21 11:18:47 1999