(280) previous ~ index ~ next
To: TDT Distrib <tdt-distrib@ldc.upenn.edu>
From: Jonathan Fiscus <jonathan.fiscus@nist.gov>
Subject: Update to Software (v2.0) and Dry Run index files
Date: Wed, 30 Aug 2000 08:46:34 -0400
Folks,
I'm modified script to build index files in the evaluation suite to
conform to the new TDT evaluation specification V1.4. I've also
released a new version of the dry run index files that conform to the
evaluation specification. The URL to the updates are listed below.
The modifications involve the link detection and tracking index files.
Link Detection
--------------
The new link detection index files have more story pairs in the test
set. This was done to ensure there were enough trials to adequately
assess performance for the new conditions included in the evaluation,
Mandarin to Mandarin story comparisons and English to Mandarin story
comparisons. If you are participating in the link detection evaluation,
please use these new index files and report any problems, or interesting
results!
Tracking
--------
The new tracking index files have had a number of changes to them. The
file formats of the index files have not changed, however the
organization of index files within the release has changed.
First, per the evaluation specification, experiment control files were
added to the release. A control file exists for each evaluation
condition of: data source (nwt+bnasr or nwt+bnman), training language
(eng or man, the content language was deleted since it was redundant),
test language (mul,eng ot mul,nat) and topic training (NT=1, 2, 4 or
V). The previous release did not explicitly include the NT conditions.
They have been added since the test topics vary for the Nt condition
since there is a possibility of having topics with fewer that 4 training
stories.
The control files are a list of topic index files for the evaluation
condition which are located in single, monolithic directory. The idex
filenames have been changed to reduce their length, but still contain
the necessary information to disambiguate their contents. Please take a
look at this release as I will be releasing this structure for the fall
evaluation. If you find it difficult to deal with these new file names,
I could write a script to build links to them in old style.
URLS:
-----
New Eval software:
ftp://jaguar.ncsl.nist.gov/tdt/tdt2000/TDT3eval_v2.0.tgz
New Dry Run index files:
ftp://jaguar.ncsl.nist.gov/tdt/tdt2000/dryrun2000/dryrun2000_indexfiles.20000825.tgz
Regards
Jon
(280) previous ~ index ~ next
Last updated Tue Sep 19 14:30:59 2000