(262) previous ~ index ~ next
To: tdt-distrib@ldc.upenn.edu
From: Yiming Yang <yiming@cs.cmu.edu>
Subject: detection evaluation: no-deferral not allowed?
Date: Fri, 11 Dec 1998 15:05:19 -0500
All,
I just failed when using the NIST's procedure, "TDT2det.pl", to
evaluate the output of our detection system with the deferral period
set to 0. Here is the error message:
yiming@moscow 19:37 [203] [dec98.expr] Illegal deferral period '0' != 1, 10 or 100 at /usr/local/bin/TDT2det.pl line 370, <SYS> chunk 1.
Illegal deferral period '0' != 1, 10 or 100 at /usr/local/bin/TDT2det.pl line 370, <SYS> chunk 1.
Illegal deferral period '0' != 1, 10 or 100 at /usr/local/bin/TDT2det.pl line 370, <SYS> chunk 1.
The question here is, are we allowed to compare our system under the
conditions of with and without decision deferral? Without such a
comparison, how would we know whether the deferral makes any
improvement? Moreover, do we have the choice of submiting detection
results without deferral if it turns out with a better result than
using deferral?
Any clarification would be helpful.
- Yiming
***************************************************************************
Yiming Yang
Senior Research Scientist (Associate Professor)
Computer Science Dept & Language Technology Institute
Cyert Hall 260 Phone: 412-268-1364
Carnegie Mellon University FAX: 412-268-6298
5000 Forbes Avenue http://www.cs.cmu.edu/~yiming/
Pittsburgh, PA 15213-3891 Secretary: Catherine S Morrow 412-268-2623
***************************************************************************
(262) previous ~ index ~ next
Last updated Mon Dec 14 09:26:55 1998