(326) previous ~ index ~ next
To: TDT Distrib <tdt-distrib@ldc.upenn.edu>
From: Jonathan Fiscus <jonathan.fiscus@nist.gov>
Subject: Evaluation of Topic Detection
Date: Thu, 12 Jul 2001 12:55:09 -0400
Folks,
At the last meeting, we discussed an alternative technique for
evaluating a topic detection system using a "link" metric. The problem
we wanted to address was the erratic nature of the topic detection error
rates when a parameter is varied. For example, Dragon presented a graph
that showed this behavior
'http://www.nist.gov/speech/tests/tdt/tdt2000/Papers-n-slides/Dragon-pres/NistDetectionTalk_files/v3_document.htm'.
They ran a series of detection runs that varied a parameter to limit the
topic size, and the resulting error rates are erratic.
Since the last meeting, I've implemented some scripts to do the link
error calculation this, but I need some experimental data to test the
software. Is there anyone actively developing a topic detection system
that could supply me with this type of data? I'd need the index file
and a set of system output files that where produced by varying a model
parameter.
Thanks
Jon
-------------------------------------------------------------
To unsubscribe from tdt-distrib, email majordomo@ldc.upenn.edu
with "unsubscribe tdt-distrib" in the body of the message.
(326) previous ~ index ~ next
Last updated Wed Aug 22 16:07:34 2001