(141) previous ~ index ~ next
To: Paul van Mulbregt <paulvm@dragonsys.com>
From: Jonathan Fiscus <jonathan.fiscus@nist.gov>
Subject: Re: ASR output
Date: Fri, 02 Jul 1999 08:31:31 -0400
Paul,
There is another reason for having two recognizer outputs (although
combining the recognizers sounds like a good experiment), and that is to
make the TDT3 ASR evaluation data be generated by the same recognizer
for the training data.
Since Dragon will not be running their recognizer on the TDT3 English
data, NIST ran a BBN recognizer on TDT2 and we'll soon be running it on
TDT3 English for the evaluation.
Jon
Paul van Mulbregt wrote:
>
> TDT'ers,
>
> At the meeting I requested that all ASR output meet the original spec
> regarding marked silences, speaker cluster and confidence. There are
> two reasons for this.
> The first is so that the data files run through the systems without any
> changes.
> Obviously a simple perl script could add X lines, insert a zero speaker
> cluster id,
> and add conf=NA to the lines with word records, making such runs possible.
>
> But the second addresses why we even have multiple ASR output at all. If
> the idea is rerecognize each year and get better recognition, and the TDT2
> ASR rerecognition probably isn't any better (according to Rich) than the
> original recognition, then results on this data don't tell us much
> (assuming the two recognizers have been tuned for WER rather than to produce
> good TDT output). However if the confidences are available, then the
> option exists of "roverizing" the outputs and coming up with a better
> recognition, and then one can start to measure the WER effect on
> the ASR segmentation task.
> So even though the confidences by themselves may not help
> segmentation/tracking/detection performance, when used in conjunction
> with other recognizers, they can help answer questions about the effect of WER
> on the segmentation task, and tracking/detection without boundaries.
>
> -- Paul
>
> ------------------------------------------------------------------
> Paul van Mulbregt, Dragon Systems Inc., Newton, MA. (617) 965-5200
> email: paulvm@dragonsys.com
--
Jonathan Fiscus Snailmail: Nat'l Inst. of Stds. and Tech.
NIST 100 Bureau Dr. Stop 8940
Phone: (301) 975-3182 Gaithersburg, MD 20899-8940
Email: jfiscus@nist.gov
(141) previous ~ index ~ next
Last updated Mon Jul 12 17:16:49 1999