(031) previous ~ index ~ next

---

To: Alvin Martin <alvin.martin@nist.gov>
From: Hubert Jin <hjin@bbn.com>
Subject: Re: TDT3, a variety of issues
Date: Wed, 17 Feb 1999 12:41:53 -0500 (EST)

---

Alvin,

Do you still need more performance scores by topic for the same system run
on both the development and the evaluation data? Please let me know if you
need those.

Regarding the median test you perform on the data, I do have a concern on
it. Median test can only tell if the medians of the two distributions are
the same or not. It can not test if the standard deviations or the shapes
of the distributions are the same. Two distributions can be quite different
even if they have the same medians.

Could you also run "Kolmogorov-Smirnov two sample test" on the data to see
if the distributions are significantly different?

Thanks,

-Hubert

On Tue, 16 Feb 1999, Alvin Martin wrote:

> Ralf Brown has shared with me some of his performance results for the TDT
> tracking task, for which I thank him. This has enabled me to look at topic
> performance scores based on the defined cost function as promised by George.
> Discussion of this in Microsoft Word or postscript form is now available at:
>
> ftp://jaguar.ncsl.nist.gov/tdt98/topic2.doc
> ftp://jaguar.ncsl.nist.gov/tdt98/topic2.ps
>
> The previous discussion based only on the numbers of stories per topic is
> similarly available at;
>
> ftp://jaguar.ncsl.nist.gov/tdt98/topic.doc
> ftp://jaguar.ncsl.nist.gov/tdt98/topic.ps
>
>
> Alvin Martin wrote:
>
> > The attached document, in Microsoft Word or postscript forms, looks at the
> > numbers
> > of stories by topic for the training, development, and evaluation sets as
> > promised by
> > George.
> >
> > Does anyone have performance scores by topic for the same system run on both
> > the
> > development and the evaluation data? I still need this to consider
> > performance based
> > differences.
> >

---

(031) previous ~ index ~ next

Last updated Thu May 13 09:28:15 1999