(165) previous ~ index ~ next
To: Ron Papka <papka@dandenong.cs.umass.edu>
From: Jonathan Fiscus <jonathan.fiscus@nist.gov>
Subject: Re: Tracking Results question
Date: Wed, 09 Sep 1998 09:23:53 -0400
Ron,
This does look wrong. Let's try to get to the bottom of this. Here are
the things off the top of my head.
1: verify that you're using TDTeval_v0.3 by execution the TDT2trk.pl
command with no arguments. The usage should indicate you're using
version 0.3.
2: verify that you're using version2 of the devtest index files.
3: verify that you're using a pristine version of the devtest corpus.
4: send me your results file and index file so I can score it here.
There's got to be an explaination...
Jon
Ron Papka wrote:
>
> Jon,
>
> I was looking at our tracking results, and perhaps I'm confusing
> the values in some of the columns. For topic 39, which contains
> 61 YES or BRIEF judgements for all sources I get the following:
>
> Tracking Performance Calculations:
>
> Filename Topic Train Test Corr Corr Miss F/A Pct. Pct. Ctrack
> Story Story Det. ! Det. Story Story Miss F/A
> -------- ----- ----- ------ ------ ------ ------ ------ ------ ------ ------
> tmp.39.trk 39 4 33661 3 33575 83 0 0.9651 0.0000 0.0193
>
> If I correctly detected 3 stories, how could I miss 83 stories for a
> topic with at most 61 stories?
>
> The number of Test Stories appears to be quite large i.e. 33K vs 20K,
> Could this be the source of the strange number in the Miss Story column ?
>
> Ron
--
Jon Fiscus
NIST
Email: jfiscus@nist.gov
Phone: (301) 975-3182
(165) previous ~ index ~ next
Last updated Wed Sep 9 09:40:57 1998