(199) previous ~ index ~ next
To: "Satya Dharanipragada" <dsatya@watson.ibm.com>
From: "George Doddington" <doddington@email.msn.com>
Subject: Re: need clarification ... on TDT2 evaluation rules
Date: Fri, 16 Oct 1998 12:13:19 -0400
>I need clarification on a this minor issue.
>We do preprocessing of the data (tagging, morphing etc)
>before we use it. Technically, since we are doing
>this offline, we are violating the deferral period
>restrictions. Is this going to be a problem?
The TDT community has been enjoying a rather relaxed attitude
regarding evaluation rules. Not to say that the rules aren't
strictly obeyed, only that we hew to the INTENT of the rules
rather than a mindless anal attitude. So, with this context,
and if I understand you correctly, I see no problem with your
batch-oriented pipeline processing, provided of course that
your preprocessing doesn't itself include deferral intervals.
--------
>Also can the dev-set be used for training or is it only for
>tuning purposes?
Strictly speaking, yes, the dev-set may be used for training.
Often, however, sites will hew to common conventions that do
exclude the dev-set from training, to facilitate comparison
of algorithms between sites. This hasn't been a subject of
discussion within TDT2, so I will leave this up to informal
decisions at each site. Hopefully you will discuss this
with your colleagues at other TDT2 sites. Happy research!
-----------------------
George Doddington in McLean, VA. doddington@nist.gov or 703/556-3434
(199) previous ~ index ~ next
Last updated Wed Oct 28 14:44:11 1998