Re: training topics?

[Paul Over <over@nist.gov>]

"Arjen P. de Vries" wrote:
>
> Are you planning to release a (small) no. of training topics so people know
> what to expect?
> 
> Maybe you can point out topics from last year that are deemed "good topics",
> an option if creating two new sets of topics is problematic (I remember I
> found it hard last year to think of good things to ask on a video set)...

Arjen,

We don't plan to release any training topics. Even if we could afford to
create any, I don't think we have a reliable way of judging them to be
typical (good training material) and since the number would of necessity
be small, random sample would be deficient. There are sooo many dimensions 
to the possible variability and the importance of each dimension will vary 
from system to system I think.

We CAN say that they will be of the types listed in the guidelines and that
they will have relevent shots in the search test collection. This is a lot
of information I think. 

The topics from last year are of course possible training data. But I would 
leave it up to each group to decide, perhaps using the published results,
which are "good topics".


- Paul
-- 
Paul Over - Retrieval Group
	    Information Access Division
	    Information Technology Laboratory
	    National Institute of Standards and Technology
	    Bldg. 225  Rm. A211  (Mailstop 8940)
	    Gaithersburg, MD  20899-8940   USA
	    Voice: 301 975-6784    Fax: 301 975-5287