Topics available; how many runs?

[Paul Over <over@nist.gov>]

Hi,

The 25 search topics are now available for download for active participants.
(The userid is Trec2002 and the password is the one you got from Lori.Buckland
@nist.gov when your group applied to participate in TREC. Please, no emails
to the trecvid list containing the password!)

25 is fewer than we had planned but that's perhaps ok. At this stage in video 
evaluation I personally don't think we know what a representative sample of 
video topics for our video archive model is, one from which we could draw broad 
conclusions about the effectiveness of various approaches. It may then be more
useful to have fewer topics, since it allows for more runs and more time for 
detailed failure/success analysis of each topic's results. 

The topics directory contains a file for each topic, the dtds, and a 
subdirectory with images used as imageExamples. The urls for these images 
appear in the topics. The videoExamples are taken mostly from feature 
development and test collections. In some cases they come from the search test 
collection - imagine a sort of "I found one of the sort I want, can you find 
more for me?" topic.

<http://www-nlpir.nist.gov/projects/t2002v/topics/>

The webpage (section 6.3)  contains the details for the submission formats. By 
the middle of next week I expect we will make a decision about the maximum number 
of runs a group can submit. 

Input about how many runs groups are seriously contemplating creating would 
be useful input this week.

Happy searching,
Paul

-- 
Paul Over - Retrieval Group
	    Information Access Division
	    Information Technology Laboratory
	    National Institute of Standards and Technology
	    Bldg. 225  Rm. A211  (Mailstop 8940)
	    Gaithersburg, MD  20899-8940   USA
	    Voice: 301 975-6784    Fax: 301 975-5287