Re: Topics available; how many runs?

[Djoerd Hiemstra <hiemstra@cs.utwente.nl>]

Dear Paul,

Some comments on the topics:
  - I found a small error in topic 92: The second image example
    starts with "(imageExample" instead of "<imageExample"...
  - The 3rd and 4th video example of topic 79:
    "kids play in the sand" and "people on the beach with palm trees"
    seem to be different frames in our collection (the second shows
    a book...)
  - many files in imageExamples are not referenced in any topic.
    e.g. cadwell3.jpg, mushroom.jpg, skull.gif, and many more.

Best,  Djoerd.

> Date: Wed, 7 Aug 2002 08:33:56 -0400 (EDT)
> Originator: trecvid@nist.gov
> From: Paul Over <over@nist.gov>
> To: Multiple recipients of list <trecvid@nist.gov>
> Subject: Topics available; how many runs?
> X-Listprocessor-Version: 6.0c -- ListProcessor by Anastasios Kotsikonas
> X-To: trecvid@nist.gov
> 
> 
> Hi,
> 
> The 25 search topics are now available for download for active participants.
> (The userid is Trec2002 and the password is the one you got from Lori.Buckland
> @nist.gov when your group applied to participate in TREC. Please, no emails
> to the trecvid list containing the password!)
> 
> 25 is fewer than we had planned but that's perhaps ok. At this stage in video 
> evaluation I personally don't think we know what a representative sample of 
> video topics for our video archive model is, one from which we could draw 
broad 
> conclusions about the effectiveness of various approaches. It may then be more
> useful to have fewer topics, since it allows for more runs and more time for 
> detailed failure/success analysis of each topic's results. 
> 
> The topics directory contains a file for each topic, the dtds, and a 
> subdirectory with images used as imageExamples. The urls for these images 
> appear in the topics. The videoExamples are taken mostly from feature 
> development and test collections. In some cases they come from the search test 
> collection - imagine a sort of "I found one of the sort I want, can you find 
> more for me?" topic.
> 
> <http://www-nlpir.nist.gov/projects/t2002v/topics/>
> 
> The webpage (section 6.3)  contains the details for the submission formats. By 
> the middle of next week I expect we will make a decision about the maximum 
number 
> of runs a group can submit. 
> 
> Input about how many runs groups are seriously contemplating creating would 
> be useful input this week.
> 
> Happy searching,
> Paul
> 
> -- 
> Paul Over - Retrieval Group
> 	    Information Access Division
> 	    Information Technology Laboratory
> 	    National Institute of Standards and Technology
> 	    Bldg. 225  Rm. A211  (Mailstop 8940)
> 	    Gaithersburg, MD  20899-8940   USA
> 	    Voice: 301 975-6784    Fax: 301 975-5287
>