Re: search submissions
- Subject: Re: search submissions
- From: Giridharan Iyengar <giyengar@us.ibm.com>
- Date: Thu, 12 Sep 2002 09:43:05 -0400
- Content-type: text/plain; charset=US-ASCII
Paul:
Thanks for the tabulation of search submissions. The number of submissions
looks very good and let us hope this trend continues to grow.
I have a question regarding classification of runs as type A or B. Since an
Interactive run means you look at the search content while answering the
query, I am a little confused about some interactive runs that are
classified as type B. It seems that interactive runs have to be type A
automatically.
Could you explain my possible misinterpretation?
Thanks,
-giri
Giridharan Iyengar
Research Staff Member
Human Language Technologies
IBM TJ Watson Research Center
Yorktown Heights, NY 10598
914 945 2474
Paul Over
<over@nist.gov> To: Multiple recipients of list <trecvid@nist.gov>
Sent by: cc:
trecvid@nist.gov Subject: search submissions
09/11/2002 08:18
AM
Please respond to
trecvid
Hi,
We received 39 search runs from 12 groups!
Most runs required some corrections and xml fixes. I've been
in touch about the serious ones.
Many included example shots from the topic. I commented those out.
Here is our mapping of topic examples to master shots:
tNum="076"
shot133_37
shot133_38
shot133_39
shot133_43
shot133_44
tNum="090">
shot78_7
tNum="094"
shot58_7
tNum="096"
shot17_59
tNum="097"
shot161_36
shot158_58
shot158_59
A couple groups (I've emailed them) included result items from files
not in the test collection. I commented those out.
For your information, I'm including the following table to give
you an overview of what was submitted as search runs. You'll
need to view it using a monospace font. Run/sysId names have been
truncated. You should be able to recognixe yours. Only in one
or two cases did I change them for clarity's sake.
I = interactive, M = manual; A = (trained on test set)
B = not; next number is a run priority; then the topic number
if a non-empty result was submitted for that topic.
Monospace required- !
Sys1 M B 1 75 76 77 78 79 80 85 86 87 90
93 94 97
Sys2 M B 2 75 76 79 86
93
Sys3 M B 3 75 76 77 78 79 80 85 86
93 94
Sys4 M B 4 75 76 77 78 79 80 83 84 85 86 87 89 90
92 93 94 95 97 98 99
MKM-1 I B 1 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90
91 92 93 94 95 96 97 98 99
MKM-2 M B 2 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90
91 92 93 94 95 96 97 98 99
MKM-3 M B 3 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90
91 92 93 94 95 96 97 98 99
MKM-4 M B 4 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90
91 92 93 94 95 96 97 98 99
0_T M B 2 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90
91 92 93 94 95 96 97 98 99
0_TIac M B 1 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90
91 92 93 94 95 96 97 98 99
0_TIsc M B 3 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90
91 92 93 94 95 96 97 98 99
0_TIscG M B 4 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90
91 92 93 94 95 96 97 98 99
Video.1 I B 1 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90
91 92 93 94 95 96 97 98 99
Video.2.A I B 1 76 77 78 79 83 84 86 87 88 89 90
91 92 93 94 95 97 98 99
Video.3 M B 1 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90
91 92 93 94 95 96 97 98 99
_MT5 I B 1 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90
91 92 93 94 95 96 97 98 99
_MT1 M B 2 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90
91 92 93 94 95 96 97 98 99
_MT2 M B 3 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90
91 92 93 94 95 96 97 98 99
_MT3 M B 4 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90
92 93 94 95 96 97 98 99
ci M B 1 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90
91 92 93 94 95 96 97 98 99
M-1 M B 1 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90
91 92 93 94 95 96 97 98 99
M-2 M B 2 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90
91 92 93 94 95 96 97 98 99
M-3 M B 3 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90
91 92 93 94 95 96 97 98 99
M-4 I A 4 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90
91 92 93 94 95 96 97 98 99
INTERACTIVE_2 I B 1 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90
92 93 94 95 96 97 98 99
InfInt1 I B 1 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90
91 92 93 94 95 96 97 98 99
_MANUAL1 M B 1 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90
91 92 93 94 95 96 97 98 99
_MANUAL2 M A 4 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90
91 92 93 94 95 96 97 98 99
_MANUAL3 M A 5 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90
92 93 94 95 96 97 98 99
S+ASR+X M A 1 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90
91 92 93 94 95 96 97 98 99
S+ASR M A 2 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90
91 92 93 94 95 96 97 98 99
S+X M A 3 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90
91 92 93 94 95 96 97 98 99
Trec11B I B 1 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90
92 93 94 95 96 97 98 99
Trec11C I B 2 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90
92 93 94 95 96 97 98 99
Trec11B I B 3 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90
92 93 94 95 96 97 98 99
Trec11C I B 4 76 77 78 79 80 81 82 83 84 86 87 88 89 90
92 93 94 97 98 99
Vf1 I A 1 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90
91 92 93 94 95 96 97 98 99
Mqtrec M B 2 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90
91 92 93 94 95 96 97 98 99
MNAqt M B 3 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90
91 92 93 94 95 96 97 98 99
Iqtrec I B 1 76 77 78 79 80 82 83 84 86 87 88 89 90
92 93 94 95 96 97 98 99
Assessing begins on Tuesday.
- Paul
--
Paul Over - Retrieval Group
Information Access Division
Information Technology Laboratory
National Institute of Standards and Technology
Bldg. 225 Rm. A211 (Mailstop 8940)
Gaithersburg, MD 20899-8940 USA
Voice: 301 975-6784 Fax: 301 975-5287
Date Index |
Thread Index |
Problems or questions? Contact list-master@nist.gov