search submissions
- Subject: search submissions
- From: Paul Over <over@nist.gov>
- Date: Wed, 11 Sep 2002 08:16:06 -0400
- Content-Transfer-Encoding: 7bit
- Content-Type: text/plain; charset=us-ascii
- Organization: National Institute of Standards and Technology
- Sender: over
Hi,
We received 39 search runs from 12 groups!
Most runs required some corrections and xml fixes. I've been
in touch about the serious ones.
Many included example shots from the topic. I commented those out.
Here is our mapping of topic examples to master shots:
tNum="076"
shot133_37
shot133_38
shot133_39
shot133_43
shot133_44
tNum="090">
shot78_7
tNum="094"
shot58_7
tNum="096"
shot17_59
tNum="097"
shot161_36
shot158_58
shot158_59
A couple groups (I've emailed them) included result items from files
not in the test collection. I commented those out.
For your information, I'm including the following table to give
you an overview of what was submitted as search runs. You'll
need to view it using a monospace font. Run/sysId names have been
truncated. You should be able to recognixe yours. Only in one
or two cases did I change them for clarity's sake.
I = interactive, M = manual; A = (trained on test set)
B = not; next number is a run priority; then the topic number
if a non-empty result was submitted for that topic.
Monospace required- !
Sys1 M B 1 75 76 77 78 79 80 85 86 87 90 93 94 97
Sys2 M B 2 75 76 79 86 93
Sys3 M B 3 75 76 77 78 79 80 85 86 93 94
Sys4 M B 4 75 76 77 78 79 80 83 84 85 86 87 89 90 92 93 94 95 97 98 99
MKM-1 I B 1 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
MKM-2 M B 2 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
MKM-3 M B 3 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
MKM-4 M B 4 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
0_T M B 2 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
0_TIac M B 1 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
0_TIsc M B 3 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
0_TIscG M B 4 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
Video.1 I B 1 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
Video.2.A I B 1 76 77 78 79 83 84 86 87 88 89 90 91 92 93 94 95 97 98 99
Video.3 M B 1 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
_MT5 I B 1 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
_MT1 M B 2 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
_MT2 M B 3 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
_MT3 M B 4 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 92 93 94 95 96 97 98 99
ci M B 1 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
M-1 M B 1 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
M-2 M B 2 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
M-3 M B 3 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
M-4 I A 4 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
INTERACTIVE_2 I B 1 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 92 93 94 95 96 97 98 99
InfInt1 I B 1 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
_MANUAL1 M B 1 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
_MANUAL2 M A 4 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
_MANUAL3 M A 5 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 92 93 94 95 96 97 98 99
S+ASR+X M A 1 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
S+ASR M A 2 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
S+X M A 3 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
Trec11B I B 1 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 92 93 94 95 96 97 98 99
Trec11C I B 2 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 92 93 94 95 96 97 98 99
Trec11B I B 3 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 92 93 94 95 96 97 98 99
Trec11C I B 4 76 77 78 79 80 81 82 83 84 86 87 88 89 90 92 93 94 97 98 99
Vf1 I A 1 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
Mqtrec M B 2 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
MNAqt M B 3 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
Iqtrec I B 1 76 77 78 79 80 82 83 84 86 87 88 89 90 92 93 94 95 96 97 98 99
Assessing begins on Tuesday.
- Paul
--
Paul Over - Retrieval Group
Information Access Division
Information Technology Laboratory
National Institute of Standards and Technology
Bldg. 225 Rm. A211 (Mailstop 8940)
Gaithersburg, MD 20899-8940 USA
Voice: 301 975-6784 Fax: 301 975-5287
Date Index |
Thread Index |
Problems or questions? Contact list-master@nist.gov