Re: interactive A vs B

["John R Smith" <jsmith@us.ibm.com>]

Hi Paul,

Since we seem to be among only a couple of participants classifying an
interactive run as type "A", to clarify, our interactive run was done
without any knowledge of search test for system development.  Therefore,
given the distinction below, we can consider it to be type "B" for system
development and type "A" for processing.

Thanks,

John

---------------------------------------------------

Manager, Pervasive Media Management
IBM T. J. Watson Research Center
30 Saw Mill River Road
Hawthorne, NY 10532
(914) 784-7320; jrsmith@watson.ibm.com

---------------------------------------------------



|---------+---------------------------->
|         |           Paul Over        |
|         |           <over@nist.gov>  |
|         |           Sent by:         |
|         |           trecvid@nist.gov |
|         |                            |
|         |                            |
|         |           09/12/2002 10:50 |
|         |           AM               |
|         |           Please respond to|
|         |           trecvid          |
|---------+---------------------------->
  >-------------------------------------------------------------------------------------------------------------------------------|
  |                                                                                                                               |
  |       To:       Multiple recipients of list <trecvid@nist.gov>                                                                |
  |       cc:                                                                                                                     |
  |       Subject:  interactive A vs B                                                                                            |
  |                                                                                                                               |
  |                                                                                                                               |
  >-------------------------------------------------------------------------------------------------------------------------------|




Giri -

Clearly interactive runs normally make use of knowledge of the search test
collection, So I understand you're expectation that all interactive runs
should be predictably/redundantly labeled A, given that the guidelines talk

about A vs B in terms of system development AND  p r o c e s s i n g.

I suspect some groups may have ignored the "processing" part, and used the
A/B
distinction just to tell us about system development. Then an interactive
run
could make use of an A or B system. This actually seems like an improvement
on
the guidelines since the use of search test collection info during the
query
processing is covered by the manual vs interactive distinction.

But let's let those who labeled their interactive runs B (or A for that
matter),
tell us what they meant.

- Paul
--
Paul Over - Retrieval Group
                 Information Access Division
                 Information Technology Laboratory
                 National Institute of Standards and Technology
                 Bldg. 225  Rm. A211  (Mailstop 8940)
                 Gaithersburg, MD  20899-8940   USA
                 Voice: 301 975-6784    Fax: 301 975-5287