Question w.r.t. manual style B participation

["Arjen P. de Vries" <Arjen.de.Vries@cwi.nl>]

Hi Paul a.o.,

We had two different interpretations of the "rules" of participation in the
search task w.r.t. style A vs. B and manual vs. interactive participants,
and would like to check your opinion before we are labelled as "cheaters" :)

While style A uses knowledge of the search test set, and style B does not, 
there is also the distinction between manual and interactive, boiling down to
"manual does not have an interaction loop".

Now the question is whether a manual run can try out a proposed query on a
comparable collection (like the feature training set), never looking at the
search test set; in one view, this remains a style B manual run as the results
have not been tried on the search test set, or automatically become an
interactive run because a user has inspected search results (although on a
different collection).

In other words, is a manual participant allowed to try out, during topic
creation, its performance on a comparable collection, or does that necessarily
imply an interactive run?

Warm regards,

Arjen

====================================================================
CWI, room C0.11          Centre for Mathematics and Computer Science
Kruislaan 413                           Email: Arjen.de.Vries@cwi.nl
1098 SJ Amsterdam                       tel:       +31-(0)20-5924306
The Netherlands                         fax:       +31-(0)20-5924312  
===================== http://www.cwi.nl/~arjen/ ====================