Stephen Robertson <firstname.lastname@example.org> writes:
> You might like to note that the original version has the rather
> curious property that rank 2 gets no discount. A version commonly
> used in Microsoft is adjusted to give a discount even on rank 2. How
> flexible are you making your implementation? Any gain
> function/discount function/truncation point?
A few people are using such a variant that discounts by log(rank + 1).
I like this approach better myself, but at this point I want to make
sure I've caught all the bugs and to do that I want to make sure I match
the specification of the measure in the ToIS article.
My implementation is based on trec_eval, and will allow specifying gain
values per relevance level. I think I will leave the log base as 2, as
I have not seen much research that shows much sensitivity in that
parameter. Truncation will be as usual in trec_eval (using the -M
I don't know if this will make it into the official trec_eval, but if
not I'll probably post my patch someplace.
- RE: NDCG
- From: Nick Craswell <email@example.com>
- From: Ian Soboroff <firstname.lastname@example.org>
- RE: NDCG
- From: Stephen Robertson <email@example.com>
Date Index |
Thread Index |
Problems or questions? Contact firstname.lastname@example.org