RE: NDCG

[Nick Craswell <nickcr@microsoft.com>]

> Thank you everyone, now I understand that Microsoft's nDCG
> does not have a "persistence parameter" (the log base).

That's a good way of putting it.

> Sorry

Actually: I apologise, I said something about NDCG that is only true of the MS variant.

> Anyway, which paper was the first to use this version of nDCG?

Perhaps:

Burges, C., Shaked, T., Renshaw, E., Lazier, A., Deeds, M., Hamilton, N., and Hullender, G. 2005. Learning to rank using gradient descent. In Proceedings of the 22nd international Conference on Machine Learning (Bonn, Germany, August 07 - 11, 2005). ICML '05, vol. 119. ACM, New York, NY, 89-96.
http://doi.acm.org/10.1145/1102351.1102363

cheers,
Nick.

> -----Original Message-----
> From: ireval@nist.gov [mailto:ireval@nist.gov] On Behalf Of Tetsuya
> Sakai (office)
> Sent: 26 November 2007 04:24
> To: Multiple recipients of list
> Subject: Re: NDCG
>
> Thank you everyone, now I understand that Microsoft's nDCG does not
> have a
> "persistence parameter" (the log base).
> Sorry about my dumb comments, Nick.
>
> Anyway, which paper was the first to use this version of nDCG?
>
> Dr. Tetsuya Sakai (sakai@newswatch.co.jp)
> http://voice.fresheye.com/sakai/
>
> >> From: ireval@nist.gov [mailto:ireval@nist.gov] On Behalf Of Kalervo
> >> Jarvelin
> >
> >> This is a good summary. A point to add is that if one wants to
> >> consistently discount the gains, the log(rank+1) variant runs the
> risk
> >> of boosting (instead of discounting) at ranks below the log base.
> With
> >> base=2 this does not matter.
> >
> > Just a small comment on this.  It is true that the absolute gain
> values
> > are boosted in those conditions -- however, the relativities still
> work
> > (the "discounted" gain for rank n+1 is always less than the
> "discounted"
> > gain for the same relevance value at rank n).  Furthermore, the issue
> > disappears as soon as you normalize.
> >
> > Stephen
>