- Blind mode tutorial
lichess.org
Donate

Ratings Science is Hard

@biscuitfiend said in #10:

It's pretty ironic that you didn't credit xkcd/Randall Munroe for the comics.

Sorry... I fixed this now. I've credited him many times before if I recall, yet I failed to here.

@biscuitfiend said in #10:

As for the topic: if you looked at the data and that's what the data said, then I have to admit I can't argue; but the format of this blog post probably isn't the clearest way to present an in-depth data analysis, and I think @tpr has a good point.

In other words: if all you did was consider 1500s in this analysis, then you didn't really do your due diligence, because the assumption that it's the same at all rating levels is far from obviously true. If you can point to Glickman's work and say "he proved/justified the assumption" then that's another matter. But it's not immediately clear to me from reading the blog post that either of those previous "if" conditionals are met, meaning that in total, this blog post has raised more questions (for me) than it has answered.

I guess that means I agree with the title!

Indeed... I did look at other rating bands, and I did read Glickman's work, however data did not support this hypothesis. At the same time, having read Stockfish code and watched the j3m Elo explanation video and skimmed Elo's book, the only contrary hypothesis I've seen is: US Chess uses a logistic rating distribution model (to account for uncertainty at rating extrema), whereas FIDE uses a Gaussian model, so the first move advantage can't be equal of a fixed number of points using both distributions.

Stockfish's implementation of UCI_Elo models Elo loss (AI handicap levels) as a random value added to SF's evaluation which increases linearly with the number of Elo points lost. However, this isn't a proof that go, shogi, etc. handicaps can be applied to chess (that rating/rank differences equate to evaluation differences which also equate to piece or first move handicaps):
https://en.wikipedia.org/wiki/Universal_Chess_Interface#Features

@biscuitfiend said in #10: > It's pretty ironic that you didn't credit xkcd/Randall Munroe for the comics. Sorry... I fixed this now. I've credited him many times before if I recall, yet I failed to here. @biscuitfiend said in #10: > As for the topic: if you looked at the data and that's what the data said, then I have to admit I can't argue; but the format of this blog post probably isn't the clearest way to present an in-depth data analysis, and I think @tpr has a good point. > > In other words: if _all_ you did was consider 1500s in this analysis, then you didn't really do your due diligence, because the assumption that it's the same at all rating levels is far from obviously true. If you can point to Glickman's work and say "he proved/justified the assumption" then that's another matter. But it's not immediately clear to me from reading the blog post that either of those previous "if" conditionals are met, meaning that in total, this blog post has raised more questions (for me) than it has answered. > > I guess that means I agree with the title! Indeed... I did look at other rating bands, and I did read Glickman's work, however data did not support this hypothesis. At the same time, having read Stockfish code and watched the j3m Elo explanation video and skimmed Elo's book, the only contrary hypothesis I've seen is: US Chess uses a logistic rating distribution model (to account for uncertainty at rating extrema), whereas FIDE uses a Gaussian model, so the first move advantage can't be equal of a fixed number of points using both distributions. Stockfish's implementation of UCI_Elo models Elo loss (AI handicap levels) as a random value added to SF's evaluation which increases linearly with the number of Elo points lost. However, this isn't a proof that go, shogi, etc. handicaps can be applied to chess (that rating/rank differences equate to evaluation differences which also equate to piece or first move handicaps): https://en.wikipedia.org/wiki/Universal_Chess_Interface#Features