- Blind mode tutorial
lichess.org
Donate

In Search of Brilliancies in Chess

@HollowLeaf

Fascinating.

So a potential brilliancy = Low probability of Stockfish Best Move according to Maia. [Where difference between Stockfish Best Move and Stockfish 2nd Move is large]

Good idea!

You asked for thoughts so here they are:

I hope you enjoyed reading this blog. I just wanted to put my thoughts onto paper and see what people think. Has this been done before.

It's been 'half done' before. https://saychess.substack.com/p/finding-brilliant-moves-with-engines

Julian from Chess Engine Lab had the same brilliancy detector that you suggest. But they used Leela's policy output (without Monte Carlo Tree Search) instead of Maia.

A brilliancy is detected if:

  • The probability of the Stockfish Best Move was below 2.5% according to Leela's policy output.
  • The Win probability of Stockfish Best Move is at least 15% larger than the Stockfish 2nd Best Move.

Julian concluded :

The results look very promising, but since brilliant moves are quite subjective, it’s hard to determine how “correct” this approach is. My main hope was that not too many moves should be called brilliant because brilliant moves should be rare. I do think that my current approach might even be on the more conservative side.

Would be interesting to try with Maia. Maia is in the 1100-1900 range so the brillancies will be relative to this skill level. Stronger players might not find them to be brilliancies.

So can brilliancies be relative? I guess. Brilliant for your level.

@HollowLeaf Fascinating. So a potential brilliancy = Low probability of Stockfish Best Move according to Maia. [Where difference between Stockfish Best Move and Stockfish 2nd Move is large] Good idea! ----- You asked for thoughts so here they are: >I hope you enjoyed reading this blog. I just wanted to put my thoughts onto paper and see what people think. Has this been done before. It's been 'half done' before. https://saychess.substack.com/p/finding-brilliant-moves-with-engines Julian from Chess Engine Lab had the same brilliancy detector that you suggest. But they used Leela's policy output (without Monte Carlo Tree Search) instead of Maia. A brilliancy is detected if: - The probability of the Stockfish Best Move was below 2.5% according to Leela's policy output. - The Win probability of Stockfish Best Move is at least 15% larger than the Stockfish 2nd Best Move. Julian concluded : ``` The results look very promising, but since brilliant moves are quite subjective, it’s hard to determine how “correct” this approach is. My main hope was that not too many moves should be called brilliant because brilliant moves should be rare. I do think that my current approach might even be on the more conservative side. ``` -------- Would be interesting to try with Maia. Maia is in the 1100-1900 range so the brillancies will be relative to this skill level. Stronger players might not find them to be brilliancies. So can brilliancies be relative? I guess. Brilliant for your level.

I believe a similar idea has been proposed for assessing the difficulty of chess puzzles. Though I believe the idea there was to feed the puzzle to Maia bots of different rating and look for the rating level at which Maia correctly solves the puzzle.

I don't think that a "brilliant" move necessarily needs to be an only move. Nor do I think that it even needs to be the best move. It just needs to be something that works well enough despite looking like it should lose. To quantify it, maybe it should be within a centipawn or two of Stockfish's best move while being a low-probablility move from Maia.

Do you plan to count known opening traps as "brilliant"? That's a tough one. I think conventionally those would be considered "book" moves and not "brilliant" for someone who has studied them. Though they most certainly would have been brilliant for the first person who played them.

I believe a similar idea has been proposed for assessing the difficulty of chess puzzles. Though I believe the idea there was to feed the puzzle to Maia bots of different rating and look for the rating level at which Maia correctly solves the puzzle. I don't think that a "brilliant" move necessarily needs to be an only move. Nor do I think that it even needs to be the best move. It just needs to be something that works well enough despite looking like it should lose. To quantify it, maybe it should be within a centipawn or two of Stockfish's best move while being a low-probablility move from Maia. Do you plan to count known opening traps as "brilliant"? That's a tough one. I think conventionally those would be considered "book" moves and not "brilliant" for someone who has studied them. Though they most certainly would have been brilliant for the first person who played them.

@UncleRogerJr said in #2:

Love this idea!

Thanks, it is an interesting thought so I am curious about the result when implemented.

@UncleRogerJr said in #2: > Love this idea! Thanks, it is an interesting thought so I am curious about the result when implemented.

@RuyLopez1000 said in #3:

@HollowLeaf

@RuyLopez1000 said in #3:

Fascinating.

So a potential brilliancy = Low probability of Stockfish Best Move according to Maia. [Where difference between Stockfish Best Move and Stockfish 2nd Move is large]

Good idea!

Yeah. I am drawn to the only move scenario i.e. a huge gap between the first and second, line from stockfish, and unlikely according to the Maia level.

@RuyLopez1000 said in #3: > @HollowLeaf @RuyLopez1000 said in #3: > Fascinating. > > So a potential brilliancy = Low probability of Stockfish Best Move according to Maia. [Where difference between Stockfish Best Move and Stockfish 2nd Move is large] > > Good idea! Yeah. I am drawn to the only move scenario i.e. a huge gap between the first and second, line from stockfish, and unlikely according to the Maia level.

@RuyLopez1000 said in #3:

Julian from Chess Engine Lab had the same brilliancy detector that you suggest. But they used Leela's policy output (without Monte Carlo Tree Search) instead of Maia.

A brilliancy is detected if:

  • The probability of the Stockfish Best Move was below 2.5% according to Leela's policy output.
  • The Win probability of Stockfish Best Move is at least 15% larger than the Stockfish 2nd Best Move.

Julian concluded :

Thanks for this, I will check it out and see what has been done before.

@RuyLopez1000 said in #3: > Julian from Chess Engine Lab had the same brilliancy detector that you suggest. But they used Leela's policy output (without Monte Carlo Tree Search) instead of Maia. > > A brilliancy is detected if: > > - The probability of the Stockfish Best Move was below 2.5% according to Leela's policy output. > - The Win probability of Stockfish Best Move is at least 15% larger than the Stockfish 2nd Best Move. > > Julian concluded : Thanks for this, I will check it out and see what has been done before.

@RuyLopez1000 said in #3:

Would be interesting to try with Maia. Maia is in the 1100-1900 range so the brillancies will be relative to this skill level. Stronger players might not find them to be brilliancies.

So can brilliancies be relative? I guess. Brilliant for your level.

Yep. Once the implementation is done, I would be curious about the results. If it is good, I would probably try to incorporate this into my applications.

@RuyLopez1000 said in #3: > Would be interesting to try with Maia. Maia is in the 1100-1900 range so the brillancies will be relative to this skill level. Stronger players might not find them to be brilliancies. > > So can brilliancies be relative? I guess. Brilliant for your level. Yep. Once the implementation is done, I would be curious about the results. If it is good, I would probably try to incorporate this into my applications.

@goodspellr said in #4:

Do you plan to count known opening traps as "brilliant"? That's a tough one. I think conventionally those would be considered "book" moves and not "brilliant" for someone who has studied them. Though they most certainly would have been brilliant for the first person who played them.

I am probably going to try to exclude known opening theory, probably using the Lichess openings database. I don't think that opening theory should be included, at least not for this purpose.

@goodspellr said in #4: > Do you plan to count known opening traps as "brilliant"? That's a tough one. I think conventionally those would be considered "book" moves and not "brilliant" for someone who has studied them. Though they most certainly would have been brilliant for the first person who played them. I am probably going to try to exclude known opening theory, probably using the Lichess openings database. I don't think that opening theory should be included, at least not for this purpose.

I think you guys are missing an important point. This method uses two different and very large engines. This means resource utilisation in terms of space, processing and battery life (and money, if planning to do it on the server). And in the end you still don't know WHY it is brilliant.

I think you guys are missing an important point. This method uses two different and very large engines. This means resource utilisation in terms of space, processing and battery life (and money, if planning to do it on the server). And in the end you still don't know WHY it is brilliant.