Skill differences (comparison of chess and go)

[Schu]

Some of the questions I hear about board games kinda make me laugh, as well as the statements. One, from a rabid asiophile and big fan of go, was "go is more skillful than chess", after which I challenged him to a game of chess and he declined. Also I told him about chess' origins being Indian and therefore asian, just to mess with him a little.

But I find it interesting when people ask (or say) which game is harder, more skillful, whatever. I don't think most people realise what they're asking. I mean, first of all, a game itself can't be skillful. Second of all, what definition of harder are you going for? Requires more effort/time? In an even game, both chess and go should take equal time and effort from both players, so that's not it. Harder to learn? Both games have comparatively simple rules, though chess' in this case would have to be harder, but I don't think that's what they're asking. More prone to failure? Failure at chess is extremely contingent on the skill relative to you of your opponent.

I think the only real way to give any answer at all to the question is to frame the question as "which game makes it such that the smallest increase in skill results in one player having a reliable advantage against the other" or something like that. Now obviously skill is only measured in relative terms, so the "smallest increase in skill" has to be determined in a different way, most likely through analysis of rankings and ratings.

The way I'd do it is something like this: look up results for when players of a game ranked about #10 play players ranked about #100, and see what kind of elo point difference that would imply. do the same for #100 and #1000, and if world rankings still allow, #1000 and #10000. Then, to approximate players ranked about #100,000, you're probably looking for club champions etc., then for 1,000,000, maybe active competitive players etc.

Now, if multiplying the ranking by a factor of 10 results in a higher implied elo difference for one game than another, I think that game would have at least some claim to being the "more skillful game" or something like that, there are probably better words for it. I wonder though, I think checkers could possibly pip both, considering that the top player went decades with fewer than 10 defeats. How about sporting pursuits? Rarely do you see Federer or Nadal lose to someone not ranked in the top 20, but it certainly does happen.

Any thoughts?

[ER]

A very interesting topic for discussion!
I think rules and their application must also be taken under consideration in deciding the complexity, depth, skills required for a game/sport.
Luck (or dearth of it) is another factor as well as the physical, mental, psychological condition of players involved!
Upsets in mental sports as in Chess are very frequent and in the highest level!
BTW and in passing, could Fischer vs Spasky (1972) and Kramnik vs Kasparov (2000) World Championship results be considered as upsets?

[Schu]

Fischer's win was perhaps an upset in terms of psychology, since he'd never beaten him before, but I think most people realised that Fischer had been the best chessplayer around for about a decade, and if he ever got a chance to play at his full capacity for a WC and got over his own hoodoos, he'd have a good a chance as anyone. Just think about him destroying Taimanov and Larsen 6-0 and destroying Petrosian too. Kramnik's was a genuine upset. Euwe was another, Khalifman another, they happen.

Upsets are going to still happen though, you can't control everything, and people will play better than usual some times, worse other times. The only thing I'm worried about in this context is how much skill difference is needed before other factors become almost impossibly unlikely to change the outcome.

[pappubahry]

The problem with making these sorts of cross-sport/game comparisons is that there are two things that are changing:

- the structure of the game (this is what you're interested in studying)
- the number of people who play the game/sport

To elaborate on the second point, suppose someone wanted to answer your questions, but instead of chess and go, he compares the Australian sport of chess and the Russian sport of Шахматы.

They would see that in chess, the best player is Zhao, and he has about a 500-point Elo gap to the 50th best player. But the best Шахматы player is Jakovenko, and he's only about 200 points clear of the 50th best player. You can't conclude that the top tier of Australian chess is more skilful than Russia's....

It should be possible to properly control for differences in participation rates, but I'm not sure off the top of my head how you'd go about doing so.

[Kevin Bonham]

Kramnik's was a genuine upset. Euwe was another, Khalifman another, they happen.

Kramnik did have a very good record against Kasparov specifically and it was one of the rare cases where head-to-head scores would have been a better predictive tool than ratings. (I recall Sonas listing Kasparov-Kramnik as one of only five established player pairings for which this was demonstrably true though I do not recall how he derived that.)

Most wins of the FIDE knockout pseudo-World Championships were upset winners - Khalifman, Ponomariov and Kasimdzhanov were all nowhere near being the strongest players in the events they won. Only Anand was an expected winner. The short-match format with fast tiebreakers made it highly unlikely that the best player would actually win.

[Kevin Bonham]

I wonder though, I think checkers could possibly pip both, considering that the top player went decades with fewer than 10 defeats.

That is probably simply because checkers is more drawish when played at top level than chess; it is easier to avoid defeat.

Determining how to compare skill factors between different games seems very challenging to me. One issue is straining out the influence of luck. In a game that contains a strong chance element even great differences in skill will not necessarily result in a reliable victory for the superior player. But it may still be the case that massive differences in skill exist. I previously mentioned the Monopoly club I used to be involved in at the Uni, in which I ran a rating system and found that win rates by player in games with an average of about five players could be as low as 5% or as high as 40%, over very large numbers of games.

Another issue, related to chance, is the influence of sample size. Is a game of chess really comparable to a five-set match of tennis in terms of the extent to which the result will reflect a skill difference should one exist? Or is a game of chess more comparable to one set, or to a three-match series? Suppose you rated tennis on a similar system to ELO, the ratings would be closer together if you did the ratings set by set than if you did them match by match.

Sports with "granular" scoring systems also mess with the idea that a great difference in skill leads to a reliable victory. In soccer one team will sometimes win the match with a goal scored against the run of play, converting 1 of its 2 serious opportunities while the other team converts say 0 of 5. But that doesn't mean there is less skill in the game than in, say, Aussie Rules; what it means is that the disconnect between the skill and the scoreboard might be greater. Chess is a bit like this too because in a game between two good players of slightly different strengths, the most likely result is a draw.

[Schu]

The problem with making these sorts of cross-sport/game comparisons is that there are two things that are changing:

- the structure of the game (this is what you're interested in studying)
- the number of people who play the game/sport

To elaborate on the second point, suppose someone wanted to answer your questions, but instead of chess and go, he compares the Australian sport of chess and the Russian sport of Шахматы.

They would see that in chess, the best player is Zhao, and he has about a 500-point Elo gap to the 50th best player. But the best Шахматы player is Jakovenko, and he's only about 200 points clear of the 50th best player. You can't conclude that the top tier of Australian chess is more skilful than Russia's....

It should be possible to properly control for differences in participation rates, but I'm not sure off the top of my head how you'd go about doing so.

I'm sure there are ways. You could work out where players lie on a percentile score of all chess players that at the very least play club chess, or something like that. I'm no statistician though, so I wouldn't know how to go about that exactly.

[Schu]

That is probably simply because checkers is more drawish when played at top level than chess; it is easier to avoid defeat.

Well since checkers is solved as a draw, that is not so surprising. Clearly in checkers it is extremely diffucult to lose to someone that's worse than you, but the question remains how much advantage is required to overcome checkers demonstrated drawishness and get a win.

[Desmond]

I'm sure there are ways. You could work out where players lie on a percentile score of all chess players that at the very least play club chess, or something like that. I'm no statistician though, so I wouldn't know how to go about that exactly.

That would tell you how skillful a chessplayer is at chess against other chessplayers but not sure what inferences could be drawn compared to other games.

[Schu]

That would tell you how skillful a chessplayer is at chess against other chessplayers but not sure what inferences could be drawn compared to other games.

One could infer that if, by comparing game x and game y, x players come out as having results such that smaller degrees of skill difference (measured in possibly many different ways: percentile, standard deviation etc.) result in, say, 80% results, then that game is "more skillful" in the sense that this topic is concerned with.

[Desmond]

One could infer that if, by comparing game x and game y, x players come out as having results such that smaller degrees of skill difference (measured in possibly many different ways: percentile, standard deviation etc.) result in, say, 80% results, then that game is "more skillful" in the sense that this topic is concerned with.

But is skill just the probablity to win? Let me put this to you:

Let's say that to get from the level of beginner to master of a given game there is a number of "units of skill" that need to be attained. These units might be nuances, patterns, strategies, tactics etc whatever you want to call them depending on the given game, but let's just give them a number for simplicity.

The number of these units varies game to game. So for example game A might have 100 units and game B might have 1000.

A master in each game might have that 80% edge you mention, but how is your system going to report on the 100 vs 1000 thing?

The other point I was thinking of was that maybe in game A those 100 units of mastery might only yield a 10% advantage whereas in game B those 1000 units might yield 80% (or vice versa) but maybe that would be caught by the different measures you mention.

[Schu]

I'm not really sure why you bring units into it. Having more elements to a game doesn't neccesarily make it more skillful. I don't think the amount of elements in a game makes a difference to what I say, and that is how I'm interpreting your "units", correct me if I'm wrong there. You can't really measure skill with these units, because people will not have entire mastry of each of elements. The only way you can measure skill is by results.

All that my idea is trying to measure is how much of a skill difference (measured in percentiles, standard deviations, whatever system is appropriate at the time) transfers to what kind of advantage in terms of average results, and my assertion is that the game that has the least amount of skill difference required for a certain named advantage, this game is the "most skillful", or at least the one where the elements affecting the result other than skill have the lease influence.

Determining how to compare skill factors between different games seems very challenging to me. One issue is straining out the influence of luck. In a game that contains a strong chance element even great differences in skill will not necessarily result in a reliable victory for the superior player. But it may still be the case that massive differences in skill exist. I previously mentioned the Monopoly club I used to be involved in at the Uni, in which I ran a rating system and found that win rates by player in games with an average of about five players could be as low as 5% or as high as 40%, over very large numbers of games.

Another issue, related to chance, is the influence of sample size. Is a game of chess really comparable to a five-set match of tennis in terms of the extent to which the result will reflect a skill difference should one exist? Or is a game of chess more comparable to one set, or to a three-match series? Suppose you rated tennis on a similar system to ELO, the ratings would be closer together if you did the ratings set by set than if you did them match by match.

Sports with "granular" scoring systems also mess with the idea that a great difference in skill leads to a reliable victory. In soccer one team will sometimes win the match with a goal scored against the run of play, converting 1 of its 2 serious opportunities while the other team converts say 0 of 5. But that doesn't mean there is less skill in the game than in, say, Aussie Rules; what it means is that the disconnect between the skill and the scoreboard might be greater. Chess is a bit like this too because in a game between two good players of slightly different strengths, the most likely result is a draw.

I agree that this would be fairly problematic. There must be some statistical method for scaling to compensate for drawishness. I suppose a simple way would be to force all games to replay draws until a result is achieved.

[Hobbes]

I remember seeing some sort of comparison between chess skill and go skill once (30 seconds of googling failed to find it, sorry).

The method used there was to start with the best player in the world. Find somebody he scores 75% against. Then, find somebody who player B scores 75% against, etc, until you get reach the level of a complete beginner. Count the number of steps.

I think 200 rating point advantage means you expect to score 75%? Then in chess we would take about 15 steps to go from the world champion to a complete beginner. If I remember correctly, there were significantly more steps in Go than there were in chess.

[Desmond]

I'm not really sure why you bring units into it.

To make it easier to compare different games. So maybe Lucena position in chess is a skill. Maybe you want to assign that 1 unit of skill. Maybe in Go there is a skill that requires similar knowledge/experience/aptitude and you also want to assign that to be 1 unit. Then you add up all the skills in each game and you get the totals.

Not sure if that is what you meant by "elements" or not.

[Schu]

To make it easier to compare different games. So maybe Lucena position in chess is a skill. Maybe you want to assign that 1 unit of skill. Maybe in Go there is a skill that requires similar knowledge/experience/aptitude and you also want to assign that to be 1 unit. Then you add up all the skills in each game and you get the totals.

Not sure if that is what you meant by "elements" or not.

That's what I understood it to be. But I don't really get the applicability to this topic, if only because you can't quantify chess like that, you can accumulate skills like that but it won't represent your actual skill (though there will be some correlation of course). You can only judge a person's skill by results.

Hobbes: that's roughly what I'm thinking about, yeah, though I think getting the top player isn't that good idea, he might be an outlier. Thanks for your recollection 200 difference in elo gives 75.974% results, so close enough