Page 5 of 17 FirstFirst ... 3456715 ... LastLast
Results 61 to 75 of 251
  1. #61
    Monster of the deep Kevin Bonham's Avatar
    Join Date
    Jan 2004
    Posts
    37,598
    Quote Originally Posted by Bulldozer View Post
    It can be interesting, especially for Kevin. http://centralsquares.com/2015/06/11...mihaela-sandu/
    Bayesian inference makes sense. What do you think? Yes, we don't have enough statistical data about cheaters yet but the method is interesting.
    Getting everything right in such a calculation is very difficult. The following are among the issues I have with it:

    1. The analysis assumes equal probability for a win with each colour. In reality this is not the case. A win against a stronger player is significantly more likely with white and significantly less likely with black. This makes Sandu's streak somewhat less probable (not sure how much).

    2. The analysis assumes that when the transmission is off, Sandu reverts to the scoring probability given by her rating. But that assumes that if Sandu is cheating now then she has never cheated before. It may be that she has cheated before and that she is actually weaker than her rating.

    3. The same point you made: if Sandu was only cheating for some moves then her win probability if cheating is probably lower. Probably not as low as .8 though. However because the cheats who are caught are the stupid cheats (the ones who cheat more or less every move, who would have win probabilities higher than .9) we have no idea how prevalent careful cheating is. I hope it's not as common as 1/500 or even 1/1000 but it might be. Getting the prior for a low-probability event right is one of the very difficult things in this sort of analysis.

    The analysis also assumes the non-transmission of one game was accidental. It was apparently deliberate on the part of the organisers. On that basis one could factor in the possibility that this adversely affected Sandu during that game giving her an increased chance to lose it (that again increases the probability of innocence.)

    I think that the same thing happens whatever changes are made to the assumptions though: the probability of cheating may come out as well over 50% but it won't end up as 99%. This vindicates the FIDE approach that no matter how suspicious the results or games, you don't convict someone on game evidence alone - there has to be further evidence such as suspicious behaviour (even if that evidence is not by itself conclusive.)

  2. #62
    Monster of the deep Kevin Bonham's Avatar
    Join Date
    Jan 2004
    Posts
    37,598
    Quote Originally Posted by Vlad View Post
    It is not interesting, it is plain wrong.

    I have already tried to explain why it is wrong on the crestbook forum. If Dr Ricy has time, he might give his 2 cents too.
    Looking at your comments on the centralsquares page, you write:

    "4) Finally, the derivation of 0.3 probability makes no sense to me. Your earlier calculations were produced under different hypotheses. One can’t just combine all these numbers and plug them all into one formula."

    Actually this is exactly what Bayesian analysis does. It computes the probability of an event occurring (Sandu cheating) given Sandu's results based on:

    * the assumed a priori probability that any given player would cheat in this event
    * the probability of Sandu obtaining her results assuming that she was cheating
    * the probability of Sandu obtaining her results assuming that she wasn't cheating

    It's a method that can work extremely well for assessing probabilities of a wide range of things - especially things like false positives in medical tests - but it is very sensitive to the assumptions made and very prone to "garbage-in-garbage-out."

    In simple terms this is also about false positives. If a given crime is rare and someone can look guilty of it without being so, then a very high level of evidence is required to show they are actually guilty.

    From an earlier post:

    Strong players 2400+ (just cut this pesky Michael Baron) would put a very high weight on that Sandu was getting some help, while under 2000 prefer to believe in a fair tale.
    It seems a certain retired very strong Australian player believes in fairy tales too.

  3. #63
    CC International Master
    Join Date
    Jul 2005
    Posts
    2,154
    No, that formula is applied incorrectly.

    А - Sandu is a cheater, nА - Sandu is not a cheater.
    B - sequence (4 wins, a loss, a win) happened, nB - did not happen.

    Now we write the Bayesian formula

    p(nА/B)=p(B/nA)*p(nA)/p(B).

    She calculated that p(B/nA)=1/69,000 (I am happy to use this number). In the worst case scenario when nobody cheats p(nA)=1. Assuming p(nA)<1 will make my estimate better.

    The probability of sequence B occuring is 1/3 in power 6; that is, p(B)=(1/3)^6=1/729. I am assuming that a win, a draw and a loss are equally likely. If you assume that draw is less likely than either a win or a loss it will make my estimate better. Combine all the values to derive:

    p(nA/B)=0,01. So the correct result is 1% not 30% she claims.

    She made a few assumptions that lead to unrealistic result.
    Last edited by Vlad; 16-06-2015 at 12:49 PM.

  4. #64
    Reader in Slood Dynamics Rincewind's Avatar
    Join Date
    Jan 2004
    Location
    The multiverse
    Posts
    21,569
    Regarding Kevin's points above I don;t think they are major objections for the following reasons

    1. The blogger did analyse the effect of higher draw percentages overall. There appeared to be a roughly linear dependence on the posterior probability so unlikely to be make a decisive difference.

    2. I think this is tacitly built into the hypothesis. As mentioned in the article Sandu's rating was stable over a number of years in the range 2300-2400 and it is assumed that this is her base level of ability. However I understand she has been less active over the last couple of years and so we have less evidence if that is the correct base level (it could be higher or lower) but as that is the best we have then we use that.

    3. Some more parameter analysis could determine the sensitivity of the posterior based on the expected result for cheating. I would probably assume a certain base-rating and cheat-rating or perhaps cheating-rating-boost. Rather than arbitrarily just pull probabilities out of the air. The reason being that we have better intuition about what a rating means.


    The benefit of the Baysian approach is that the reasoning is sound and it focuses the discussion on the hypotheses being tested and the prior probabilities being posited. This this case the base rate of cheating seems to be the main driver. How likely is cheating is a difficult question to answer. Perhaps it is quite prevalent for low boosts but quite rare for large boosts. Whether rare means 1 in 1,000 or 1 in 10,000 is a very open question.

    However even in large boost cheating is only 1 in 1,000 rare then there still is insufficient evidence to convict Sandu on the evidence of results alone. People can still have a good run of form and I don't believe there is any conviction possible without some physical evidence. There is cause for suspicion but that suspicion should only entail an investigation where the presumption of innocence is not infringed.
    So einfach wie möglich, aber nicht einfacher - Albert Einstein

  5. #65
    Reader in Slood Dynamics Rincewind's Avatar
    Join Date
    Jan 2004
    Location
    The multiverse
    Posts
    21,569
    Quote Originally Posted by Vlad View Post
    No, that formula is applied incorrectly.

    А - Sandu is a cheater, nА - Sandu is not a cheater.
    B - sequence (4 wins, a loss, a win) happened, nB - did not happen.

    Now we write the Bayesian formula

    p(nА/B)=p(B/nA)*p(nA)/p(B).

    She calculated that p(B/nA)=1/69,000 (I am happy to use this number). In the worst case scenario when nobody cheats p(nA)=1. Assuming p(nA)<1 will make my estimate better.

    The probability of sequence B occuring is 1/3 in power 6; that is, p(B)=(1/3)^6=1/729. I am assuming that a win, a draw and a loss are equally likely. If you assume that draw is less likely than either a win or a loss it will make my estimate better. Combine all the values to derive:

    p(nA/B)=0,01. So the correct result is 1% not 30% she claims.

    She made a few assumptions that lead to unrealistic result.
    Her assumption is either Sandu cheated or she didn't cheat

    P(B/A) = 1/29000, P(B/nA) = 1/69000 and thus

    P(B) = P(B/A) + P(B/nA) = 4.89755E-05

    In your estimate of P(B) you assume results happen entirely an random and come up with P(B) = 1.37E-2 but B entails you observing the results games between players of specified ratings.
    So einfach wie möglich, aber nicht einfacher - Albert Einstein

  6. #66
    Monster of the deep Kevin Bonham's Avatar
    Join Date
    Jan 2004
    Posts
    37,598
    Quote Originally Posted by Vlad View Post
    The probability of sequence B occuring is 1/3 in power 6; that is, p(B)=(1/3)^6=1/729. I am assuming that a win, a draw and a loss are equally likely. If you assume that draw is less likely than either a win or a loss it will make my estimate better. Combine all the values to derive:
    This part is completely wrong. P(B) is the prior probability of Sandu achieving that sequence in the absence of any assumption about whether she is cheating or not. It is not the prior probability of two completely random players playing each other. You cannot assume a win is 1/3 likely since if Sandu is not cheating, which on the prior assumption is overwhelmingly likely, then a win will be much less than 1/3. How much you increase the win probability above the rating expectation depends on how strongly the possibility of cheating is weighted in the prior assumption - but to get the win probability up from say .12 if not cheating to .33 you need an a priori assumption that there is at least a 1/4 chance of cheating.

  7. #67
    CC Candidate Master
    Join Date
    May 2013
    Location
    Sydney
    Posts
    73
    Quote Originally Posted by Rincewind View Post
    Her assumption is either Sandu cheated or she didn't cheat

    P(B/A) = 1/29000, P(B/nA) = 1/69000 and thus

    P(B) = P(B/A) + P(B/nA) = 4.89755E-05
    Not exactly. 1/29000 is P(B/A)*P(A) = (0.90×0.90×0.90×0.90×0.58×0.90) * (1/10000) = 1/29000

    She calculated P(B) in this way:
    P(B) = P(B/A)*P(A) + P(B/nA)*P(nA) = (0.9^5*0.58) * (1/10000) + (0.12^5*0.58) * (9999/10000) = 0.000034248 + 0.000014431 = 0.000048679

    And, finally:
    P(nA/B) = P(B/nA) * P(nA) / P(B) = (0.12^5*0.58) * (9999/10000) / 0.000048679 = 0.296448423

    Do you agree?

  8. #68
    CC Candidate Master
    Join Date
    May 2013
    Location
    Sydney
    Posts
    73
    Quote Originally Posted by Vlad View Post
    I have already tried to explain why it is wrong on the crestbook forum. If Dr Ricy has time, he might give his 2 cents too.
    On Crestbook, nobody understands even a little bit of what we are talking about. So, I decided to come here.
    Last edited by Bulldozer; 16-06-2015 at 01:28 PM.

  9. #69
    Reader in Slood Dynamics Rincewind's Avatar
    Join Date
    Jan 2004
    Location
    The multiverse
    Posts
    21,569
    Quote Originally Posted by Bulldozer View Post
    Do you agree?
    I certainly agree with your numbers not with your notation but your notation is probably better.
    So einfach wie möglich, aber nicht einfacher - Albert Einstein

  10. #70
    CC International Master
    Join Date
    Jul 2005
    Posts
    2,154
    Quote Originally Posted by Rincewind View Post
    Her assumption is either Sandu cheated or she didn't cheat

    P(B/A) = 1/29000, P(B/nA) = 1/69000 and thus

    P(B) = P(B/A) + P(B/nA) = 4.89755E-05

    In your estimate of P(B) you assume results happen entirely an random and come up with P(B) = 1.37E-2 but B entails you observing the results games between players of specified ratings.
    Then I think there is a problem with the approach. You are not allowing me to take an average p(B), but taking an average p(nA) yourselve. That is sort of unfair. I can say that given that the player is old, performed poorly in the previous tournament, etc, the probability p(nA) should not be just an average across all players but rather much higher. More like p(nA)=1/4, as Kevin suggested.

  11. #71
    Reader in Slood Dynamics Rincewind's Avatar
    Join Date
    Jan 2004
    Location
    The multiverse
    Posts
    21,569
    Quote Originally Posted by Vlad View Post
    Then I think there is a problem with the approach. You are not allowing me to take an average p(B), but taking an average p(nA) yourselve. That is sort of unfair. I can say that given that the player is old, performed poorly in the previous tournament, etc, the probability p(nA) should not be just an average across all players but rather much higher. More like p(nA)=1/4, as Kevin suggested.
    I think the problem is that you are not calculating the conditional probabilities. Either Sandu cheated or she didn't and so P(B) = P(B|A)*P(A) + P(B|nA)*P(nA) (using the improved notation of Bulldozer). If you agree with this then how do you account for the additional probability mass in your value of P(B)?

    PS I don't really understand where Kevin gets the 1/4 number from. The prior P(A) and P(nA) are the prevelance of cheating in the general chess population. I cannot accept that it is anything like 1 in 4. If you need that sort of prior for a conviction then your case is doomed.
    So einfach wie möglich, aber nicht einfacher - Albert Einstein

  12. #72
    Reader in Slood Dynamics Rincewind's Avatar
    Join Date
    Jan 2004
    Location
    The multiverse
    Posts
    21,569
    PPS Using P(B|nA) = 1/69000 and P(B|A) = 1/3 then to get P(B) = P(B|A)*P(A) + P(B|nA)*P(nA) = (1/3)^6 what you need is P(A) = 1/250.

    Note the original blog post also says to get P(A|B) > 0.99 then you need P(A) of 1/240 which explains why your value of P(B) implies a posterior of around 99%.
    So einfach wie möglich, aber nicht einfacher - Albert Einstein

  13. #73
    CC International Master
    Join Date
    Jul 2005
    Posts
    2,154
    I do not think you have followed my argument. I said you should not average out across all chessplayers to derive the probability of cheating p(A). You should consider only old, rusty, etc chess players who are capable to achieve B. The probability p(A) in such a case is certainly much higher because you have a selection bias. You are selecting cheaters more often than on average.

    The second problem with the calculation, that I mentioned on crestbook, but have not yet here, is the assumption that with computer insistence Sandu can achieve B only in 1/3 cases. Really? In my opinion, it is more like 90-100%.

    Give me an access to a computer and I will kick Magnus Carlsen's butt. He may get occasional draw, but the score out of 10 games will be at least something like 9:1. It clearly indicates to me that she is not a chessplayer and does not understand that humans and computers are complementary. Well, at least at the master level.

  14. #74
    Reader in Slood Dynamics Rincewind's Avatar
    Join Date
    Jan 2004
    Location
    The multiverse
    Posts
    21,569
    Quote Originally Posted by Vlad View Post
    I do not think you have followed my argument. I said you should not average out across all chessplayers to derive the probability of cheating p(A). You should consider only old, rusty, etc chess players who are capable to achieve B. The probability p(A) in such a case is certainly much higher because you have a selection bias. You are selecting cheaters more often than on average.
    I'm willing to accept rusty players as a reference class if you have some objective basis for the premise that they are more likely to cheat. Thinking of well publicised cases of cheaters and those suspected of cheating the main to in recent times are Feller and Ivanov. Neither were particularly old or rusty.

    Quote Originally Posted by Vlad View Post
    The second problem with the calculation, that I mentioned on crestbook, but have not yet here, is the assumption that with computer insistence Sandu can achieve B only in 1/3 cases. Really? In my opinion, it is more like 90-100%.
    The argument is premised on those cheating at key positions and not all the time. That is players who are cheating but careful to avoid cheating allegation from game analysis and so just spot-cheating. I think on this assumption the probabilities are reasonable. But you can argue about different probabilities if you like. I think you will find it is difficult to get P(A|B) up to .99 based on that parameter.

    Quote Originally Posted by Vlad View Post
    Give me an access to a computer and I will kick Magnus Carlsen's butt. He may get occasional draw, but the score out of 10 games will be at least something like 9:1. It clearly indicates to me that she is not a chessplayer and does not understand that humans and computers are complementary. Well, at least at the master level.
    I think that objection is dealt with the premise of spot cheating to avoid detection via game analysis. What would be your score over Magnus if you had to take care not to be detected as using a computer by (a) suspicious behaviour and (b) game analysis?
    So einfach wie möglich, aber nicht einfacher - Albert Einstein

  15. #75
    CC International Master
    Join Date
    Jul 2005
    Posts
    2,154
    Quote Originally Posted by Rincewind View Post
    I'm willing to accept rusty players as a reference class if you have some objective basis for the premise that they are more likely to cheat. Thinking of well publicised cases of cheaters and those suspected of cheating the main to in recent times are Feller and Ivanov. Neither were particularly old or rusty.
    Again you are missing my point. I suggested to consider the population that is both rusty and performing much better than their ratings. Among this population there will be a significant percentage of cheaters.




    Quote Originally Posted by Rincewind View Post
    I think that objection is dealt with the premise of spot cheating to avoid detection via game analysis. What would be your score over Magnus if you had to take care not to be detected as using a computer by (a) suspicious behaviour and (b) game analysis?
    Yes, that is exactly what I mean. Say I have a device on me that sends me a signal each move but I only use it in 10-20% cases. In 80-90 % cases I am strong enough to make a judgement myself. If I train for a couple months, I doubt anybody in Australia will survive. First 15 moves I can just play remembering theory. In the next 10-15 I may have to use 5-8 moves. Once we come to an endgame and I have some advantage, I do not require any assistance. So 5-8 moves in the entire 50 + game.

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. FIDE general discussion thread
    By Kevin Bonham in forum General Chess Chat
    Replies: 47
    Last Post: 16-08-2019, 12:29 AM
  2. Do we really have a lot of computer cheating in Australia?
    By MichaelBaron in forum Australian Chess
    Replies: 17
    Last Post: 14-08-2019, 08:56 PM
  3. 46th Parliament General Discussion Thread
    By Kevin Bonham in forum Politics
    Replies: 18
    Last Post: 07-08-2019, 06:51 PM
  4. 44th Parliament General Discussion Thread
    By Kevin Bonham in forum Politics
    Replies: 693
    Last Post: 08-05-2016, 06:50 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •