PDA

View Full Version : Rybka disqualified and banned from World Computer Chess Championships



Max Illingworth
29-06-2011, 11:28 AM
ChessVibes report (http://www.chessvibes.com/reports/rybka-disqualified-and-banned-from-world-computer-chess-championships/)

I wonder if the programmers of the competing chess engines set this entire incident up to get rid of Rybka so that they can win the title, trophy and prize money. The impression I get from the ICGA's announcement is that they have banned Rybka because they could not prove Rajilch innocent, rather than because they had proof that he was guilty. The statement regularly refers to 'evidence' without providing it.

Or maybe I'm just being too cynical. :P

Oepty
29-06-2011, 11:57 AM
There are links to downloads below the article linked to above, which are described as evidence. It is however quite technical and far beyond my level of understanding.
Scott

Rincewind
29-06-2011, 11:57 AM
ChessVibes report (http://www.chessvibes.com/reports/rybka-disqualified-and-banned-from-world-computer-chess-championships/)

...
The statement regularly refers to 'evidence' without providing it.


At the bottom of that page you can download some of the evidence. For example,

http://www.chessvibes.com/plaatjes/rybkaevidence/RYBKA_FRUIT_Mar11.pdf

Gives the details of the comparison of Rybka 1.0 Beta and Fruit 2.1. There is a fairly detailed description of the disassembly of both engines and summary of the similarities / differences. From this report it looks very much like Rybka 1.0 Beta started with Fruit 2.1 code as a significant starting point.

Max Illingworth
29-06-2011, 12:58 PM
At the bottom of that page you can download some of the evidence. For example,

http://www.chessvibes.com/plaatjes/rybkaevidence/RYBKA_FRUIT_Mar11.pdf

Gives the details of the comparison of Rybka 1.0 Beta and Fruit 2.1. There is a fairly detailed description of the disassembly of both engines and summary of the similarities / differences. From this report it looks very much like Rybka 1.0 Beta started with Fruit 2.1 code as a significant starting point.

Thanks for this link Rincewind, I overlooked the downloads section on the Chessvibes report.

Having read the article, I think it can be safely said that Rybka 1.0 was based on Fruit 2.1. But if Rybka 1.0 did not participate in the World Computer Chess Championships (I think Rybka 2.3.a participated in the 2007 WCCC, please correct me if I am wrong), then I do not think Rybka is 'plagiarised'. Most chess computers have some similarities to other computers anyhow, and similar fallacies, such as the horizon effect and recognising fortresses.

Oepty
29-06-2011, 01:43 PM
Thanks for this link Rincewind, I overlooked the downloads section on the Chessvibes report.

Having read the article, I think it can be safely said that Rybka 1.0 was based on Fruit 2.1. But if Rybka 1.0 did not participate in the World Computer Chess Championships (I think Rybka 2.3.a participated in the 2007 WCCC, please correct me if I am wrong), then I do not think Rybka is 'plagiarised'. Most chess computers have some similarities to other computers anyhow, and similar fallacies, such as the horizon effect and recognising fortresses.

If Rybka 1 was based on Fruit and Rybka 2 was based on Rybka 1 and Rybka 3 was based on Rybka 2 and Rybka 4 was based on Rybka 3 then isn't it fair to say Rybka 4 is based, at least in part, on Fruit?
Scott

Max Illingworth
29-06-2011, 01:46 PM
If Rybka 1 was based on Fruit and Rybka 2 was based on Rybka 1 and Rybka 3 was based on Rybka 2 and Rybka 4 was based on Rybka 3 then isn't it fair to say Rybka 4 is based, at least in part, on Fruit?
Scott

I think it's fair to say that Rybka 4 is partly based on Fruit but I don't think this is sufficient evidence to claim that the engine is plagiarised.

Jesper Norgaard
30-06-2011, 01:26 PM
I think it's fair to say that Rybka 4 is partly based on Fruit but I don't think this is sufficient evidence to claim that the engine is plagiarised.

There is no doubt to me that Rybka 4 contains several independent parts that are genuine Rybka-team development. There is also no doubt to me that some basic constructs in Rybka 1 are very similar to Fruit (in some areas) and Crafty (in other areas), and that these are to a large part unaltered blocks of Rybka 4. So from that comparison Rybka/Rajlich looks "guilty".

However, being a programmer and evaluating the "proof" I don't feel convinced by the arguments. It seems that the general idea is that if one program use specific internal data structures (for instance to evaluate King Safety or Rook mobility), then no other program can use something similar (even with values initialized slightly different) because "it is plagiarism". They are almost saying if program x uses a bitboard of the possible pieces present in a position, another one can't use the same idea.

Also I do have a problem with the process of evaluating guilt. The French Federation clearly felt their 3 "fellers" were guilty and went ahead with public statements about this. But in the end they handled the case in a neutral court with each party being able to come forward with proof and arguments.

Here the panel of evaluation consists of highly biased people that stand to gain from banning Rybka so that their software can get a boost. It would have been much better to fight this in some court, presenting evidence to a neutral judge and jury.

Mark Crowther has some wise words of caution: "Should new programs have to reinvent the wheel or be able to build on well understood elements of the past? If so, what are these elements? I'm not clear on this. Also in an era of free engines and their heavy use by players there may be commercial implications for the future."

It all comes back to which parts of programs can be patented and copyrighted, and which parts cannot. It is still an area that is highly difficult to regulate essentially because you don't want every programmer to have to look into all other code bases to check if he is doing something that somebody did before and could have patented or copyrighted.

Oepty
30-06-2011, 03:53 PM
I have read the threads regarding the decision on Talkchess.com and I am reading the discussion on rybkaforum.net
I cannot see any reason to doubt the decision of ICGA that there was wrongdoing. As far as the make up of the panel, Robert Hyatt author of Crafty was on the panel that gathered evidence. He describes his role/the panels role as being the prosecuter, not the judge or jury. The ICGA was the judge. The defendant chose not to enter a defence and given the evidence presented it left the ICGA no choice but to find guilt.
Scott

Jesper Norgaard
30-06-2011, 04:54 PM
... The ICGA was the judge ...
That was the problem.

Gadaffi puts in 15 of his government ministers to prosecute the rebels for treason to Libya. Then he steps back and say "See? I'm not affecting the trial at all!". Do you think the rebels will get a fair trial?

Oepty
30-06-2011, 10:14 PM
That was the problem.

Gadaffi puts in 15 of his government ministers to prosecute the rebels for treason to Libya. Then he steps back and say "See? I'm not affecting the trial at all!". Do you think the rebels will get a fair trial?

Perhaps I don't know enough about it so I will shut up.
Scott

Adamski
30-06-2011, 10:54 PM
I like your new sig, Scott. Interesting thread too, Viewed.

tanc
01-07-2011, 03:16 PM
That was the problem.

Gadaffi puts in 15 of his government ministers to prosecute the rebels for treason to Libya. Then he steps back and say "See? I'm not affecting the trial at all!". Do you think the rebels will get a fair trial?

No offence but I trust Ken Thompson's words over yours.

Rincewind
01-07-2011, 04:24 PM
No offence but I trust Ken Thompson's words over yours.

Ken Thompson is a highly respected figure. (I saw a talk he gave on computer chess c.1989). Also Fabien Letouzey comments are difficult to taint with self-interest since he is (as far as I know) still not actively working in computer chess at the moment with no immediate plans to resume.

Comparing these guys to something happening in a rogue state like Libya is uncalled for.

Jesper Norgaard
01-07-2011, 07:52 PM
No offence but I trust Ken Thompson's words over yours.

There might be a misunderstanding here. I never said I thought Rajlich was innocent. I objected to the manner of settling the dispute in what is in my opinion an example of a "kangaroo court" (an Americanism from Texas meaning a sham legal proceeding or court. The outcome of a trial by kangaroo court is essentially determined in advance, usually for the purpose of ensuring conviction, either by going through the motions of manipulated procedure or by allowing no defense at all). I mention this not to discredit the members of the panel or the ICGA, but to say that once they started this process, the result was given in advance - they started moving this in the first place *because* they thought he was guilty. In a normal court if you don't prepare your defense a lawyer is provided for you - not even that did the ICGA provide. The 5 judges were of course highly biased - they started the process in the first place!

In this case it is clear that Rajlich had every possibility to defend himself, he chose not to, and I can only guess at possible reasons why, but one could be that he didn't recognize the authority of the court set up - or maybe he felt guilty, I don't know. In any case I'm not objecting to the guilty verdict, but to the process itself.

The 34-person panel was supposed to be the prosecutors, so why did they vote if they thought he was innocent? The ICGA was supposed to be the judge, so why didn't it show who were those 5 members exactly (I could only count 4 board members). If you as a judge make a ruling, that ruling usually has argumentation and specifies what has been proved and what has not according to either the 1 judge or 4 jurors opinion. But in the official statement I only see going into the penalty in great detail, nothing about process in the ruling.

In a copyright infringement case what can be evaluated is not if the same ideas were used, but if the expression of those ideas were the same. But I don't see any evaluation to which standards should the proof gathering be held. In fact, a lot of the crucial arguments revolves around the evalution algorithm. For almost every single item there is something slightly different from Fruit to Rybka, but still the argument is that they are identical. One example is piece mobility by counting how many squares the piece can go to (either capturing or moving). However, what they prove is that the idea is similar, but once the idea is settled, I don't see how the expression of it could be different, after all these are fast processes gathering information quickly for each position, it simply isn't possible to express that in so many ways. Or did they think copying ideas were "software plagiarism" as they state, avoiding the direct "copyright infringement"?

There is a comparison finally of similarity of Fruit with other engines (30-35%) and with Rybka (64%) with the peculiar conclusion that Rybka is "almost identical". This is in fact the only consideration of "due process" where also other comparisons than direct Fruit-Rybka or Crafty-Rybka is used, and the conclusion is "almost identical"? Then they convert that to a chance in 10,000,000 that it was a coincidence? Huh?? I really had to pinch my arm to believe that, but maybe I misunderstood something.


Ken Thompson is a highly respected figure. (I saw a talk he gave on computer chess c.1989). Also Fabien Letouzey comments are difficult to taint with self-interest since he is (as far as I know) still not actively working in computer chess at the moment with no immediate plans to resume.

Comparing these guys to something happening in a rogue state like Libya is uncalled for.

Perhaps my choice of example was unlucky. I did not want to compare the panel with a rogue state. But I did want to compare the process to any other process where rights of the defendant has been diminished by one-sidedness. In fact all ICGA cases has been convictions as far as I know. Doesn't that look at least - peculiar? Of course those starting the cases were always - ICGA themselves too, while still being judge and jury and prosecutors!

By the way I would not blame Ken Thompson or Letouzy for self-tainted interest - they were on the prosecutors team and had every right to be biased. It was the 5-member judge team headed by David Levy which showed only one single interest : a guilty verdict.

So do I think Rajlich is guilty to some extent? Probably. Did he get a fair trial? Absolutely not.

Rincewind
01-07-2011, 09:15 PM
Did he get a fair trial? Absolutely not.

We are talking about the organisers of a competetion who investigated the circumstances of one of the entrants who they have satisfied themselves did breach at least one of the rules. I don't have the conditions of entry on hand but I would assume they include something along the lines of players who breach the rules will be disqualified and the decision of the judges will be final.

This isn't a criminal trial with no right to a fair trial or presumption of innocense. It would be up to Rajlich to take the ICGA to court for various things (perhaps defamation for example) and/or the ICGA to take Rajlich to court to actual return the prizemoney as requested.

Jesper Norgaard
02-07-2011, 02:17 AM
We are talking about the organisers of a competetion who investigated the circumstances of one of the entrants who they have satisfied themselves did breach at least one of the rules. I don't have the conditions of entry on hand but I would assume they include something along the lines of players who breach the rules will be disqualified and the decision of the judges will be final.

This isn't a criminal trial with no right to a fair trial or presumption of innocense. It would be up to Rajlich to take the ICGA to court for various things (perhaps defamation for example) and/or the ICGA to take Rajlich to court to actual return the prizemoney as requested.

I agree with you perfectly, and there might be some good in running this process before a real court case since the arguments get clearly stated, at least on ICGA's side. And the process is cheaper for all parties than a court case.

God knows what Rajlich's point of view is at the moment. One way or the other, he must have sought legal advice by now I assume, so eventually he will have to act one way or the other. It is also curious, that Chessbase which is selling Rybka from their web site, have been absolutely silent about the case, even though they had Friedel on the panel (as a silent and observing member, though).

The panel claims that they found code from Crafty in Rybka's code that were obsolete (dead code) and other that made no sense because it was mending a flaw in the tablebases that were handled previously by Crafty, which were in a different format presumably in Rybka. If the findings are true, they nailed him not on the code he copied that gave Rybka a boost, but on the code that he copied that made no sense. :owned:

I have not been able to find the documentation for the dead code, but (if true) these seem to be the most damning findings since in the evaluation code which was supposed to give Rybka a 800 points boost, you could well claim that ideas were copied, not implementations or expressions of the idea, which is valid in some circumstances under the umbrella of copyright infringement. I am no software IP expert, but the whole area is kind of grey to me. There are fora for chess computer developers where all kinds of discussions go on, and I think no single programmer has created a chess program that do not use some of the original constructs like the alpha-beta search algorithm so today no chess program that has a high strength exists which hasn't code that uses ideas that have drifted to the public domain without being explicitly published as such.
In fact most chess programs copy ideas heavily from each other, in most cases without copying source code directly I assume. But some of them are open source which makes it very easy to copy code directly although forbidden unless documented.

Talon
02-07-2011, 07:31 PM
To me, Rybka always seemed a little fishy.

Jesper Norgaard
03-07-2011, 10:46 AM
To me, Rybka always seemed a little fishy.

Hilarious little joke, playing on that Rybka in Russian means "little fish" :lol:

My main objection against the process that ICGA has undertaken is not that they started out with a bias, but that their process did not take into account the whole purpose of what they were trying to prove - that copying had taken place. It is strange that when every single engine running in the championship would come with a copyright attached in some way (even GPL and open source, or whatever), then they omitted to evaluate that at all, instead using their own definition that it should be "an original work" which is presumably something that has never before been evaluated in a real court because no legislation is made to adhere to it, since it is an ICGA-only definition. Perhaps that happened because ICGA made this definition long before copyright was born, I don't know, but the problem is that they therefore did not adhere to any industry standards of evaluating copyright infringement, but invented their own methods along the way. I am sure they tried their best to make this approach work, but in my mind the result still looks amateurish.

If they had set out to evaluate copyright infringement, they could have used the Abstraction-Filtration-Comparison test
see http://en.wikipedia.org/wiki/Abstraction-Filtration-Comparison_test
which is a standard that has been used in court cases in US, UK and France, and which describes how to filter elements that may be copyrightable, and which elements might not. One of the first activities is to lay out a clear analysis from the plaintiff which parts of his code he believes have been infringed, and isolate those in a clear manner, before even attempting to compare with the defendant's code (or disassembled code in this case). I have not seen a comprehensive analysis of what parts of the Crafty and Fruit code has supposedly been copied.

Instead there is an analysis to compare Crafty, Fruit, Rybka 1.0 beta and a later Rybka version plus some other (inferior) open source programs on certain elements of the Evaluation functionality (the one that puts a number to how is the relative strength between the white and black position, e.g. -2.0 means in broad terms that white is relatively 2 pawns behind). In this case a number of sub elements of the evaluation are lined up for each program, for instance isolated pawn, Rook mobility, King safety. In most of these cases there are slight differences in the way to calculate each element between Fruit and Rybka, and the weight set to each element. It is clear that in comparing which elements are included and which not in the Evaluation, it seems that the (unexpressed) argument is that if Rybka used the same elements as Fruit, no more no less, it must have been copied code or rather non-original work. This is opposite to copyright as I understand it where ideas can't be copyrighted, but expression of ideas can.

I found some interesting comments from a judge that applied AFC in a court case, from this link
http://digital-law-online.info/lpdi1.0/treatise24.html

the judge comments:
"For example, as Altai indicates, even with its overly structured mode of analysis, it will be very important for me to know whether the essential function being performed by the copyrighted program is a function that can be accomplished with a wide variety of source codes, which will strengthen the case for protection, or, on the other hand, is a function capable of execution with very few variations in source code, or variations of such triviality as to be disregarded, in which event protection will be unlikely. For me, this mode of analysis is essentially what in other contexts we call the merger doctrine – the expression is said to have merged with the idea because the idea can be expressed in such limited ways that protection of the plaintiff’s expression unduly risks protecting the idea itself."

When you determine that Rook mobility is important, then there are only so many ways to calculate it, and then weight it in the overall evaluation. In this case the panel apparently didn't care that the algorithm was slightly different for most items (comparing Fruit and Rybka 1.0 beta), and the weight was slightly different (they have the audacity to call it obfuscate whenever the weight used was different), but just concluded that the Evaluation was "identical".

Of course to have a meaningful comparison we would have to take into account the other extreme of copyright infringement. Suppose someone takes the full Crafty program and just changes one single thing, tweak the weights of these positional/tactical elements. All other statements are the same. By sheer luck they might come up with a better evaluation function even though their only input was by no means an original work. So how can we distinguish the two? In my opinion a "real" judge would have set up guidelines for the jurors how to distinguish the two cases and I have seen nothing from David Levy to suggest he did that. In fact it seems that the judges thought that the panel report (the prosecutors report) could stand for itself with nothing added.

Garvinator
03-07-2011, 06:07 PM
In this case it is clear that Rajlich had every possibility to defend himself, he chose not to, and I can only guess at possible reasons why, but one could be that he didn't recognize the authority of the court set up - or maybe he felt guilty, I don't know. In any case I'm not objecting to the guilty verdict, but to the process itself.
I have a strong feeling that the reason Vasik Rajlich chose not to defend himself is that to be able to defend himself properly would mean that he would have to hand over his trade secrets about the Rybka engine and 'source codes'. That is valuable information and he would have legitimate concerns about who would end up in possession of that information. Seeing that concerns have been raised that some of his business competitors are involved in this judging business, Vasik Rajlich would not want to hand over this information.

So he has probably decided that it is a smarter business decision to wear a guilty verdict, than to hand over the trade secrets.

Btw, this is an Australian chess forum, so we are well aware of the term= kangaroo court :lol:

sleepless
13-07-2011, 07:13 PM
Rybka forum link
http://rybkaforum.net/cgi-bin/rybkaforum/topic_show.pl?tid=22402


Youtube interview with Rajlich
http://www.youtube.com/watch?v=cQshTNJ4pSM

Garrett
05-01-2012, 07:39 PM
Looks like he may be innocent.

http://www.chessbase.com/newsdetail.asp?newsid=7813

Kevin Bonham
05-01-2012, 08:04 PM
Looks like he may be innocent.

http://www.chessbase.com/newsdetail.asp?newsid=7813

That seems a very well-written and argued piece. I will be interested to see if there is any kind of reply to it.

Agent Smith
05-01-2012, 08:39 PM
Hmmm... The article doesn't sway me much. Just speaking broadly - the community decision to investigate Ryba was a petition signed by virtually every engine coder going around, after extensive analysis. This article's author's credentials are "Søren Riis is a Computer Scientist at Queen Mary University of London. He has a PhD in Maths from University of Oxford. He used to play competitive chess (Elo 2300)", which is kind-of laughable really. And of course Chessbase used to sell Rybka.

In all likelihood Rajlich inappropriately used GPL-ed code. But it's nothing to get worked up about... Robert Houdini's fairly blatant appropriation of Ippolit/Robbolito code (which itself is probably a reversed engineered Rybka, but i'm not sure) makes trying to unravel the whole thing kind-of pointless, especially since there are several free engines (Critter-1.4, Stockfish-2.2, Ivanhoe, Strelka 5) that are virtually as strong anyway.

Garvinator
05-01-2012, 08:43 PM
While I do not believe that Rajilch has not done anything wrong in this saga, the Chessbase information should be viewed with the knowledge that Chessbase and Rajilch have a commercial relationship.

Probably the hardest part of all this to work out is how could Rajilch genuinely defend himself when some of those who were standing in judgement against him are those who would benefit from knowing how Rybka works in detail.

For Rajilch to defend himself, he would have had to reveal his secrets and how the engine has been developed, which would be very useful information for his business competitors.

Kevin Bonham
05-01-2012, 08:48 PM
Hmmm... The article doesn't sway me much. Just speaking broadly - the community decision to investigate Ryba was a petition signed by virtually every engine coder going around, after extensive analysis.

How neutral are the other "engine coders"? How many of them were competitive rivals of the program?


While I do not believe that Rajilch has not done anything wrong in this saga, the Chessbase information should be viewed with the knowledge that Chessbase and Rajilch have a commercial relationship.

That is well worth mentioning. Thanks.

Agent Smith
05-01-2012, 10:19 PM
How neutral are the other "engine coders"? How many of them were competitive rivals of the program?
Well - of course they are not "neutral". But these people spend their whole lives writing engines and i believe (after spending a few months reading through all the diatribes) that they have integrity. In my experience, people interested in making money and keeping quiet, have a lot less integrity. Two or three things Rajilch have said seem to condemn him. Briefly - he said unreservedly that Strelka was a Rybka clone, and it was later seen by examining the source that Strelka was a derivative. Rajilch claims to have lost the source to early Rybka... which is nothing less than flabbergasting to me as a programmer. Other things too.

I don't deny there's a chance all these people are wrong. Only examing the early source of Rybka will show for sure, and that doesn't seem to be going to happen.

It's funny Chessbase have brought this up, just when they are selling another engine with marginal copyright foundations. The origins of Rybka weren't obvious for a long time, partly because Fabien (Fruit's Author) dropped out of the scene. But Houdini is almost definitely a derivative of Ippolit, the origin and license of which seems indeterminable. :hmm:

Agent Smith
10-01-2012, 09:33 PM
Ok - Houdini is sold with Aquarium, not Chessbase :doh:

It suprises me the fanatacism some Rybka users have for Vasik Rajlich.
I guess he was a pioneer in some ways , apart from having the strongest program around ?

Kevin Bonham
13-01-2012, 11:33 PM
A lengthy technical rebuttal of the Riis artcles has been posted here (http://chessbase.com/news/2011/watkins01.pdf)

Agent Smith
21-01-2012, 08:50 PM
Jeroen:
Maya experts have denied that 2012 is the end of the world ....
........
So you advise me to buy truck loads of doritos!? :wink:

Vasik Rajlich:
Either that or a Colt M1911, with ammo. This can be useful for restocking operations. :smile:

Jeroen:
Ehm, possessing guns is not allowed in Holland.....

Vasik Rajlich:
No problem, just get a big knife. You won't have to worry about "bringing a knife to a gunfight". :smile:

Nick:
I'm beginning to see where the "psycho" nickname came from :grin:

Vasik Rajlich:
Good.

http://rybkaforum.net/cgi-bin/rybkaforum/topic_show.pl?tid=23957 (1000+ posts)

Kevin Bonham
07-02-2012, 07:08 PM
Levy interview (part 1) :

http://www.chessbase.com/newsdetail.asp?newsid=7899

Craig_Hall
08-02-2012, 06:56 PM
It seems that Levy's position on it is that Rybka is a derivative of Fruit, but wasn't declared as such on the entry form, hence the retroactive DQs.