Monday, April 27, 2015

Desert Island books

A desert island book is a book which you - as only book - would choose bringing to a desert island. An interesting question for a chess player is which chess-book would you for sure put in the suitcase (we don't accept an i-pad fully loaded with pdf-files).

Implicitly we mean: " which book did you enjoy the most " or you can also interpret this literally. Which book would give you the most pleasure on a desert island - which probably would generate a different choice. It is very well possible that a book made a big impression but has no great value on a desert island, while maybe an average book about endgames, studies could provide many hours entertainment. For sake of easiness we assume that there is enough time to make a board and pieces. Maybe with some luck something comes ashore - another advantage of a chess-player: chess-materials float (unless you prefer weighted pieces) .

But let us not force the issue and agree to select 10 books to make our live on coconuts and crab more enjoyable.

It is well possible that I saw the concept of "a desert island book " in one of the books which surely pops up in my shortlist  The Even More Complete Chess Addict from Mike Fox and Richard James.
It is a fine book, with relatively few games but more anecdotes and original material. It resembles the work of Tim Krabbé, but it is fully independent. It includes a pinch of “Guinness Records”, but it is more than just that. I found it recently back in one of my moving boxes and when I saw it back after many years it was a small disappointment : was it only that? Immediately a nice illustration of the double position from the beginning of this article: what is a desert island book... Nevertheless it was still a nice refreshment of trivia and about anything happening at the boarders of the chess-world. There is a list of best problems, studies, games - almost a summary of what each player should know from technical perspective. A very enjoyable reading the first time but you won't read it 5 times.

A second book on my shortlist is the biography of Pal Benkö. It is a very beautiful work, hardcover, good layout, diversity with pictures and biographic material. A lot of games with diagrams so a lot to read. Benkö also composed studies and tasks so something which will last for years on the island.
A third book which I surely consider is De Koning. I can't recall anymore the number of times I reread the book. Once I copied the book page after page, later bought it in English and eventually got a nice second-hand copy in Dutch. It remains a milestone in chess-literature but should we really thank  Krabbé and  Pam with their selection - is this really the best of Donner? I don't know - were his other articles also at the same level? Maybe there were more timebound, maybe of less quality, maybe more boring (although difficult to imagine). It can't be a bad idea to make a “De Koning 2” even if just to have a more complete view of Donner the chess-author. If the book of Benkö belongs to the top 5, then this one too.

The fourth book is a book which maybe can be called the best tournament book ever: San Luis 2005. Yes I do have also Zurich 1953 (of course the one from Bronstein (and signed by the man himself - yes! ), not the one of Najdorf), but this more recent work lifts the concept of tournament book to a higher level. The games are very well analyzed, facilitated by pictures and sufficient prose which allows the first week on the island to use it as reading book. Later you can replay the games, check the very extensive analysis,...
This book guarantees long afternoons and evenings under the palm-tree without feeling the hunger or getting the reflex of running to the not-existing refrigerator for a fresh white whine at the sunset. What is the strength of this book? Primary the analyses which are at the same time thorough as easily accessible. The lines are checked by the engines. They are not endless but explained verbally. The many diagrams support the text. The introduction, pictures and considerations complete the book.

Number five is VanPerlo – maybe the funniest endgame-book which I ever had. However here I start to hesitate: although more than 1100 positions are included, maybe the content is rather weak in the prospect of years on the island. You could study 1 position each day but it is very difficult to put the book aside after just 1 position so very likely you cruise through the book.

If my house burns down and I must save books then very likely I bring the numbers 2 till 5 first into safety.

Number six will be a book of Kasparov - but which one from the series? The first, without any pretensions looking at the beautiful classical era of the first professional chess but of which the analysis aren't faultless? The book Revolution in the 70’s – which is a nice mix between theory and history? The book of his first long match with Karpov? Or the one about Fischer? I prefer Rev in the 70’ies. You never know if I meet Friday on my island, by coincidence also able to play chess. Some opening-knowledge can be pretty handy. And that book of Kasparov is a real masterclass in openings.

Number 7 - I have to choose somewhere a good openingbook, the last one came close but is no 'real' openingbook? But which one? The twin-book from Shipov about the Hedgehog is breaking the rules - no 2 books. And the 2 books about the Sicilian Dragon from Schneider: pity, but the same. My first chess-book, the one from Unzicker (Thieme’s nieuwe schaakboek) mainly covered openings but is rather weak compared with the above mentioned masterpieces. I really like the book but I fear that I will not look at it anymore after a week on my island. Of course the problem is that I never really studied well openings - a minor interest for the Sicilian dragon, Centergambit as white, Scandinavian and Kan as backup and in fact except Benkö nothing concrete against 1.d4. Against 1.c4 I play symmetrical English because of the vague resemblance with other fianchetto- openings with black and especially because the mainline is easy to remember. But an opening-bible? I believe the opening-vademecum from Roessel is a nice little book, a good balance between simplicity and details, but it is not very crucial. So no opening-book? Ach, why not - Friday will likely play like a turtle and if necessary we start from scratch and skip openings. So no number 7.

Number 8 - I still need a tournament-book. I am a big fan of tournament-books and Caïssa editions provided quite some good materials, old tournament-books very nicely republished but with a modern flavor (engine checked analysis).
But the book of IM Forster (one of the rare authors not despised by Winter) about the chessclub of Zürich can't be ignored. A great reading, a lot of games ( imagine: players over a time-span of 150 years are presented often with their most beautiful game, and this decorated with lots of materials about tournaments in Zürich). It is one of my favorite books.

Number 9 - yet another biography, but which one -McFarland has a strong series. The book from Winter about Capablanca is a classic, but I've always been very charmed by the one from Soltis about Marshall – very readable and a good mix between games and biographic info. And the gigantic biography about Amos Burn – pity, but again 2 books in the English publication of McFarland (The German copy is 1 colossal book). The book from Soltis brings Marschall to live, something in which he didn't succeed with the book about Botvinnik. It represent a nice - although sometimes summarized - look at the live of one of the last bohemian players from before WOII.

Number 10 is a bit cheating: a bookbinding of the magazine Kaissiber from Stefan Bücker. I also have a year-bookbinding of British Chessmagazine, but the timebounded character of a normal magazine is less interesting than the very readable Kaissiber, which besides also covers some very rare openings (do you know the Vulture or the Habichdich ?).
Above collection represents well my interests: I like chess but I still like more reading about chess, the history, how the famous players lived but also the rather peculiar sides of chess. 

What did I miss? Maybe a chess-novel - there exists already quite some readable work of acceptable level. From Stefan Zweig to Nabokov, just to mention the 2 most known ones. Or books about chess - I think about the oeuvre of Tim Krabbé or HansRee. But as I dropped number 7, I still have 1 spot left in the (already heavy) suitcase. Maybe I keep this for a favorite book of the readers of this blog and I discover a beauty of which I never heard about the existence - a special treat for the first evening at the shores of my island.

HK5000

Wednesday, April 22, 2015

Automatic detection of cheating

Automatic detection of cheating: mature?

One of the advantages to work at a (large) consulting firm is to have access to scientific papers and other materials which are e.g. stored in sciencedirect.com. It is a database containing a lot of scientific knowledge which of course invites an ex-player to look for chess related stuff. This way I recently discovered the paper:  ‘On the limits of engine analysis for cheating detection in chess’ by David J. Barnes and Julio Hernandez-Castro, both from the School of Computing, University of Kent.

It is a relative new paper (October 2014), published in the journal Computers & Security 48 (2015), 58-73. Both writers are amateur-chessplayers but their background is rather number crunching, which they applied on chess.

Let us start with the conclusions. Unfortunately the paper tells us that today there exists no water-proof method to find out somebody has used a computer during a game. As strong handhelds with engines only exist for about 5 years, we can use an interesting data-set for this problem. More than 90% of the games are guaranteed engine-free (from Morphy till Kramnik). That is a solid base to check the algorithm - it is impossible that games before 1990 were played with the help of engines (except maybe some correspondence games but the level was still very low at that time).

The paper uses as starting-point that the problem mainly pops up on internet-sties - where games 'live' can be checked, it will be easier to detect cheaters.

In over the board (OTB) chess, if there is no strange behavior of a player to notice, then it is very hard to proof something especially if his level does not deviate much from his actual (historical) level. Besides an automatic method if not fully conclusive, has the drawback of creating 'false positives'. The paper indicates some interesting examples of perfect play, long before the appearance of computers.

This paper is not the first publication about this subject. Kenneth Regan is a pioneer on which this paper built further. Regan used a method taking into account the elo of the player before and after a tournament, his tournament-result and the move-selection of the player, compared with the move-selection of equal players in similar positions (an experienced player will immediately see objections against such approach: shape of the player, under-rated youth-player, style of play, mood, environmental circumstances or even the occasional blunder). These defects are also identified by Barnes and Hernandez.

Stunningly the authors consider the current method applied by chess-servers inadequate on the long term. The servers keep their methods secret to detect cheating but this ‘security by obscurity’ eventually fails - once it is cracked the statistical method becomes useless. And every code is after some time cracked. Apparently the protocol of  ICC contains more holes than a sieve. It is not only possible to steal credit-data but even to take over an account and use the account to cheat using engines.

Concretely the analysis happened on the database of Chessbase, extended with TWIC-databases till 2013. In total 7 million games, but for the analysis only 120.000 games were used; however the big database was used to build an opening-tree of 87 million positions. The analysis were executed by Stockfish 3.0. The games were cut into single positions which were analysed in multi-PV mode. When a played move didn't belong to the top 5 choices of an engine then this was detected. However the authors used a slightly different method compared with Regan. Regan compared each played move with the best move of the computer. On the other hand they used a parameter (coincide value) which compared the correspondence of the number of moves played by humans with the number of moves of an engine with approximately the same evaluation. A subtle but important difference. They took into account that some moves can have the same effect which shouldn't impact the cheating-detection (e.g. in an endgame a king can walk from a2 to c1 via 2 ways a2-b2-c1 or a2-b1-c1). On A UCI-based Chess Game Analyser you can find the C++code.

Nevertheless we should not underestimate cheaters - at least not smart ones. In quiet positions they will maybe choose for the second or third choice of an engine, and when the position is won, then it does not matter anymore if it is mate in 15 or 19. Therefore it is necessary to also consider the average deviation of the played move with the preferred move by the engine. A very smart cheater, playing every time sub-optimal moves, will still be able to win against somebody of <2500 points so even if regularly a slight advantage is thrown away. In the end many games are decided by blunders, and only 1 good move is sufficient to keep a permanent advantage.

The opening-tree saves for each of the moves the date it was played for the first time (something which surely is interesting). This permits to check if a strong move was already discovered earlier (and more likely known by the player) or if it is a novelty (likely more often an engine-move). However most of all the database was used to define from which point the games were original so a real good cheating-detection system could start. It makes no sense to check how good a player has studied the theory.

The analyses started with 70.000 games till 1950 and 50.000 games from 1950 till 2005. The games are analysed at a depth of 8 plies. The most suspicious games were analysed more extensive (25.000 on ply 10). This process was continued till only 250 games remained which were scrutinized till ply 22. This looks a very shallow approach, taking into account the current calculation power of our computers but if you check how much time is needed to analyse 250.000 games at ply 8 or 10 then you will surely look at it differently (e.g. Chessbase gui permits automatic analysis).

Along the logical things identified, we also notice confirmations of the human behavior: the longer the game, the more mistakes are made. Long games with invariable parameters are seldom and easily detectable. Nevertheless such games (of the era before the engines) exist and do demonstrate that 2 humans can play in exceptional cases a perfect game. On the other hand there were also many perfect short games which rather can be attributed to homework.

The article concludes with an illustration of ‘false positives’; games played close to perfection, but from before the computer-era. People thinking that players from the romantic era were only able to play spectacular attacks and exploiting the weakness of the opponent, will be surprised as the first example is Kennicott-Morphy, 1857. The game is 14,5 moves 'theory' (= moves played earlier and likely known by both players). After that Morphy plays 10 perfect new moves (perfectly matching Stockfish at depth ply 18. Morphy still pops up a few times with perfect performances (blind-game with black against Schulten, New York 1857; the seventh matchgame against Mongredien in Parijs 1859 and Morphy-Muarian in New Orleans 1866). Now we 'finally' see the respect confirmed which even world-champions have for Morphy.

If there were engines in 1889, then surely Max Weiss would need to explain his game against Burill in New York. Weiss played after 6,5 moves not less than 26 consecutive perfect moves and won the game.

The writers also discuss Mamedyarov-Kurnusov, a game which went around the world because of the sore loser Mamedyarov, but which using their method would not even be a ‘false positive’ as too short and the fact Kurnosov only played 6 perfect moves. Reversed engineered, if Kurnosov cheated then he would have committed the perfect murder. In the meanwhile we know that he was honest - the respected arbiter Geurt Gijssen did the investigations on-site.

The writers continue with their research and investigate if single thread or multi thread influences the analysis. The conclusion is clear. Only single thread analysis can be reproducible. Multi-thread generates variations (to be explained by the way the computer distributes the different tasks at the different cores).

Interesting is the start-up of a web-service which the writers want to launch to detect cheating quicker. This should increase the chances of being caught considerably, also when using a perfect camouflage (e.g. by using a wireless communication or if a player uses sub-optimal moves from an engine but still good moves) .

To conclude I still give some games which were detected as 'suspicious' if only they were not played before the computer-era. It is just like you watch a car participating at the horse-race of Ben-Hur J J. Carames-Fedorovsky, Buenos Aires 1965 and Browne-Timman, Wijk aan Zee 1980, worth looking up and replaying!


HK5000

Saturday, April 18, 2015

Harakiri

Some of the strong reactions on my previous article are definitely deserved. Showing only examples of opponents having little or not prepared, created the wrong impression that people don't have to fear preparations (except from myself).

A small minority is willing to spend time at building a dangerous preparation. Steven made earlier rightfully the remark that in norm-tournaments (exclusively consisting of the 1% highest rated players) that this minority very likely transforms to a majority. If you don't take this into account then this can quickly cost points which I experienced myself this season even on a modest second board in second division.

In the 6th round of the Belgian interclubs I was trounced by the French FM Manuel Ippolito. After only 3 hours of play we were already analyzing at the bar. Besides I used twice as much time as my opponent.

In the database I couldn't find any games of my opponent with this opening but in the postmortem it became clear that he was very well aware about a huge amount of lines. He even admitted that he used my game played in Open Leuven 2013 against Mark Davey as guideline in his preparation. I don't know if his improvement on Kasparovs 19...Bxa3 still was part of his preparation but there exists no doubt that the preparation was a deciding factor in this game.

So do I admit my mistake in my article about passwords? No as I only told one side of the story. To explain the other side we must return to the analysis made on my game of 2013 with this opening.

White achieves with correct play a clear advantage but during my game against Manuel I could not reconstruct the analysis despite long reflections. At move 17 I mix Bb6 with Nb6 and after that it quickly goes downhill in the minefield. After the game I even discovered that my analysis of 2013 were optimistic about blacks problems in this opening.

The repetition of moves with Bg5 already shows my unsureness. I know at that moment already that the game can become very tactical if I avoid the repetition. 13.Bd3 was the last chance to deviate with an interesting continuation and likely would throw the opponent out of book. However again I follow the scientific approach and more or less commit harakiri by deliberately ignoring the signals (opponent plays something very sharp for the first time, I can't remember the analysis).

After the game I had a lively discussion with some of my teammembers about my harakiri. Some were of the opinion that you are morally obliged to play the line if you studied it at home and concluded this was giving an advantage. Why would you make the analysis if you don't dare to play it? Besides often you start to remember things when you are a few moves further in the opening. Not everybody agreed with this. Losing is inherent of the game but losing in such way is nonsense. You don't learn anything and you just make a fool of yourself.

It is not the first time that I forget analysis and it won't be the last time either. In my last 100 games I can find 7 games in which I forgot or mixed up the analysis with an important impact on the further course of the game. Some of this silliness already was covered on this blog: chess-intuitionchess-intuition part 2. If I look at a decade ago then I notice that I forget today more often something. Am I growing old? Next year I will be 40 so I am not young anymore.

Although I believe age has nothing to do with it. I just have much more analysis to remember than a decade ago. Last couple of years the amount of theory exploded. A very recent article on hln confirms my supposition that failing memory isn't the culprit. My short-term memory surely has passed the peak but the long-term memory can still improve till pensionable-age. It is the long-term memory which counts here so I may still have some hope.

I complain about the amount of theory but what about 2700 players. Last Karjakin lost a game against Nakamura about which he tweeted that the worst way to lose a game is, when you know the line until a draw,but, can not remember how it goes and get a losing position immediately.

I found it disappointing to see how with bluffing today you can reach a 2800 rating. I call it bluffing as choosing a line with white of which you know it is a forced draw isn't very flattering.

If a young player like Sergey Karjakin already experiences problems to remember the variations then his colleagues won't do much better. Last year on the site of Tim Krabbe there was a funny article called Fischer Random, anyone? with a long list of opening-errors caused by forgetting the analysis made by topplayers. It is remarkable that not once Carlsen is mentioned. I don't want to claim he doesn't forget anything but he rather chooses openings which require less knowledge of theory.

This should close the topic. Passwords, harakiri... are about exact opening-knowledge. There exist good alternatives to neutralize sufficiently such opening-knowledge without using drastic measurements like Fisher Random. Most important is not to be stuck in routines which makes you too predictable.

Brabo

Monday, April 13, 2015

Password

When the first databases appeared, everybody wished to see their name popping up. Of course vanity which slowly transformed first into indifference and finally annoyance. Players started to realize that the publication of their games is a poisoned gift as eventually future opponents got valuable information for preparation.

A few years ago I discussed in my article gamepublications how increasingly often tournaments stopped putting effort in digitizing games.  Today we see this behavior intensifies as players are demanding tournament-organizers not to share their games with the public. Only participants sometimes extended with some friends still get access to the information often with the help of a password. In Flanders we see this new policy already well established see e.g Zottegem , LeuvenBrasschaatBruges experttournament, ...

Clearly a lot of players are afraid of preparations. Articles like e.g. the modern french part 2 and using databases naturally just amplify these negative thoughts. However Kara earlier already remarked on my blog that I very likely prepare my games much more intensively than the average chessplayer. It surely would be interesting to find out how other players prepare their games to get a more balanced view and maybe also relieve partly the fear.

Attempts to attract other writers like in my article free didn't bring any results and on the internet you will find barely or no information about how others prepare themselves. Therefore I decided to use a different approach by inspecting the openings of some of my recent games from the opponents point of view. Lets start with my interclubgame in Wachtebeke against the strong Dutch FM Miguoel Admiraal.

Despite I played on my usual 2nd board, my opponent clearly didn't read my blog. In 2012 I already wrote in the article the neo scheveningen that I would play Be3 instead of Be2. Now I do realize that my blog is less popular in the Netherlands but he was neither aware of my game against Vermaat which can be found in any commercial database and was a very clear warning.

A second example is from the currently on-going clubchampionship of Deurne. Guy Dugailliez knows me for many years already so he is well aware about my blog. I expected that he would try to use the information on my blog in his advantage for our mutual game.

Afterwards Guy admitted that he did search on the blog for some useful information but clearly not very extensively otherwise he should have discovered that I recommended 8.Rc1 in my articles Aljechin with g6 and archiving.

The 3rd example is again extracted from the interclubs. My opponent the Belgian FM Christophe Gregoir shows up apparently completely unprepared.

20 years I already play the Dutch stonewall which can be discovered at a glance in any commercial database. My game of 2010 against Franssen demonstrates clearly that I know 8.Bb2 is not critical. 10 minutes of preparation was sufficient to find this information.

The last example is a very recent game again of the clubchampionship. Robert surprised me in a very special way.

If you don't remember playing the same line in a standardgame against the same opponent then clearly we can't speak about any serious preparation. However I should add that Robert has played according to KBSB already 2659 standard-games. Compared with my 432 then it is evident that we are talking about a huge amount of games. The importance of a good system to archive games is here once again proven.

The 4 examples are from my 10 most recent games. The preparation of my opponents was nonexistent or had no substantial influence on the opening despite they had more than sufficient time for it. I selected the most striking examples but also in the other games I notice that the preparation of my opponents often only consists a fraction of the time and energy which I spent.

Last couple of years I've published tons of articles on my blog with valuable information which easily can be used for a game-preparation against myself but I experience hardly any negative effects. I also notice that only few try to profit from my games published in the databases. If we combine this with the fact that the chess-opening has rather a limited influence on the final result then I do wonder if the use of passwords isn't slightly exaggerated.

Finally if somebody anyway uses some information of the blog against myself then this would only push me to work harder. Maybe it also learns me something new which otherwise I never would discover. Besides the most beautiful present a writer can get, is to discover that his articles are read.

Brabo

PS.
Last week somebody of my club made the remark that it is difficult to search an article/ subject on this blog (to prepare a game). Well there is a simple searchengine on top of this blog. Where and how can be checked in below screenshot.
An example of how to search a topic on this blog
I also make regularly use of this searchengine as I don't remember the full content of all my articles and I try to avoid repetitions.