Saturday, July 27, 2019

Statistics

"Best by test", said former-worldchampion Fischer about 1.e4. He got the best results with 1.e4 so he kept playing the same lines which brought him fortune earlier. Chess is about results so logically we like playing openings which have won us games before.

On the other hand some amateurs ditch an opening from the moment they have lost a game with it. They change openings at the same rate as they change their underpants. They play any opening but know nothing. They don't want to make any effort to find a solution for a problem in the opening or worse they think wrongly that the defeat is because of the chosen opening.

In the book Ivan's Chess Journey Unravelled the strong Dutch grandmaster Ivan Sokolov gives therefore the advise to test at least an opening a couple of times before taking the decision to give it up. Next to that he also made the interesting comment that he sometimes removes openings from his repertoire although they are theoretically sound. If the resulting positions don't fit your style and results remain under par then obviously you can't keep on playing them as a professional.

So statistics influence our choices in chess but in comparison with computerchess this is child's play. In computerchess statistics have always been used very intensively to make progress. Each new minor-upgrade of an engine is tested extensively not only to remove any bugs but especially to define any change of the playing-strength. Besides in the course of the history of computerchess we see an increasing use of techniques embracing statistics.

Speed had always been the key. The faster we can evaluate, the faster we can make new changes. The first logical step was to make automatic testsessions to avoid the slowness of the human operators. In the last decade as hardware kept gaining speed, it became also more and more interesting to choose for rather short and more than slow and long (games). This change of analysis I already explained in 2015 here see computers achieve autonomy and nowadays I use it myself regularly.

In the last 2 years programmers even started to work with only bullet-games or faster. The path was cleared for the neural networks which more than ever need huge amounts of games to learn. This way AlphaZero played against itself 44 million games and learned to play the best ever chess performed in history in just a couple of hours. It was only given the rules of chess and the rest was figured out by the engine. At least that is what we were told everywhere as you could read e.g. at chessbase. The real story behind was much more difficult to discover. To learn more about it I decided to buy the very hyped book Game Changer.
I am halfway with the book but meanwhile I do realize the book wasn't meant to explain the code of Alphazero. The authors are in the first place players so not surprisingly they concentrate on the chess-content of the games. It is still a nice book to read but to learn about the technical aspects of the engine, you better study the scientific document of Google/Deepmind.

That file explains that the engine works with a probability-distribution of 4672 possible moves. Knowing the starting position only contains 20 possible moves then this is clearly something very complex which not everybody can create. Even the experienced Leela-developers which used the input from Alpha Zero had still many questions. Maybe this was also the purpose of Deepmind. They try to give a new method to the world for creating a superstrong engine but leave things open so developers are forced to use their imagination.

Besides such very sophisticated learning-process is totally useless for a chessplayer. Playing 44 million games in a couple of hours is no option. The only statistics based on games played by humans with some value, are openingbooks extracted from a database (see my article green moves). Still even then the value is rather limited which I experienced first hand recently. Last season I played a game in which I played after the moves 1.e4 e5 with black 19 consecutive moves considered to be the most popular one at master-level. The Chigorin-variation of the Spanish remains today the undisputed number 1.
Only at move 21 I deviate from the 4 last master-games in the big-database 2019. It already had little statistical value but more remarkable is that black was landed in an unpleasant position. This combined with a well prepared and excellently playing opponent made a deadly combination.
Benjamin is a redoubtable opponent about which I wrote before already on my blog see ambitions part 2. Likely we can consider this game one of his better or even his best. For me this defeat was a sign to study properly again the Spanish Chigorin. 20 years of experience doesn't mean you are done with an opening. Fortunately I was lucky as last year a book about this opening was written by 2 grandmasters: Ivan Sokolov and the Spanish grandmaster Ivan Salgado Lopez.
As the authors notice in the book, it is weird no similar book was ever written before about this opening despite it is the most popular line of the Spanish. In the meantime I finished the book and I can definitely recommend it for anybody interested in this opening. We get a very good and extensive overview of the opening and it seems the opening is still fully playable. Also many improvements are given upon old evaluations. At chesspub the critic was given that it is incomplete. It is not totally unjustified as also 20.d5 isn't treated in the book. Still we shouldn't be too harsh either. With Na5 you can find more than 8000 mastergames in the big database 2019 and we still need to add the correspondence-games. In the book there are numerous alternatives given so you can get with ease around any missing lines with both colors.

Brabo

Monday, July 8, 2019

Computers achieve autonomy part 3

A non-chessplayer can't understand why somebody would love to stare for hours to a board with some wooden pieces while in the meantime the weather is nice to organize a barbecue with some friends. I rarely talk about chess with non-players. However also inside our chess-world there exists a lot of variety. The people playing competitions is the largest and most important group but we also have niches of which the computer-freaks are likely the greatest weirdos. They are the nerds of the nerds or some sort of super-nerd.

At least that was the case till recently as last year I noticed a clear change. The arrival of AlphaZero end of  2017 created a chain-reaction. This is very different from what we experienced after Deep Blue defeated in 1997 the reigning world-champion Garry Kasparov. At that time the revolutionary engine got dismantled leaving many questions unanswered. This time the momentum was kept as the code of the new engine was shared so other developers could create an AlphaZero for a standard PC. I am talking of course about lc0 or also called leela (more about this in a later article).

After decades of traditional alpha-beta programming we now see the steep and very spectacular raise of neural networks of which we could witness some very interesting clashes of styles in the most recent tcec-championships. Attractive games combined with a good marketing-strategy caused an explosive increase of chess-fans wanting to follow computer-chess see below graphic which presents the number of viewsessions per month for TCEC.
Bron: http://www.chessdom.com/tcec-season-15-to-ab-or-nnot-to-ab-that-is-the-question/








Getting 2 million viewsessions per month by just showing 1 game at once is definitely something extraordinary. Ok Carlsen does still much better but 99,9% of the other professionals never gets this kind of views. Besides this is not the only thing which proves that computer-chess is booming. On youtube we see a huge increase of videos in which games of engines are discussed. However even more stunning is that reporters now start to refer to games played by engines when they cover tournaments played by grandmasters. A couple of years ago this would've been totally unacceptable. The only reason of engine-games were till recently to understand which engine is stronger and almost nobody would value them equal to games played between humans. Well today some reporters do compare games played by top-grandmasters to a game played between Lc0 and Stockfish as happened e.g. in fide grand prix moscow semifinal chess.
Although the grandmasters started their game a couple of hours later than the engines, I do think this was just a coincidence. Nakamura had played this line already several times even just a couple of weeks ago in a rapid and such things aren't missed by a top-professional like Grischuk in his preparation. On the other hand I find it very remarkable that the engines manage to select a hyper-modern opening without using any openingbook nor any human intervention happened.

The fact that the engines can autonomously create games with interesting openings is something very important. Obviously top-players have discovered this too. Since 2010 I already maintain a database of games played by engines to use for my opening-analysis see e.g. using databases. I guess at that time I was an exception but today I am sure any ambitious professional does this even Magnus Carlsen. Well more likely his entourage takes care of it which I deduct from a twitter of his helper, the Danish strong grandmaster Peter Heine Nielsen: "Computer-chess is spectacular."

Besides those engine-games have also influenced Magnus' game. Many of the so called novelties already popped up in games played by engines. We see proof of this in the worldchampionship played against Caruana end of last year as in later tournaments.
The opening is the most obvious phase to learn from but also later phases can be instructive. The American top-grandmaster Sam Shankland would've never resigned below game if he had seen the 35th round of the tcec super-final of season 12. I already used the position in my article fake news to illustrate the gain of strength in endgames of Stockfish.
Earlier an engine was nothing more than a tool. Today more and more people consider engines as an entity with its own identity.  Many players cheer for their favorite engine via the chat-box or in fora. Some even make their own website for them as you can see in this example mytcecexperience.

So we see less and less difference between games played by engines or humans. Last Tuesday Ex-Machina was shown once again at the tv but this seems for chess to be today more reality than fiction.

Brabo