Thursday, March 14, 2019

Lc0 vs Stockfish superfinal TCEC season 14

1. Pre-match expectations of Leela.

Beforehand SF was considered the clear favorite. There are a number of reasons for that: the rating of Leela (Lc0 v0.19) at Ccrl40 was not great: the engine wasn't mentioned in the top-100 (version v0.20 was later during the match ranked at the 45th place). Second, Lc0 performs weakly using classical hardware compared with the traditional engines. That is no surprise as Lc0 is built for fast chips of graphical cards (GPU's), instead of CPU's. On a normal CPU Leela can't obtain maximum strength just like Fritz3 initially couldn't perform well as it was lacking RAM for the hash-tables. See for more information in the article of Frederik Friedel at Chessbase (the adventure of chess programming part 3). Finally I also thought the engine was tactically not mature yet which we saw in the previous tcec competitions and based upon my own usage of the engine. So a match against the undisputed leader of ccrl40....

2. Summary of the match

At the beginning SF was clearly the better engine, twice it took a lead. First 2-0 after only 10 games, but after game 13 the score was again tied. After that SF won 3 games on a row!, but again this didn't last very long: after game 29 Leela equalized the score. After that Leela took the lead: games 49 and 53 were won by Leela. It became an exciting match with switching leaders: we hadn't seen this for a longtime between engines. Maybe it didn't happen anymore since the Braingames "candidates-match" between Fritz en Junior in Cadaques 2001, when the engines were playing for a match against Kramnik. Junior got in that match almost 5-0 for free, after only 5 games but Fritz equalized in the second half of the match (24 games in total) and won the play-off with 2-0.

However the lost games were very painful for Leela - it reminded me to the match Botvinnik-Bronstein: Bronstein played ingenious chess, used new concepts, played very differently than Botvinnik. Botvinnik tried to reach draws by adjourning games and had a lot of trouble to score some wins. Leela wasted several half points by lacking tactical awareness. A good example is the 20th game, in which Leela plays the losing move (39...Rb6-d6) with an evaluation of 0.26, but SF answers with 40.Rg3+ and immediately shows +8.56 - probably the "boom" of the match. Maybe it wasn't a draw (SF was already giving +2,5 to itself), but more blunders would occur. In the next game it happened again: in an equal position Leela blunders once more and SF hits back immediately with taking at f2 (-4.46). Game 66 again. A very weird loss was the 85th game, the last decisive game of the match: Leela still believed it was a draw (overvaluing a far advanced free a-pawn) while SF considered the position for white already for a longtime as totally hopeless. When Leela realized it then the evaluation plumbed to -14.28 (SF was given already mate in 41…). In a very rare case Leela missed a certain win (65th game) in which SF (using 6-men tbs) was already 100% sure of the loss. This was the consequence of the low search-depth and less extensive usage of tbs. At game 80 the score was tied again.

3. Learnt lessons about openings, playing-style and other aspects

Besides the impressive performances in the middlegame (Leela) and endgame (Stockfish), there were also a number of important learnt lessons about the openings.
A first highlight was game 11, in which Leela gradually increases the white advantage from the opening (French). This was very impressive, especially as the evaluation of Leela was more than 10-20 moves ahead to the one of SF - it looked like grandmaster against amateur - only, the amateur has the strength of a super-grandmaster. Beside the evaluation of Leela is something you need to take with a grain of salt: in the first game the evaluation of Leela at move 104 jumps up to 2.65, while SF sees no problems. The same happens in game 9 : suddenly Leela shows an evaluation of +2,24 when it can exchange queens and obtain a bishop-endgame with a free a-pawn - SF again evaluates the position as fully equal and the game ends in a draw. Also in the 85th game: it seems Leela puts to much trust into the free a-pawns? There are many other examples of the too optimistic evaluation.

As stated before, the variance in the opening is good, but the engines seem capable of turning the most sharpest openings into forced sequences leading to boring drawn endgames. Fortunately some nice middlegames were played, but when SF after e.g. 1.e4 c5 2.Nf3 d6 3.d4 cxd4 4.Nxd4 Nf6 5.Nc3 a6 6.Be3 e6 7.a3 b5 8.g4 Bb7 9.Bg2 h5 10.g5 Ng4 11.Bc1 Qb6 with white and black gives very quickly 0.00 then such perfect play looks not exciting anymore. The same scenario in the 5th and 6th game: SF makes very quick draws in the Kings-Indian.

Leela couldn't do much with the kings-gambit and lost - SF could just hold the draw with white. Although a couple of games can't define the correctness of an opening, it is rather symptomatic that it is the kings-gambit leading to troubles for white. It seems the romantic opening isn't more today than just a surprise-weapon? As earlier written, the French game (game 11.1) was a highlight for Leela in the match - this time SF couldn't show the same quality for white. The French opening seems something Leela knows best as also in game 35 we saw SF having big troubles after the opening.

The win of Leela in the Nimzo-Indian was rather related to some small errors in the middlegame of SF so not due to the opening. In game 16 SF practically destroys the Pirc in the opening. The Pirc had a rough time in this match: in game 55 SF almost didn't make the draw. It became the longest game in the TCEC-history: 264 moves before Leela agrees with the draw. In game 71 we see the Pirc creating another dramatic turnaround. Leela has a stranglehold but can't break the defense despite dominating the whole board. White has everything - black even has to evacuate the king to the a-column - but it is not enough.

Generally we see that the smaller openings don't stand very well the test, nor do the sidelines of the big openings: it is striking that the white advantage is only disappearing after move 25. An example of this is game 23.

A great win is scored by Leela with a white stonewall in game 25. A Philidor/ Lion-opening in game 27 is annihilated by Leela - one of the seldom moments in which Leela manages to get SF away from the 0.00 evaluation and wins deservedly.

Games 59 & 60 show what "sharp" lines for humans mean for engines: the Sicilian Dragon creates 2 short draws. Even the Spanish is used but survives a Leela evaluation of 5.76 in game 75. The end of the match looked like football, in which team A has a lot of ball-control, but team B scores twice via the counter in the final quarter.

4. What could've been better in the match set-up?

Contrary to earlier tcec super-finals, this match of 100 games seemed "too short". The engines were very close in strength, and after 70 games there was still only a gap of 1 point. After 100 games this is also the final difference: 50,5 - 49,5. Also the openings could've been chosen a bit better. In most cases positions were chosen after 5 moves which were more or less neutral, mixed up with some deeper lines (which were randomly chosen from the opening-book created by Jeroen Noomen). Those deeper lines were not always creating interesting middle-games. The problem (for humans) with fun openings (like Marshall-gambit, Sic Dragon, Botvinnik-gambit, kings-gambit, Albin countergambit, Sveshnikov, Sämisch KID, Sevilla-variant Grünfeld, …) are that they equalize quickly ( due to a too forced mainline) or almost always give a win/ loss for white/black (as one side has a too big advantage). Some openings are complex for humans but that is not necessary also for engines. Nonetheless I agree with the critics to use more starting-positions from grandmaster-games. this would improve the relevance and use (for the practical player) more. Maybe this is something for the superfinal of season 15?

5. Other things which we can improve?

SF had the advantage of using 6-men tablebases (tbs), Leela only worked with 5-men. That difference for sure meant for 1 game the difference between win and draw, and had - with equal weapons- given a tie so 50-50 as final score. Now engines can already use in the opening those endgames tbs, so this is important in a match. Probably over 10 years we will have 8-men tbs (so having a solution for all rook-endgames with 2 pawns each - great !) so this aspect will become in the future even more important. Or maybe we should do the opposite so forbid the engines using tbs at all?

6. Conclusion: are we close to perfect chess?

Positionally Leela is close as to beat SF this needs a very high level of play. Tactically SF is still (a lot) stronger. The great search-depth avoids missing any tactical traps. Also this allows SF to defend some very difficult positions. One aspect of the development of Alpha Zero and Leela reminds me of what professor Jonathan Schaeffer experienced when developing his prefect playing checkers-engine Chinook (by the way if you want to read a beautiful and emotional story about the first engine beating a reigning world-champion then I recommend very much "One Jump Ahead"). It is something what Schaeffers team and also recently the team of Demis Hassabis (Deep Mind and Alpha Zero) noticed: further development leads to an increase of the draws (an indication that chess is a draw when played perfectly, or that there is a limit to further improvement). That effect can be partly explained by the fact that "a draw is a draw" for an engine. In other words: the simple evaluation of 0.00 should be added with other parameters as otherwise the first move in the list leading to 0.00 will be played (see for that behavior to an article of Tim Krabbé about pealing an orange in Alaska ("morons"). An intelligent add-on would be that the engine selects the move bringing the most chance to errors for the opponent (let us not consider contempt). This can be a line with many forced moves or avoiding exchanges. I guess some modern engines already use such parameters doing something like that but it is not yet working perfect.

The advantage of Leela is that "the engine" can now do the development - sooner or later the development of Stockfish (despite all tests the engine plays against itself eventual with a self-learning function) will stop when reaching the limits of human programming. Leela has the absolute minimum needed code to seek maximum results. One of the Leela developers wrote on his blog that if a developer of a classical engine (e.g. SF) takes a holiday for a week then the engine remains the same while with Leela after a week it became by itself again a bit stronger. It appears Komodo already hit the ceiling: at ccrl40 release 11.3 has 3 points more than release 12 and 12 points more than release 12.3. And also the MCTS-version of Komodo is getting close to the classical one. It looks like the Americans have reached their tipping point.

But as I have said: the evaluation of Leela - contrary to SF - is not fireproof: never did I see so many positions with a "winning" evaluation (+2, +3, +4, or even +5 and more…) not transformed to a win. The winning line is often so small for engines that one small deviation is sufficient to lose the advantage. Leela is not yet able to avoid those mistakes - so this looks like us humans playing chess :-) However for the practical player, this disadvantage can become an advantage, and Leela is definitely a good addition for new ideas (plans), or to find practical chances in non-tbs endgames (something which SF will rather evaluate as 0.00 without giving a view about practical chances).

Epilogue:

Some critics have pointed to the openings as not well chosen, so additionally a rapidmatch was created without any pre-selected openings. Leela won this one with 56-44 (see e.g. s14 bonus match leela stockfish. That is a very large margin especially as tactics are normally much more dominant in rapid-play. The next season will have to answer the open questions.

HK5000

Thursday, March 7, 2019

The Leningrad

I bought during the first years of my chess-career from each (big) opening in my repertoire a book. The Spanish, the French, the Caro-Kann, the Aljechin, the Pirc and the Dutch (stonewall) were the openings. It costed me about 3000 Belgian francs (approximately 75$) as the euro was not existing yet at that time. For a student with almost no money, we talk about the beginning of the 90's, this was a serious investment.

Later I never wanted to buy any new opening-books. The rise of the engines and databases allowed me to create my own opening-analysis. Besides if I would like to keep my repertoire up to date by buying always the newest opening-books then this hobby would become very expensive. Also I noticed that many books are repeating a lot of what has been written before so you get less and less new interesting information. Finally I experienced a lack of motivation to study all the theory from opening-books. Very often an opening-book was only used as a reference work so it was never studied properly.

Only beginning of last year I made an exception. After more than 20 years I bought another opening-book: The Leningrad Dutch. I had decided to add this opening to my repertoire (see why previous article) but I realized immediately this can't be done quickly without external help. The Leningrad has a labyrinth of different variations. It is completely impractical to check 10.000 games in the databases to build an overview of the opening. So it is necessary to ask an expert for advice, somebody having played the opening for many years. The easiest and cheapest way to do that is to buy a book which I did. There are a few options but eventually I chose the book written by the Ukrainian grandmaster Vladimir Malaniuk. He was probably the biggest pioneer as he played the opening during more than 3 decades against other grandmasters. I use the word "was" as he died in 2017 (some sources say he was assassinated).
Meanwhile I already read the book twice through. In my 2 recent standard-games with the Leningrad I was able to use immediately some recommendations from the book. However when I investigated the correctness of those ideas later at home then I discovered that the author is often too optimistic for black in his evaluations. I start the review with a first position which I had on the board see again my previous article.
The Leningrad Dutch page 123 variant 1
The author tells us that white should try to equalize with accurate play. However after the novelty exf5 followed up with Qc2 my engines show a clear advantage for white.
The Leningrad Dutch page 123 variant 2
We have moved 2 moves in the same game. This position is reached in the book via a different sequence of moves than in the game. The author evaluates the position as equal while the engine again shows a clear advantage for white after Nce2. Fortunately my opponent missed that idea in our game.

A fragment of another Leningrad played in one of my standard-games, was already published in the article desperado part 2. Again we see that the author is too optimistic.
The Leningrad Dutch page 269
White can't equalize and the author continues to prove this with the weaker 14.Rf3. However I already showed by using the engines that 14.Qg4 is still sufficient for equality.

I have bought the book mainly for the Leningrad but I also had a look to what the author tells us about the anti-Dutch systems. I was curious if the author had noticed a quite recent development in the Staunton-gambit.
The Leningrad Dutch pages 57 and 62
In the book is written that black's extra pawn compensates the small lack in development. However 14.Nd4 is completely ignored and already known from practice since 2009. White scored already 4 important victories with it in mastergames. The engine is very positive about the move for white.

I wasn't able to check most of the analysis of the book but I fear that I only revealed the top of the iceberg. It again confirms my perception that most books are written for the casual chess-player but don't withstand scrutiny.

Fortunately I didn't have high expectations in advance so I wasn't disappointed. I wanted a skeleton of the Leningrad and that is what the books supplies. The author has built a repertoire for black by showing for each of the important lines of white how he would respond to it. You could say that you have the shell of a house but all the rest still needs to done. This book is sufficient to start playing the opening after you read it but if you want to regularly play the opening then you better do a lot of additional analysis.

Finally I also want to add that I can remember a lot of analysis from the book but I also forget still a lot. I started to read the book for a third time but that will still not be enough to remember all the sidelines. Nevertheless especially in the Leningrad it is very useful to also remember what needs to be done in less frequently played variations. Some suggest to type all the lines from the book to a pgn-file so you can practice with e.g. chess position trainer. This is a Herculean labor which I don't want to do. Only the mainlines is for me the maximum. A recent new initiative is chessable. They do the job of digitizing for you. It is not free but neither much more expensive than a book which you can economize. Unfortunately the site offers today only a limited choice of books. The Leningrad Dutch is not included in their base. I have no experience with Chessable but looking at their increasing numbers of subscriptions, this could become a big player in the future of sharing chess-information.

Brabo

Tuesday, February 26, 2019

Dutch steps in the English opening part 2

6 years have passed already since I published part 1 on this blog. As I recently made some interesting discoveries, I thought this would be good to share in a follow-up. Besides I see many theoretical developments speeding up lately in a lot of openings which should be linked to the ever increasing strength of the engines. In 2015 I already forecast this effect in the article computers achieve autonomy. I notice in the last couple of years a clear progress in the domain of opening-strategy of the top-engines. Computers are able to find more often the right ideas also in openings which don't involve much tactics. They start to find critical setups. I expect in the next years we will see the impact of this in master-practice. A number of strategically dubious openings will almost completely disappear.

Indeed the Dutch defense should be categorized under the dubious openings. Till now I managed to fill the gaps but it becomes harder and harder. By the way it is not only the frequency of the problems but also the magnitude. I believe it is unwise to ignore and hope nobody would play those annoying lines on the board against you. For sure such narrow view will hamper the own development. The future of the Dutch opening is dark. So some people will wonder why I keep playing this opening and don't study something new immediately. On the other hand I don't think it matters a lot for my career if I play another year the Dutch. I am almost 43 so there isn't much reason for creating big changes suddenly.

In the article to analyze using the computer part 3 I gave a hint already by telling that I studied the classical Dutch. At that time I didn't go into details of why and what I eventually concluded. Today a year later I am willing to share my analysis as information has already been leaking. Let us start where we ended last time. Since 2012 I answered 1.c4 f5 2.Nf3 Nf6 3.g3 e6 4.Bg2 every-time with Be7 which maintained the option to choose between d6 or d5 dependent of how white would continue. I played 10 official games with it. I never found the resulting positions easy in this opening but it was the last game played in the 6th round of Open Leuven 2017 against the Belgian IM Stefan Docx which broke the system.
Whites opening-advantage is not big. However black can't neutralize it. Besides black has zero counterplay so this means a very long and difficult defense for maximum a half point. It is no surprise that I failed to achieve the draw in the game. It is normal to defend a bit in a game with black but equality should be reachable.

The classical Dutch is the most obvious solution but after several weeks of analysis, I didn't like it. The correctness of the opening is currently under discussion. It is also not just 1 line which bothers me but several critical lines are annoying. Finally it looked absurd to swap one dubious opening with another one.

All my work wasn't a waste of time as I was able to help my student Sterre Dauw to prepare for a critical game in the Flemish youth-championship category -18 which he won. In the 5th round he met the Belgian FM Jasper Beukema, his strongest rival and specialist of the classical Dutch. Sterre asked me if I knew an interesting anti-dote which I obviously did. At chesspub that idea was already mentioned but in practice it is rather unknown as was the case for Jasper.
There are a number of errors in the game but we can't deny that white has a clear edge out of the opening which black never was able to fully neutralize. Jasper already alternates the classical Dutch with other openings but I think it is smarter to just ditch the opening at least in serious games.

Maybe Jasper should once consult his older brother and IM Stefan as he is a specialist of the Leningrad. The Leningrad is the most reliable opening in the family of the Dutch. It also has the advantage that you can play it against a wide range of white setups which does include the English opening contrary to other Dutch lines.

In 2012 I wasn't ready yet to use the Leningrad. Today I don't have a choice anymore if I still want to play the Dutch against the English opening. Meanwhile I already tried it out twice in standard-games. My most recent one was played in the last round of Open Leuven 2018. After the game my opponent Marc Kocur told me that he plays the Leningrad himself already for years which explains why I didn't get an easy position from the opening.
I had definitely troubles in the opening. I clearly miss experience. Nevertheless it is a relieve to play this kind of dynamic chess compared to e.g. the stonewall. Although I regularly lose control, this is much more fun. Theoretically this still looks reasonable. Anyway this is likely the final step in the Dutch defense against the English opening.

Brabo

Monday, February 18, 2019

Big Database

Less and less players are still willing to invest into a big database. You can find today a lot of free online (see the introduction of my article ultracorr-x) and without the (too) expensive updates most don't have enough energy to keep a database up to date. Besides the current top-engines offer in most cases much stronger moves than the ones played by any grandmaster.

In Chessbase part 1 I wrote that many CB users never use 95% of the features. I am convinced that this is largely because people don't have easily access to a good database. CB15 and in a lesser extent Fritz lose (almost) all flexibility without a database.
  • Automatic game-analysis with references to games played with the same opening
  • Opening-reference-function
  • Creating an opening-book from a selected database
  • Searching themes
  • Plan explorer
  • Endgame function 
  • Automatic preparation against an opponent
  • Calculating elo-ratings in a database
  • Speed of research and maintenance of databases
  • Exploring databases
Anyway it is weird to buy CB when you are not going to use any of the features mentioned above.

Therefore I advise CB- and Fritz-users to maintain a database which can be used as reference. The next question is of course which database. For this we need to check a number of criteria:
  • What is the price not only of purchasing the database but also keeping it up to date?
  • How often are updates done to the database?
  • Can we find games of amateurs so it is possible to prepare for them?
  • Of which countries games are stored in the database?
  • Are pure engine-games, anonymously online played games (mostly blitz),... added to the database just to pump up the size?
  • Can you find old, historical games in the database? Are efforts made to expand this archive?
  • Are names, ratings, places ... correctly inserted for each game in the database?
  • Can you get automatic updates of the database?
  • Are the games annotated?

Well it is impossible to compare all available databases in the world. At chessgameslinks.lars-balzer.info you can find more than 100 links to different databases and I am sure this summary is not complete. On the other hand if we ignore the price then I am sure that the Mega Database upgrade (from the previous bigbase or previous mega database) is the best choice. Quality, quantity and service is not equaled by any competitor. However 120 euro each year is not cheap. Also you could wonder how much value do have annotations. Within a couple of years the analysis are outdated  see my article of 2016 in which I indicated that the top-engine gains averagely each year 55 ratingpoints.

So I recommend to not spend lots of money for annotated games. CB-users have a much cheaper alternative with the online update reference-database for only 60 euro per year. Fritz interface-users should choose between big database 2019 for 70 euro per year or otb-openingmaster for 59 euro per year. The otb-openingmaster is somewhat cheaper and on top you get 3 updates per year. However you can only get the database via a download-link and it is not a CB-product.

Finally the cheapest alternative without losing much quality is probably still good old TWIC. By investing 10 minutes per week, you can download and add a nice collection of recently played games to your reference-database completely for free. If you don't find it critical to get each week the newest games then you can choose to bundle the downloads twice per year like I do. That way you only need twice 1 hour per year. So in 2 hours I saved 60 euro or is this too optimistic? This would be nice to find out so I bought Big database 2019 and compared that with the Mega database 2016 complemented with the twics of the last 3 years. I think there were about 1-2 weeks difference in favor of twics which can explain some small deviations. Below table shows a detailed comparison of the numbers in different categories: total, + 2500, + 2300, world-top 10, Belgian top 10, Belgian players and history. Just for information I also added the numbers of the free online database chess.db although you can't use this as a reference-database.
More games in a database don't mean a more interesting database. We see chess.db claims to have 2 million games more than the Big Database although it lacks many relevant games of Belgian players.

Concerning twic it is remarkable that you don't miss any games of grandmasters. Still we can't ignore the 650.000 missing games over 3 years. Chessbase clearly makes an extra effort to also include games into their database of the amateurs. They know that their customers are in most cases not top-players but clubplayers interested in what is played by their direct rivals.

Despite few will consider the 1800 "new" historic games as valuable and likely it is not interesting financially for Chessbase, I do like the nice bonus. Fortunately Chessbase not only focus to the commercial interest of the database but also takes the role of archivist of the chess-history. It takes a lot of time to digitize old newspapers and magazines contrary to the few clicks needed to download a new collection of games from a website.

Personally I find it rather expensive to buy each year a new big database for the extra you get compared to the twics so I only do it once every 3 years. Also I detected another advantage of the big database. Twic only shows the first initial of the first name and often goes wrong with the spelling of surnames (especially Chinese players). The data of the games is in the Big Database much more complete which allows to search quicker and easier.

This year I bought a new big database while using the prize-vouchers of my son. Beside hereby I also got a new up to date powerbook. Few players are aware about it but the powerbook 2019 offered by Chessbase for 70 euro is something you can create from the Big database by yourself. I even created 2 different ones:  1 openingbook with games of which 1 player has at least +2300 elo and 1 openingbook with games of which both players have +2500 elo. You must have patience as on my 4 year old laptop it took 12 hours to create the first and a bit less than 2 hours for the second.
Openingbook of games with at least 1 player +2300 elo, filtered from the big database 2019
Openingbook of games with both players +2500, filtered from the big database 2019
It is funny that the rating barely influences the popularity of the first move. I guess most amateurs like to copy what the professionals are playing. However we do notice that the advantage of the first move increases slightly for the higher ratings. We see the advantage goes up after 1.e4 from 62 elo to 78 elo, after 1.d4 from 60 elo to 74 elo, after 1.Nf3 from 42 elo to 64 elo and after 1.c4 from 40 elo to 60 elo.

Naturally statistics are one of the main assets of a database. Anyway most of the treasures are hidden in the games. I started my article by telling that some people think engines are sufficient to get a good evaluation of an opening. Well I think this is a bit too simplistic as we have a very rich history of chess. Many ideas can be discovered again in a database which are still valuable today and which engines won't able to show you. No don't think engines know always more. Just look at the ongoing TCEC superfinal of season 14 in which the so called invincible Stockfish is at this moment a point behind while we are less than 1rd from the finish. LeelaChessZero (derived from Alphazero) is busy writing history.

Brabo 

Tuesday, February 5, 2019

Chessbase 15 part 2

Using CB15: some first experiences

First I must say that I came from CB12 (and before CB7) so probably I will like some features which a lot of readers already are used to. I also have Fritz 12 and 13 so this will not always be the best comparison with the latest Fritz-GUI. I am already a customer of Chessbase since Fritz5.32 and a reasonable early version of Chessbase (with dongle-security - how long ago was that). Before I had already Fritz2 which I used quite regularly and liked very much. After that I worked with Chess Genius 3 (very strong), Rebel 6, and many other good and less good DOS-chessprograms. I also still possess a Big/Megadatabase (1980-1994) with no less than … 215.000 games. In other words I saw practically the complete rise of computer-chess.

Anyway this review is personal as everybody has other preferences. Something which I find interesting will be disregarded by somebody else and vice versa. I am not going to talk about the most obvious things here of CB15. Neither will I try to cover everything. I want to discuss some things which impressed me good or bad in my daily usage of the software. On the internet you can find other reviews of CB15 so this is my contribution based on service pack 7.

First CB15 has improved one of their old features. I remember in the old CB-releases (and still in the Fritz-Gui) it is not easy to filter a database with positions to get only white (or black) to move despite this was already available in the 90ties by Nicbase. I see they solved this issue in CB15: in the search-mask you click next to maneuvers white (or) black. See the next figure in which I tapped on “B” (black). It is a pity that not everything in the window has been translated to Dutch. If other languages are offered then you would expect the translation in release 15 is done more professional.
OK, so that is one thing which I like about the new search-mask. Still Chessbase has not yet included Chess Query Language (CQL) contrary to their concurrent Chess Assistant. In Chessbase you can only use what has been programmed. That is a lot as you can see in the window above. If you click on examples then you can find 10 extra themes - see picture below. Only 10 yet so there are still a lot of other things to develop in the next releases: e.g. what about different sacrifices or exchanges?
Let us try the next search-task (*) “queen decoys into fork”. That gives 10 hits in my database of Belgian games. Funny if I use the search-task “rook double check” then I get the same 10 hits (so wrong obviously), for which I sent a mail to Chessbase. With that filter (*) I found below interesting game from my period in the liga of West-Flanders. White messed up after move 17 but that makes it just more juicy to replay the game.

For those not knowing CQL, Tim Krabbé wrote a good article about it, and you can also read about it on wikipedia. The advantage of CQL is that you can create some very detailed search-masks like "white rooks sacrifices itself, starts from a white square, after that the white king runs from h1 to a8 in the rest of the game". This doesn't work in Chessbase but can be done with CQL. Of course who needs such searches besides Tim Krabbé J.

Also annoying is that CB15 always creates a search-accelerator after each search - the best to get rid of the complaints is just to accept it. Better would be that the program creates one in the background autonomously or not at all. These searches should be something basic for a programmer to develop. Chess Assistant is very fast so why isn't Chessbase? Is this linked to the messy structure of the database? Some bugs of the formatting are well-known. Today these bugs are sometimes called features (before you could therefore enter illegal moves in CB). An example of inefficient coding is a search on the won games of Michael Adams, played between 2015 and 2018, against opponents higher rated than 2600 elo in the Big database 2019. We see Chessbase checks the complete database while it is already chronologically ordered. A big improvement of the speed would be just to check the last 1.100.000 games or so. It seems to me that the search can be much faster.

A positive thing is that when you search for a theme (e.g. sacrifice on h7) then all the games start from the position where the sacrifices happens so making it much more efficient. On the other hand CB15 is often very slow and stopping a long search isn't always easy. Often it takes quite a while before the program reacts and sometimes you even get the window below:

I would expect some more stability for the 15th edition of the program but maybe my hardware plays a role here too (hint: my old HP desktop is not very stable with a new Windows). When I wanted to start using CB15 then I first needed to install service pack 5 as I couldn't save any games in any database. You would wonder if anybody tests Chessbase before releasing. Also after the necessary service pack updates, my version still crashes sometimes but ok this can maybe be caused by my computer.

Let us return to the filter (*) after the move-sequence Qxa8 Qxa8 Nc7+, for which there is no need of a specific search in the filter. You can also use a more common method to find such themes. Go to the position in which the theme occurs, click "report" and "similar moves". This immediately generates a list of games with the same theme and similar pawnstructure. It seems Kurt used something which happened already in a game of Lerner-Tseitlin played in 1973. You could even state that because of the popularity of the theme that it is a rather common trap. The advantage of this second method is that you also get other sacrifices than Qa8: like the one happened in the game Sorgic-Petrovic with Ra8.
Another plus of the search-window: if you ask Fritz to give you the won games of a player than you have to select separately white and black. In CB you can do it with one click: "only won games" and "ignore colors". This is useful but what about "only lost games"?

Searching, filtering this everybody can do but why is CB15 special? What makes a new release of MS Office, Fritz or any program again better? Well "replay training" and "planexplorer" look nice but I don't use it a lot. It are nice ideas especially for <2000, having troubles to make a good plan or to find the right squares for the pieces. The replay-training offers the possibility to guess the next moves by showing a diagram of how the position looks 3 moves later. The plan-explorer tells you which moves occur the most often in a position and what most likely should happen with your pieces in the future. Anyway this is only useful for the opening when there are still grandmaster-games in the database. Once you left theory then you can't compare anymore. So these 2 features exist for the player willing to work at chess. I doubt this is the best way to do as I learned more from the video's of Herman Grooten.

Another positive thing is -when I wanted to erase some variation and I accidentally erased everything - that I could use in CB15 ctrl+z”: which put everything back just like Word. If I want to copy a game (I often do from BB2019 or any other big database when I added comments as that would demand that BB2019 to be indexed again - read slowing down) then it is now much easier as I can just choose from a list of databases. Also nice is that you don't get anymore a popup window when adding a new line (main/side-line/ overwrite/ cancel or other stuff), as CB15 immediately inserts the new move as a new line which saves time.

So the search-tasks are fun but what else is possible? Statistics is the other great pillar of CB15. We know already for a longtime statistics of one player but now you can also find out the probability to where a piece will be played (can be useful to know in an opening where the queen-knight should normally be). You best use such statistic on a limited database of openings like www.pgnmentor.com – as a general search in BigBase2019, won't be useful). You can also ask the "surviving-probability" of a piece in a specific opening. That will tell you which pieces are normally exchanged. This is of course pretty similar to how Alpha Zero is playing chess. Sometimes more information is also too much information.

After the 11 moves of the game Belkadi – O’Kelly I checked which endgames occur in a standard-position of the Caro-Kann. Left-below you see the probabilities of the different type of endgames B vs N, N vs N or R vs R. That can be useful to know which endgames can occur in an opening. The figure right shows the same search after 6.f4 in the Sicilian Dragon: so mainly rook-endgames. Be careful as only endgames with pieces are shown and no pawn-endgames (because of double counting). It could give you an extra incentive to study rookendgames or also that you will exchange the queens and lighter pieces quicker. But again this more sounds like statistical chess.
The tab "players" in the database-window is the place to be when you want to prepare for an opponent. I wonder why they put the search-window for the name below (would've been better at the top) but ok it works. Not much changes since last releases. I still don't like that the 3-part-window of "player statistics" see picture below is still very small. It can not be very difficult today to increase the resolution and to enlarge the pop-up window so you don't need the scrollbar. The buttons below the windows (“N”, “A-Z”, “Result”, “Date”, …) should be working in both directions like Explorer in Windows: 1 click on the button (Aà Z) sorts alphabetically while a second click is the revert order  (Z à A).
And why are the sequence of the pictures in the ID-card of a player from "most recent" to "oldest". Ok maybe people are more interested in the most recent picture of an opponent so it is a detail.

CB15 clearly puts more emphasis on statistics. It is no surprise as we live today with Alpha Zero and data-mining. Eventually CB1 gave Kasparov the tool to take a sweet revenge against the team of Hamburg in his second simul. It is logical that Chessbase has chosen this path as there is still a lot to develop in that domain. The strength of a novelty can be shown by the performance (typically very good at the beginning but after some games it drops as good anti-dotes were found - when this happens fast then it mostly indicates the weakness of a novelty).

The 3rd pillar of Chessbase is publishing. Chessbase has adapted to the modern world: publishing to the internet (connection to facebook): one click. Publishing to gif which replays the game automatically (e.g. to publish on twitter): one click. Or just copy/paste to Word is still possible. Chessbase has all the modern tools so that is good. To copy paste a diagram to Word can be done in different ways: you can use the diagram in figurine font but also as bitmap with the more beautiful look and feel of CB15 (something which is also possible in Fritz).

Another good thing of the game-window is the bar with read- and evaluation symbols. Also fun is that Chessbase now immediately adds an evaluation of the moves when you upload a game so you see instantly how well the game was played. By the way this feature existed before in Fritz.

First I thought that “ctrl+shift+r” doesn't work which is something I liked a lot. Automatically some games are added to a game in which the same opening was played. It was very easy to check which master-game you followed the most moves. There is now a standard button (Novelty Comment) in the ribbon "Report". The tab "Reference" in the game-window looks similar but then you have to select yourself the right position and hope not too many games are existing (not too many but also preferably played by masters). "Opening-report" is also something similar but it is a different report and I want to have it in the game popping up. "The reference search-window" is part of the same family (drag&drop could be used in my game-window) but it is not done automatically. So I decided to consult the help-function (that small circle with “?” above right) but the following page popped up (still not corrected in CB15 sp7).

In the end I downloaded the very complete Dutch manual from their support page. A pdf of 691 pages! In that manual I found the command “shift+f6”, but that result looked very similar to above option "reference search-window". It seems they replaced the automatic reference till I discovered some new options when clicking right at the board. One of them was "search novelty" but also I noticed "novelty comment: Maj+ctrl+R”: that was the command I was searching. It was still there but only we are used to shift+ctrl+r and not maj+ctrl+r). For those wanting to know what I was talking about see the next picture: that command searches the most relevant games in the reference-database, so the ones matching your played opening.

I also want to add that I very much appreciate the support from Chessbase. They answer all your questions in a polite way and yes they also play chess. The support-pages of Chessbase are also good and on the chessbase-wiki many questions are answered already.

One other critic I have but not on CB15, but on Fritz interface: when you add a diagram where an interesting move is played, then this should be done after the previous half move (the diagram is shown in the move after move "x"). In CB15 this logic is used but not in Fritz and co as when you copy it to Word then the diagram pops up before the previous half move so ("x-1") even if you work the same way as in CB15, so putting the sign "#" after the move. It is very annoying when you mix Fritz and Chessbase for the same article so I sent another mail to Chessbase.

Maybe somebody wonders how you can have a diagram in CB15 before the first move of a game. Well you click on the first move and type “ctrl+shift+a” (= comment before the first move) and add [#] in the window. The next figure shows a diagram before and after the first move (if somebody wants to practice tactics then what do you play (after Qf6-f5) to win?). Also nice is that you don't see the diagram as “#” in the notation on the screen but like a real diagram which is easier to remember ("where did I put that diagram again") but also the layout is better, you want a diagram visible to show the actions, but also not too often diagrams.
I do consider it weird that the colored arrows and squares are not available as a ribbon command in the tab "insert", and only are possible with a combination of keys and mouse. I had to search but I found them (press alt / alt+shift / alt+ctrl and use the mouse). We are now talking about graphics which can be used also as a search-criterion. It is a bit lame but possible. Maybe some people analyze their games and like to use this search-mask to find those games. The Chessbase help-pdf also offers the possibility to add medals to your own games to find them back quicker.

I didn't cover in this article how to create and adapt openingbooks (OB) which is rather something belonging to Fritz as it is more simple there. Sure CB15 allows you to make a much more selective filter of games like games played by +2700 players between 2000 and 2015 which include exchange-sacrifices but I do wonder if an openingbook will benefit of such filters. An openingbook should preferably contain a large spectrum of variations played in the openings.

Conclusion:
Yes, Chessbase is a good program and it is much easier to use than Chess Assistant which is maybe more efficient for users very intensively working with databases, publications, research and preparations. Only the look and feel of Chess Assistant is very different. Chessbase feels more familiar, it is like MS Office of Chess, while the Chess Assistant is more like the Star or Open Office: similar but just a little less familiar. You start up Chessbase and 90% of the features are very intuitively.

It is clear some imperfections are still to be solved so the work is not yet done. But for million of players in the world, this database-program is the only big investment you have to do - the return is for years. The small defects feel like you bought a new car and then you find out that the media-system doesn't allow you to play mp3's or the windows in the back are not electric. Fact is that CB15 offers many possibilities and in this release the developers really looked to how you can improve your chess, but every user still needs to check what is useful for him/ her.

Those who only use Fritz & co for their games, do they miss something? Yes the maintenance of the databases is much easier with DB, you have more options to filter and search in CB. You just work quicker with CB, you maintain a better overview. CB15 has many features which Fritz doesn't have so you have an edge compared to your opponent during the preparation.

Do you miss a lot? No not that much - maybe 20-30% of the features in Chessbase I miss in Fritz but the rest is for me not very useful. I was e.g. not waiting for a 3D-raytraced board. If I would ask 3 extensions of Fritz to bring it closer to CB (kind of intermediate solution between Fritz and Chessbase) then those are:
  • The more expanded search-features of Chessbase: not all of them but certainly some more
  • The usage of multiple (game-) windows in Fritz
  • And of course my ctrl-shift-r please
A Fritz "CB" with those features would be the birth of an extra product in the portfolio of Chessbase ... does somebody in Hamburg read this review....? Nobody interested in a "real" Chessbase Light (so not the junk which is limited to about 8000 games)?

The other way around, what would I like to have in CB15 which is now only in Fritz? The cr-analysis which is automatic now, but maybe I would prefer to define the parameters. And the database-keys (middlegame, tactic, endgame) which Fritz has, are also nice. But for the rest no, the Fritz-Gui is a Mercedes SL roadster, a luxurious two-seater racemachine – who can drive well, will get along with it quickly but it has some limitations. CB15 is a Mercedes GLS SUV: a very versatile car, which you can use for different purposes: sportive, multifaceted, chic; something you use for years without getting the feeling that you miss something.

HK5000

Wednesday, January 30, 2019

Chessbase 15 part 1

In the last decade many free chess-software appeared online but Chessbase remains head and shoulders above. Chessbase is more than just 1 program as the company has produced hundreds of chess-products. Obviously some players get lost in the jungle of choices.

Very often we see people purchasing stuff they don't need. Most salespeople will recommend their most expensive products and often inexperienced players trust them as they believe that it is safer to pay more. After the purchase very quickly they feel fooled and disappointed. At home they discover it is not possible to play games against the program. The engine is not one of the best ones in the world. 95% of the features are never used. After a couple of years the most recent and interesting games are lacking in the database. It is not surprising that those people never buy anything again from Chessbase and use the outdated program only sporadically anymore.

I very often discovered that doom-scenario during the presentations I gave last year about chess-software see e.g. the game preparation part 2. Almost everybody had some old version of Chessbase running on their portable with an obsolete database and an engine often hundreds of points weaker than the current champions. Eventually my presentation missed largely its purpose. I wanted to teach them how to prepare and analyze optimally by using the most up to date technology but that is impossible if you don't own the right software.

I did help my students in Mechelen as I had time in the months after the presentation to support their personal journey of updating their programs. I showed them where they could download interesting software for free legally or let them copy some databases which I created myself. Only when there was no alternative I demanded from my students to buy some software while always looking for the cheapest choice. In the end they were all having the best tools without having to buy the rather expensive Chessbase. A student has often very little money so I think it is logically to weigh the investment to its return.

Besides end of 2018 a new edition of Chessbase became available: Chessbase 15 with many new features. Therefore I think it is useful to see for whom this new edition can be interesting. It is a simple question but only after a lot of research and comparing it was possible to make a recommendation. As I don't possess Chessbase 15 and the manual doesn't say everything, I asked support from HK5000. He bought the program and already wrote some articles for this blog.

First we started making a choice of 2 reasonable attractive purchase-packages financially and functionally. However at the same time they should be sufficiently different so it would be possible to reveal each of its strong and weak points. It wasn't easy as we both looked differently to Chessbase. However each disadvantage is also an advantage as our differences brought a more balanced view of the software.


Option 1: Professional-package

Buying in 2019 of Chessbase 15 Starter Package (Big Database 2019 included for free), in 2020 of 1 year Online-Update Reference Database, in 2021 of 1 year Online-Update Reference Database. HK5000 makes an important remark about the updates. They are only interesting for people playing a lot of games (at least 50 standard games per year) and have a +2200 rating. So many users of Chessbase can avoid the costs of the updates. Budget for 3 years with the updates = 319,7 euro. Budget for 3 years without the updates = 199,9 euro.
Chessbase 15 starter Package (purchase in 2019)
1 year online-update reference database (purchase in 2020 and 2021)
Option 2: Amateur-package

Buying in 2019 of Houdini 6 (Fritz 15 GUI included for free) and Big Database 2019. I preferred Houdini instead of Komodo as it is 10 euros cheaper and developed by my compatriot. It is as strong as Komodo. Fritz is offered with the newer version 16 GUI and 10 euros cheaper but still I consider it less interesting. The new features of the GUI are for me not important and the engine Fritz plays 200 points weaker than Houdini. Budget for 3 years = 149,8 euro.
Houdini 6 (purchase in 2019)
Big Database 2019 (purchase in 2019)
Next I and HK5000 made a summary of the weak and strong points we discovered in both packages. Each feature was commented and evaluated. Score 5 = I like it very much and use it often, 1 = useless, 0 = absent. The final version after many iterations can be found in the 5 tables below (click on them to read the details easier).

From the personal scores of us both to the features of the 2 packages some conclusions can already be made.
1) We are giving to both packages a total score of 100 points and more. So each package offers us both a lot of interesting features.
2) In about 1/3 we give identical evaluations. So we are clearly different type of software-users which doesn't mean that we disagree about the value of the functionality.
3) It is also interesting to define how much extra points are gained by buying the professional package on top of the amateur-package. For me it are 20 extra points. For HK5000 it are 27 extra points. If I wouldn't possess the amateur-package today then I would definitely consider to buy the professional-package.

Above information summarizes very well the qualities of both packages which to some extend will permit to make a personal judgment and make the right purchase. Nevertheless I still think it is useful to make a general recommendation especially for people not having any experience with any chess-software. It is hard to visualize some features without ever having worked with it.

Personally I recommend the amateur-package for each player starting/ learning to work with premium chess-software. I believe it is useful first to work with the more simple Fritz Gui. Besides most beginners are mostly interested in the playing aspect and don't possess yet many databases.

The professional-package is very interesting for a small group of players.
- Trainers and students exchanging intensively study-material (I assume in Belgium only a few players match this description)
- Young players with the ambition of becoming FM/IM/GM (These are +2200 players younger than 30 years so maximally 30 players in Belgium.)
- Writers of club-magazines, books or webmasters (I expect only a few dozens in Belgium)
- Finally players with 1 or more years of experience with the amateur-package and who value the extra functionality of he professional package which can be used in the daily work at home. (It is hard to say how many people but I know very few players willing to work each week at least 1 hour with chess-software)

So if 50% of the club-players are interested to start with the amateur-package then likely not more than 5% will benefit from the professional package. I also have to add that Chessbase (the firm, not the product) inserts more and more features from the Fritz Gui into each new release of Chessbase (this time the product so not the brand). It becomes less and less interesting to still choose the gui. Of course they also know that most people won't buy both packages and commercially it is more interesting to convince people to buy the most expensive product.

Brabo and HK5000

Thursday, January 17, 2019

Desperado part 2

Since 2014 IMF claims China became the biggest economy in the world. However today there is still a lot of discussion about the interpretation of their figures see e.g. Is China's economy really the largest in the world? .  Obviously I don't want to go into details here as I only touch the topic to address the huge transformation China made in the last decades. It changed from an underdeveloped country to a superpower which causes today anxiety in some countries.

Not only economically China made enormous progress. We see in almost any domain that the Chinese have acquired an important position. This is also valid for chess. Last year China obtained for the first time double gold at an olympiad but few will still remember that China was only some decades ago not much bigger of importance than a small country like Belgium. Fide doesn't offer much support but after some painstaking hours of research I was able to create some remarkable statistics about China. My first graphic shows the evolution of the average elo of the top-10 players during the last 3 decades. To compare I used as reference Russia (only from 1993 onward as earlier fide categorized the Russian players under the Soviet-Union).
In my article about elo-inflation I wrote that many players are not aware about the fact that inflation is directly linked to the number of memberships. From above graphic we can clearly see this in the elo-evolution of the Chinese top-players compared with the Russian top-players. If we look at the rankings then this effect is magnified. In below graphic I show the evolution over time of the average-ranking in the world for the top 10-players.
We notice that Russia is still number 1 today but China is very close. Besides the strongest Chinese player has now a higher rating than the strongest Russian player. Of course I talk about super grandmaster Ding Liren whom as the first Chinese ever broke the 2800 elo-barrier a couple of months ago. He achieved that in a remarkable way about which several journalists have written. Liren didn't lose any game during a period of 15 months. Finally the counter of undefeated games stopped at 100 against often very strong opposition. Out of curiosity I checked my personal database to find my longest streak of consecutive undefeated games. The maximum I recorded was 37 in the year 2011 but it only gave me a TPR = 2300 elo so many of my opponents were rather weak.

Unfortunately this splendid performance of Liren also created again jealously among some players as the reaction of Sergey Tiviakov at Chessbase proofed. I educate my children not to cheer in the proximity of the defeated opponent after winning a game so it wasn't very tactful of Sergey. Anyway the loss of Ding Liren  is quite special worth to investigate closer. In that game there occurred a very special desperado of the queen. In an earlier article I talked about the desperado-pawn in which a doomed pawn makes a last move just to win a tempo. In a queen-desperado we see a different dynamic. The queen is threatened and can be saved. However instead of that the player chooses to play the threatened queen to a square where it still can be captured. 
In above position black's queen is threatened by the royal fork Ne6+. However instead of saving the queen, black plays a desperado-move with it as white's queen is also hanging.

On chess.com I often solve some tactical exercises. The most difficult ones are sometimes very special positions. One of them I remember involved the theme of the queen-desperado. Below position is extracted from the collection of exercises I made on that site. In the solution there pops up 1 queen-desperado but you can even see 3 consecutive queen-desperados in the temptation.
A last example of the queen-desperado I encountered while analyzing my last Belgian interclub-game of previous season. Without a computer it is impossible to discover it as the desperado only appears in some very complicated tactical line.
In each example of the queen-desperado we see that both queens are hanging. This doesn't look surprising to me as there are likely very few other situations in which such drastic move is good. If a reader knows such different situation of a queen-desparado so without hanging queens then I am curious to learn about.

Brabo