Wednesday, March 27, 2019

Free of charge part 2

Free of charge, it still exists in this era of the internet 3.0 or 4.0 (or which version are we using nowadays?).

You can still find quality however more and more on payment, provided in different formats (see e.g the old sites, of which the historical one of Olimpiu Orcan is a nice example: https://www.patreon.com/urcan). Another famous example from the stone age of the internet, www.chesscafe.com, which meanwhile can only be accessed for a fee. Before it was a nice collection of articles written by professionals about many aspects of the game, but unfortunately now not anymore available for free. Fortunately I downloaded many articles so I can still access them today.

I am not going to discuss free databases here, they exist in all different sizes and qualities sometimes also including analysis. I also will not talk about free chess-software, also of that you can find many of them and I don't mean hacks but rather good software like arena, scid, and other tools/ apps to work with like making gifs of games (popular at twitter).

In this article I want to give a summary of where you can find on the internet good pdfs with chess-content, just to download legally, so no torrent sites (sooner or later just giving you a virus). I also limit myself to non-interactive pdfs, The advantage of a collection of books on pdf-format is that you can take dozens of them on USB-strick - very handy for holidays: no trunk full of heavy books needed.

About the format I also want to add that pdf is fine but dejavu (djvu) is also popular, as it compress the files a lot. And you can find as much chessbooks in dejavu as in pdf.

The first place to look is naturally google, but a more direct and fully legal place to access chess-documentation is the internet archive (https://archive.org/), where you can find a number of interesting pdf books and magazines. It is not a lot but a good start. Anybody interested in American chess-news of 50-60 years ago can download some years of Chess Review. Very nice.

A second address is http://www.chesszone.org/lib/lib.html, where you can download a limited number of pdfs and djvus. No recent publications but it looks all copyright free. The most interesting subcategory seems to be the collection of games, with some less famous work.

A third location is probably less correct : scribd.com. They show some books online but I doubt this is all legal.

Only now you should google the titles of the book so type (or just "schaak", "chess", "ajedrez", "xadrez", "schach", "skak",…) + "pdf" (or djvu) which will give you some hits, likely some fragments of the books will be given free of charge. An exception is https://www.pdfdrive.com/chess-books.html, where you can "just" download some pdfs. Or the simple search "schaak pdf" generates the complete "Ome Jan leert zijn neefje schaken".

Related to the fragments: New in Chess has the nice habit to show a number of pages free of charge of each of their books online - a nice balance between free and paid content. Also QualityChess puts similar extracts online - definitely worth a visit. It is not a goldmine, but some scrolling can let you discover some stuff.

Belgian clubs with a lot of content online: here we have to mention CREB (www.creb.be tabs archives and publications) as somehow the historical database of all what happened about chess in Brussels, Brabant and Belgium. Thanks to a.o. Etienne Cornil you can find a load of interesting information about chess in Belgium for everybody. The other important historical site about Belgium is of course still Belgian Chess History (http://www.belgianchesshistory.be/) - no pdfs but almost all games of Belgian players played in important tournaments/ matches from before the year 2000. But I digress so I go back to the pdfs.

KBSB-VSF members also get via VSF bi-weekly the VSF-magazine online in the mailbox. It is actual news so easy and interesting. No need to search for it. The magazine is a good mix between historical information, game-analysis, announcements and tournament-results. The KGSRL also puts their club-magazine online for many years already, but the games are mostly from grandmasters so less relevant for the clubplayer. Clubs like Jean Jaures, Veurne and Landegem have published some editions online but unfortunately didn't maintain that service. I probably still forget some clubs but this article is not an actual scan of what each club does in Belgium so just summarizes what I have detected in the past online.

Thanks to Rokade Westerlo I discovered http://www.chessarch.com/archive/articles.shtml, but that is more for chess-historians.

The Netherlands also have a good portion of online content. Maybe the clubmagazine with the best layout is the one of Rotterdam Charlois Europoort (http://www.charloiseuropoort.nl/triomfator/). It is a pity no more recent numbers are added to the archive.

Interesting and still today relevant is the newsletter of the Max Euwe center. It is not clear what the focus is in the articles but many lovely stories can be found (not about Euwe)(http://www.maxeuwe.nl/index.php/nieuwsbrief).

Delft (http://www.delftseschaaksite.nl/download/axioma32-web.pdf) also puts their clubmagazine online. A little advice I can give is that many clubs only show the most recent ones (e.g. the last 5) of their clubmagazine but often older ones are still on the server. You can find them by just changing the edition-number (in the case of Delft see link just change 32 to 31 or 33). They remove analysis of games from the online-magazine but you can't get everything for free always. It are just a few samples but Rotterdam I have known from the past, Delft I found immediately with some simple google-search. It also allowed me to find Groningen, Assendelft, Winterswijk...

For lovers of endgames and studies there is http://www.arves.org/arves/index.php/en/dutch/43-indexknsb). Arves allows you to download lots of material.

Netherlands has also some good sites for general information and after some searching also historical documentation: www.delpher.nl and www.gahetna.nl are 2 sits with a lot of content.

Not on the Internet but something which gets a lot of attention in Netherlands are jubilee-books about the history of a local chess-club - many small and big ones have already created one with mixed results (qua format and content - we are in the Netherlands - often with miserable language). But again I digress...

Swiss: one of the places I like to return is the address of the Swiss chessmagazine (SSZ: http://www.swisschess.ch/ssz-archiv.html): years of fun reading (and good for your German, French and even Italian) about 8 editions per year. Swiss is a country with relatively independent federations (like big provinces) each having their own organization - which generates some interesting material. And in recent years Swiss chess has been flourishing - first due to an injection by Kortchnoi, Nemet, Milov and Gallagher, but now also by their own youth (Studer, Jenni, Georgiadis and Bogner).

Germany: the magazine Schach has all their lessons online (more than 100 numbers) see (https://www.schuenemann-verlag.de/schach-magazin/index.php?include=3000). This looks useful for teachers. The section is historically grown from a mix of all kind of lessons to a more structured and thematic guide. This is educational for any player. Besides you can also subscribe to the newsletter of the German chessfederation: (https://www.schachbund.de/allgemein-newsletter.html), but I didn't do that (already enough mail) so I can't give much feedback about that.

For the magazine of the club of Roeselare De Torrewachters I am busy creating a series about chess in America. I found some good address of the American federations and clubs which contain some interesting files about local and federal level. General rule: the most populated regions have also the best websites to download material.

And I can plagiarize myself (for California): " A summary of the chess-activities can not be done without mentioning the Mechanics’ Institute (http://www.chessclub.org/news.php?n=736). That chessclub is the oldest of the USA (The Kolty chess Club in Campbell is the second oldest of that region) and has in the past organized many tournaments (also for computers). The site - and surely the Chess Room History is definitely worth a visit. But a real goldmine is www.chessdryad.com, with a wealth of old magazines, for free in pdf to download. The best of club- or tournamentreports are that you can replay some games of strong, not grandmasters which otherwise you can't find. Those games are often interesting, well analyzed and are discussing some interesting openinglines and endgames - the reason why there were chosen for publication. Briefly, it is something else than Vlaanderen Schaakt Digitaal.

In this summary I have focused myself on getting easy and quick information taking into account language, relevance, copyright and easy access. People willing to try other languages (Spanish, German,...) should try google ("schach pdf/djvu" or "ajedrez pdf/djvu" which will generate a lot of interesting stuff, not always copyright conform or free of virus I assume). I found a Spanish site with a lot of Spanish and English content but likely not fulfilling all copyright-obligations so up to the reader to investigate.

I didn't want to discuss databases as there are many sources but I make one exception: https://database.lichess.org/ which has already 600 million (!) games, played on their platform, so mainly blitz. But a blitzgame between grandmasters is still a good game so...

Another site which I found at the very last minute was the one of the chessclub of Tessenderlo (http://www.looiseschaak.be/), on which you can find a lot of club-information - the site is maintained already for more than 20 years - quite an accomplishment of the webmaster for sure.

HK5000

Thursday, March 14, 2019

Lc0 vs Stockfish superfinal TCEC season 14

1. Pre-match expectations of Leela.

Beforehand SF was considered the clear favorite. There are a number of reasons for that: the rating of Leela (Lc0 v0.19) at Ccrl40 was not great: the engine wasn't mentioned in the top-100 (version v0.20 was later during the match ranked at the 45th place). Second, Lc0 performs weakly using classical hardware compared with the traditional engines. That is no surprise as Lc0 is built for fast chips of graphical cards (GPU's), instead of CPU's. On a normal CPU Leela can't obtain maximum strength just like Fritz3 initially couldn't perform well as it was lacking RAM for the hash-tables. See for more information in the article of Frederik Friedel at Chessbase (the adventure of chess programming part 3). Finally I also thought the engine was tactically not mature yet which we saw in the previous tcec competitions and based upon my own usage of the engine. So a match against the undisputed leader of ccrl40....

2. Summary of the match

At the beginning SF was clearly the better engine, twice it took a lead. First 2-0 after only 10 games, but after game 13 the score was again tied. After that SF won 3 games on a row!, but again this didn't last very long: after game 29 Leela equalized the score. After that Leela took the lead: games 49 and 53 were won by Leela. It became an exciting match with switching leaders: we hadn't seen this for a longtime between engines. Maybe it didn't happen anymore since the Braingames "candidates-match" between Fritz en Junior in Cadaques 2001, when the engines were playing for a match against Kramnik. Junior got in that match almost 5-0 for free, after only 5 games but Fritz equalized in the second half of the match (24 games in total) and won the play-off with 2-0.

However the lost games were very painful for Leela - it reminded me to the match Botvinnik-Bronstein: Bronstein played ingenious chess, used new concepts, played very differently than Botvinnik. Botvinnik tried to reach draws by adjourning games and had a lot of trouble to score some wins. Leela wasted several half points by lacking tactical awareness. A good example is the 20th game, in which Leela plays the losing move (39...Rb6-d6) with an evaluation of 0.26, but SF answers with 40.Rg3+ and immediately shows +8.56 - probably the "boom" of the match. Maybe it wasn't a draw (SF was already giving +2,5 to itself), but more blunders would occur. In the next game it happened again: in an equal position Leela blunders once more and SF hits back immediately with taking at f2 (-4.46). Game 66 again. A very weird loss was the 85th game, the last decisive game of the match: Leela still believed it was a draw (overvaluing a far advanced free a-pawn) while SF considered the position for white already for a longtime as totally hopeless. When Leela realized it then the evaluation plumbed to -14.28 (SF was given already mate in 41…). In a very rare case Leela missed a certain win (65th game) in which SF (using 6-men tbs) was already 100% sure of the loss. This was the consequence of the low search-depth and less extensive usage of tbs. At game 80 the score was tied again.

3. Learnt lessons about openings, playing-style and other aspects

Besides the impressive performances in the middlegame (Leela) and endgame (Stockfish), there were also a number of important learnt lessons about the openings.
A first highlight was game 11, in which Leela gradually increases the white advantage from the opening (French). This was very impressive, especially as the evaluation of Leela was more than 10-20 moves ahead to the one of SF - it looked like grandmaster against amateur - only, the amateur has the strength of a super-grandmaster. Beside the evaluation of Leela is something you need to take with a grain of salt: in the first game the evaluation of Leela at move 104 jumps up to 2.65, while SF sees no problems. The same happens in game 9 : suddenly Leela shows an evaluation of +2,24 when it can exchange queens and obtain a bishop-endgame with a free a-pawn - SF again evaluates the position as fully equal and the game ends in a draw. Also in the 85th game: it seems Leela puts to much trust into the free a-pawns? There are many other examples of the too optimistic evaluation.

As stated before, the variance in the opening is good, but the engines seem capable of turning the most sharpest openings into forced sequences leading to boring drawn endgames. Fortunately some nice middlegames were played, but when SF after e.g. 1.e4 c5 2.Nf3 d6 3.d4 cxd4 4.Nxd4 Nf6 5.Nc3 a6 6.Be3 e6 7.a3 b5 8.g4 Bb7 9.Bg2 h5 10.g5 Ng4 11.Bc1 Qb6 with white and black gives very quickly 0.00 then such perfect play looks not exciting anymore. The same scenario in the 5th and 6th game: SF makes very quick draws in the Kings-Indian.

Leela couldn't do much with the kings-gambit and lost - SF could just hold the draw with white. Although a couple of games can't define the correctness of an opening, it is rather symptomatic that it is the kings-gambit leading to troubles for white. It seems the romantic opening isn't more today than just a surprise-weapon? As earlier written, the French game (game 11.1) was a highlight for Leela in the match - this time SF couldn't show the same quality for white. The French opening seems something Leela knows best as also in game 35 we saw SF having big troubles after the opening.

The win of Leela in the Nimzo-Indian was rather related to some small errors in the middlegame of SF so not due to the opening. In game 16 SF practically destroys the Pirc in the opening. The Pirc had a rough time in this match: in game 55 SF almost didn't make the draw. It became the longest game in the TCEC-history: 264 moves before Leela agrees with the draw. In game 71 we see the Pirc creating another dramatic turnaround. Leela has a stranglehold but can't break the defense despite dominating the whole board. White has everything - black even has to evacuate the king to the a-column - but it is not enough.

Generally we see that the smaller openings don't stand very well the test, nor do the sidelines of the big openings: it is striking that the white advantage is only disappearing after move 25. An example of this is game 23.

A great win is scored by Leela with a white stonewall in game 25. A Philidor/ Lion-opening in game 27 is annihilated by Leela - one of the seldom moments in which Leela manages to get SF away from the 0.00 evaluation and wins deservedly.

Games 59 & 60 show what "sharp" lines for humans mean for engines: the Sicilian Dragon creates 2 short draws. Even the Spanish is used but survives a Leela evaluation of 5.76 in game 75. The end of the match looked like football, in which team A has a lot of ball-control, but team B scores twice via the counter in the final quarter.

4. What could've been better in the match set-up?

Contrary to earlier tcec super-finals, this match of 100 games seemed "too short". The engines were very close in strength, and after 70 games there was still only a gap of 1 point. After 100 games this is also the final difference: 50,5 - 49,5. Also the openings could've been chosen a bit better. In most cases positions were chosen after 5 moves which were more or less neutral, mixed up with some deeper lines (which were randomly chosen from the opening-book created by Jeroen Noomen). Those deeper lines were not always creating interesting middle-games. The problem (for humans) with fun openings (like Marshall-gambit, Sic Dragon, Botvinnik-gambit, kings-gambit, Albin countergambit, Sveshnikov, Sämisch KID, Sevilla-variant Grünfeld, …) are that they equalize quickly ( due to a too forced mainline) or almost always give a win/ loss for white/black (as one side has a too big advantage). Some openings are complex for humans but that is not necessary also for engines. Nonetheless I agree with the critics to use more starting-positions from grandmaster-games. this would improve the relevance and use (for the practical player) more. Maybe this is something for the superfinal of season 15?

5. Other things which we can improve?

SF had the advantage of using 6-men tablebases (tbs), Leela only worked with 5-men. That difference for sure meant for 1 game the difference between win and draw, and had - with equal weapons- given a tie so 50-50 as final score. Now engines can already use in the opening those endgames tbs, so this is important in a match. Probably over 10 years we will have 8-men tbs (so having a solution for all rook-endgames with 2 pawns each - great !) so this aspect will become in the future even more important. Or maybe we should do the opposite so forbid the engines using tbs at all?

6. Conclusion: are we close to perfect chess?

Positionally Leela is close as to beat SF this needs a very high level of play. Tactically SF is still (a lot) stronger. The great search-depth avoids missing any tactical traps. Also this allows SF to defend some very difficult positions. One aspect of the development of Alpha Zero and Leela reminds me of what professor Jonathan Schaeffer experienced when developing his prefect playing checkers-engine Chinook (by the way if you want to read a beautiful and emotional story about the first engine beating a reigning world-champion then I recommend very much "One Jump Ahead"). It is something what Schaeffers team and also recently the team of Demis Hassabis (Deep Mind and Alpha Zero) noticed: further development leads to an increase of the draws (an indication that chess is a draw when played perfectly, or that there is a limit to further improvement). That effect can be partly explained by the fact that "a draw is a draw" for an engine. In other words: the simple evaluation of 0.00 should be added with other parameters as otherwise the first move in the list leading to 0.00 will be played (see for that behavior to an article of Tim Krabbé about pealing an orange in Alaska ("morons"). An intelligent add-on would be that the engine selects the move bringing the most chance to errors for the opponent (let us not consider contempt). This can be a line with many forced moves or avoiding exchanges. I guess some modern engines already use such parameters doing something like that but it is not yet working perfect.

The advantage of Leela is that "the engine" can now do the development - sooner or later the development of Stockfish (despite all tests the engine plays against itself eventual with a self-learning function) will stop when reaching the limits of human programming. Leela has the absolute minimum needed code to seek maximum results. One of the Leela developers wrote on his blog that if a developer of a classical engine (e.g. SF) takes a holiday for a week then the engine remains the same while with Leela after a week it became by itself again a bit stronger. It appears Komodo already hit the ceiling: at ccrl40 release 11.3 has 3 points more than release 12 and 12 points more than release 12.3. And also the MCTS-version of Komodo is getting close to the classical one. It looks like the Americans have reached their tipping point.

But as I have said: the evaluation of Leela - contrary to SF - is not fireproof: never did I see so many positions with a "winning" evaluation (+2, +3, +4, or even +5 and more…) not transformed to a win. The winning line is often so small for engines that one small deviation is sufficient to lose the advantage. Leela is not yet able to avoid those mistakes - so this looks like us humans playing chess :-) However for the practical player, this disadvantage can become an advantage, and Leela is definitely a good addition for new ideas (plans), or to find practical chances in non-tbs endgames (something which SF will rather evaluate as 0.00 without giving a view about practical chances).

Epilogue:

Some critics have pointed to the openings as not well chosen, so additionally a rapidmatch was created without any pre-selected openings. Leela won this one with 56-44 (see e.g. s14 bonus match leela stockfish. That is a very large margin especially as tactics are normally much more dominant in rapid-play. The next season will have to answer the open questions.

HK5000

Thursday, March 7, 2019

The Leningrad

I bought during the first years of my chess-career from each (big) opening in my repertoire a book. The Spanish, the French, the Caro-Kann, the Aljechin, the Pirc and the Dutch (stonewall) were the openings. It costed me about 3000 Belgian francs (approximately 75$) as the euro was not existing yet at that time. For a student with almost no money, we talk about the beginning of the 90's, this was a serious investment.

Later I never wanted to buy any new opening-books. The rise of the engines and databases allowed me to create my own opening-analysis. Besides if I would like to keep my repertoire up to date by buying always the newest opening-books then this hobby would become very expensive. Also I noticed that many books are repeating a lot of what has been written before so you get less and less new interesting information. Finally I experienced a lack of motivation to study all the theory from opening-books. Very often an opening-book was only used as a reference work so it was never studied properly.

Only beginning of last year I made an exception. After more than 20 years I bought another opening-book: The Leningrad Dutch. I had decided to add this opening to my repertoire (see why previous article) but I realized immediately this can't be done quickly without external help. The Leningrad has a labyrinth of different variations. It is completely impractical to check 10.000 games in the databases to build an overview of the opening. So it is necessary to ask an expert for advice, somebody having played the opening for many years. The easiest and cheapest way to do that is to buy a book which I did. There are a few options but eventually I chose the book written by the Ukrainian grandmaster Vladimir Malaniuk. He was probably the biggest pioneer as he played the opening during more than 3 decades against other grandmasters. I use the word "was" as he died in 2017 (some sources say he was assassinated).
Meanwhile I already read the book twice through. In my 2 recent standard-games with the Leningrad I was able to use immediately some recommendations from the book. However when I investigated the correctness of those ideas later at home then I discovered that the author is often too optimistic for black in his evaluations. I start the review with a first position which I had on the board see again my previous article.
The Leningrad Dutch page 123 variant 1
The author tells us that white should try to equalize with accurate play. However after the novelty exf5 followed up with Qc2 my engines show a clear advantage for white.
The Leningrad Dutch page 123 variant 2
We have moved 2 moves in the same game. This position is reached in the book via a different sequence of moves than in the game. The author evaluates the position as equal while the engine again shows a clear advantage for white after Nce2. Fortunately my opponent missed that idea in our game.

A fragment of another Leningrad played in one of my standard-games, was already published in the article desperado part 2. Again we see that the author is too optimistic.
The Leningrad Dutch page 269
White can't equalize and the author continues to prove this with the weaker 14.Rf3. However I already showed by using the engines that 14.Qg4 is still sufficient for equality.

I have bought the book mainly for the Leningrad but I also had a look to what the author tells us about the anti-Dutch systems. I was curious if the author had noticed a quite recent development in the Staunton-gambit.
The Leningrad Dutch pages 57 and 62
In the book is written that black's extra pawn compensates the small lack in development. However 14.Nd4 is completely ignored and already known from practice since 2009. White scored already 4 important victories with it in mastergames. The engine is very positive about the move for white.

I wasn't able to check most of the analysis of the book but I fear that I only revealed the top of the iceberg. It again confirms my perception that most books are written for the casual chess-player but don't withstand scrutiny.

Fortunately I didn't have high expectations in advance so I wasn't disappointed. I wanted a skeleton of the Leningrad and that is what the books supplies. The author has built a repertoire for black by showing for each of the important lines of white how he would respond to it. You could say that you have the shell of a house but all the rest still needs to done. This book is sufficient to start playing the opening after you read it but if you want to regularly play the opening then you better do a lot of additional analysis.

Finally I also want to add that I can remember a lot of analysis from the book but I also forget still a lot. I started to read the book for a third time but that will still not be enough to remember all the sidelines. Nevertheless especially in the Leningrad it is very useful to also remember what needs to be done in less frequently played variations. Some suggest to type all the lines from the book to a pgn-file so you can practice with e.g. chess position trainer. This is a Herculean labor which I don't want to do. Only the mainlines is for me the maximum. A recent new initiative is chessable. They do the job of digitizing for you. It is not free but neither much more expensive than a book which you can economize. Unfortunately the site offers today only a limited choice of books. The Leningrad Dutch is not included in their base. I have no experience with Chessable but looking at their increasing numbers of subscriptions, this could become a big player in the future of sharing chess-information.

Brabo