Houdini 3 Pro vs Deep Fritz 13

My opinion: Engines are fine to use in supplement to opening theory, but should not replace it (maybe an analogy to vitamin supplements?). I consistently use Houdini/ICCF databases for looking for improvements in my games, although these are typically past move 10.

The problem with forming a repretoire from move #1 is the disconnect from the player who actually has to play the game. Any engine (to my knowledge) does not value a move based on practical winning chances, which is sometimes what a human player has to aim for. A line where a human player (this is an extreme example!) has to play 10 precise moves just to not be losing is not as practical as 3 move line which results in a slighlty worse position, but the engine will prefer the former. With regards to openings specifically, engines can be really fickle. Even if we are talking about 40 depth (20 moves by each side), there is opening theory that goes beyond this. Pawn structures can also be an issue as the engine may think the move choice is fine for the next dozen moves, but in the endgame they may be dead in the water.

I like plutonia's shorter response about how openings must be understood. At lower levels you are more likely to see deviations from theory and thus must know the fundamentals of the opening you are playing.

For that amount of money I want also a house on a Greek Island!

I think what you are doing is great!! Wish there were more people like you. Especially in OTB tournaments. If I may ask: could you tell me about the machine you are using?

A single problem was identified and was corrected 10 days ago. It did not generate "wild evaluation fluctuations", as you incorrectly claim.

What exactly is your problem, guy? Why do you spread this trashtalk about the strongest chess engine ever? Do you feel empowered by the "IM" label to spread nonsense about other people's work?

Personally I do not buy or sell anything in here, and certainly I have no intention to bash an engine who lost against an ancient Ivanhoe compile, which incidentally had completely failed to hold hiw own against Houdini 1.5.

I have not tested H3 personally, nor I intend to do so. I use H 1.5 in parallel with an older Critter release (1.4a) for my correspondence games, they work more than fine for me, and cost nothing.

There is no sin to admit bugs in your software, unless you are in some hurry to collect some cash. Are you?

I'm not here to sell any software either, you'll notice that I never use this forum for advertising.

On the other hand, I will not let any random person with an "IM" label talk rubbish about the engine.

You don't use Houdini 3 and don't seem to know a lot about it, yet you feel entitled to make comments about "many bugs", "wildly fluctuating evaluations", "several lesser bugs" and even give general recommendations about it based on inaccurate facts.

Please stop this nonsense. If you're happy with the free Houdini 1.5, all the better. Reason the more to not spit its author in the face, as you're doing now.

Most of my computing power unfortunately is in my graphics system, and I'm currently unaware of a CUDA solution (a way to use graphic card for computation in these programs). The GPU is two GTX-480s in SLI, CPU is i7-950, and all the memory is corsair/solid state, corsair dominator RAM (only 6 GB though), and a Corsair 1200AX PSu. MB is Rampage 3 Extreme. I'm afraid the only significant stat here is the CPU and and solid state HD. If anyone knows how to use a GPU to help drive a chess engine, please share! PS. my current strategy is to assign 4 CPU cores and analyze 4 candidate moves.

To everyone else, I am in absolute agreement that engines cannot produce a repertoire in a vacuum. I have a couple dozen opening books by various authors, and it's important to read these so that the author can explain the ideas behind the openings. It's also important to study full games that use those openings so you can get a feel for how they are to end up. I am just using the engine as a referee to narrow down alternative lines to a single best choice. 

That being said, I'm actually a little concerned about how Houdini is performing. I have only had the program for one day now so I don't have much to comment on, it's just that it seems to have trouble picking a definitive winner. When DF 13 was analyzing black's response to the Zukertort it hit a depth of 33 in just a couple hours and picked a definitive winner. However, Houdini has been analyzing it for 21 hours now, is only at a depth of 32, and has three out of four of its candidates in a complete tie. While it was approaching a depth of 30 the candidate moves had different moves, but deeper, it seems to find an equal worth in each. 

I know that's a very isolated example (and a bad one since the Zukertort is so open-ended). I should be able to report back in a couple weeks with some more useful data. It is interesting though that Houdini takes longer to reach an "equal depth" as DF 13.

And one more quick word about this project. I'm just doing this as a free public service (as well as to improve my own understand of course). I will never be running any ads. Right now I only have a reseller account, but I am in the process of enlisting in the Navy; once I get more money set aside I will upgrade to a dedicated server. However, it's going to be at least a few months before this is helpful to anyone, since as I'm sure you all know, trying to exhaust opening trees is not a small task.

However, it should be easy to navigate through since it's on a website. If anyone has suggestions about how I should better organize this project, please make a suggestion now because it will be harder after I invest a couple decades building this, thanks. 

Project location: Worldthought Forums - Chess.

For those with older hardware, yes. But the glitch affects output analysis for the most part. The playing strength isn't affected too much. I'm actually using a Core 2 Quad, and Houdini 3 is demolishing Houdini 1.5a. (Which backs up the mail received from Houdart.)

So, you have not tested and do not intend to test Houdini 3, yet you make major claims about the glitch, which goes straight against all the people that used it on older hardware without any real problems.

I take your sarcasm as offensive.

Why should I bother testing H3 when this is done by various experienced people in multiple dedicated forums? Factly, these people are the ones who spotted H3 bugs, including the aforementioned one.

Why the "perfect" H3 lost a match against an old engine who could not hold a candle against H 1.5? It's not a matter of bad settings- the man who made the test is very experienced.

For any serious correspondence player, reliability is an absolute premium over speed or "objective" strength.

I take your comment as ignorant.

@FEDTEL: Yes, I am Houdini's author.

@pfren: Your interventions are malicious and uninformed. There are no known problems with Houdini 3 other than the one that was reported and corrected about 10 days ago, and which was a fairly innocent issue that impacted a minority of the users.

Houdini 3 is not "perfect" - that is your description. It is about 70 ELo stronger than Houdini 1.5a, which means that it will still lose the occasional game and even the occasional match. But over-all there are no grounds for your negative comments.

Because Houdini 3 lost a single game to a lesser engine, that mean's it's worthless? Then all other engines are even more doomed - they lose or draw more than the majority of the time against Houdini 3. You're saying, Houdini 3 lost one match out of many, that means it's unreliable; and how incredibly reliable is an engine that loses all the time against Houdini? Even less reliable.

You are the ignorant one. You don't know the severity of the glitch and you spew out ignorant comments. Yes, let's bash the best chess engine because it had a MINOR glitch that was fixed TEN DAYS AGO.

 

Ahh, I see. He sure is persistent. (And rude.)

I am having trouble interpreting how/why Houdini is taking so much longer to score a high depth on Let's Check. It is making me wait about 15 hours to score 33 depth on Houdini when it would just take a few hours on Fritz 13.

My best guess is that Fritz' self-proclaimed depth ratings are a bit inflated.

Correct if I'm wrong: the depth/knodes etc cannot be compared head to head across different versions of the same engine, let alone different engines. (What I read.)

"Let's Check" takes the three highest depths regardless of which engine and ranks them so that they can be referenced by anyone. So maybe what you are pointing out is a flaw in the system, but this works for Shredder, Rybka, Houdini, Fritz, Stockfish, or whatever else has access to "Let's Check".

I was just wondering why DF 13 thinks it is going deeper than Houdini (in significantly less time), especially since Houdini takes up more CPU resources. I just assume that DF 13 branches out in a more limited way when it does its analysis, which enables it to post higher depth ratings (total speculation I don't really know). 

I think it's fun to try and get on the top three for early and major positions; I was just noticing that it takes longer with Houdini. No idea why.

I am very confused now about how many candidate lines I should be feeding the engine.

After 24 hours of having it evaluate black's response to the Zukertort with 4 active cores and 4 candidate moves, it ended up with 1.Nf6 as the best choice at 33-depth.

As a test to see if I could even get to 34-depth, I assigned 1...Nf6 as the only candidate move by pruning the other three lines, and using 6 active cores instead of 4. It has been on 33-depth all day long for 1...Nf6, and now it suddenly changed its assigned candidate line! (to 1...e6 - Queen's Gambit Invitation). This also happens to be what Fritz ended up with at 33-depth but I am thorougly confused at how it overrode the one it was working on all day and suddenly came up with a new one (after exactly 19.5 hours). Going to let it run another 12 hours or so to watch what happens.

I know many are against the method of trying to create a repertoire in this fashion, but if you were in favor of this method, how many candidate moves would you have it examine? I am intrigued that if I only gave it one it is capable of changing its mind much later on, but obviously it's not productive if I have to wait almost a full day for it to do this. So maybe I should continue letting it examine four. 

Update: @22 hours it switched to 1...c5 (Sicilian Invitation), changing the whole line! Then at 22.5 hours it actually hit 34 depth, sticking with the Sicilian invitation. At 32 hours it still is at 34 depth and 1...c5.

I just wanted to thank everyone (even pfren) for the input.

I have been realizing (like everyone was saying) just how unreliable engines are for the very first moves. I have been analyzing 1.e4 for a few days now and it has the French towering above all the other openings by a wide margin. I know the French is a sound response, but you can tell that the computer likes it so much because it is preparing itself as it would against a supercomputer. It reminds me of Kasparov's games against Deep Blue when he would just build a little French-like fortress to shield himself against heavy brute force calculations.

So for this project I decided just to go with the three openings that interest me the most, even though they are not favored by computer analysis. For black I am going to try the Dutch Leningrad for all first moves by white except 1.e4, in which case I will use Alekhine's. The idea behind these openings mainly (especially the Leningrad) is to overwhelm white with complexity instead of trying to equalize the game. "If both players are confused this is a moral vicory to Black, who started off with the worse position" (2004, Neil McDonald).This kind of complexity, however, would no doubt strongly benefit from engine analysis; so I will continue this project for the next 20 years or so, just restricted to these two systems.

I am still not fully decided on what to do for white (if anyone wants to make a suggestion) but I am pretty sure I will go with the English. Reason being is that I feel uncomfortable whenever I play against it (also when I play it myself). It is a positional nightmare requiring many subtle move-order dependent operations, and so seems like a perfect project for an engine-assisted repertoire.

However, the earlier in the line, the less I will consider the engine's input. But by about the 4th move, I will strongly consider the engine's results. Earlier on, I have to put more emphasis on how games with these lines actually turned up, and what grandmasters have to say about them. I ordered a few books on the Leningrad so I will use that as a starting point (while I spend 2+ weeks analyzing the first move for white).

I am still interested at whatever the computer thinks is the best move for all the first positions, seems like good data even though I don't know why. So I will still be stubbornly trying to hit record depth for all primary early moves. I may set aside this information in a section like "What the computer would do?" Anyway, I would be saving people time (that have "Let's Check access) who want to see results for early position—so they will be able to see 37-ply results in a second rather than spend week(s) calculating one move.

It increasingly more common these days. The hex-core i7-3930k is just under $600 which isn't too bad. Anything past that starts to become quite expensive and some are made only for servers. Most computers have 2-cores at minimum and many desktops are quad-core.

The i7's have a virtual core for each physical core, which mean that a quad core really has 8 logical cores. The i7-980x and 990x CPUs have 6 physical cores, which means they have a total of 12 logical cores. 

Unfortunately, I went cheap on my cpu and only got an i7-950. I originally built my system for gaming and CUDA programming (which uses graphics for calculating power) but apparently cranking out floating points isn't that helpful for chess engines, so none of the chess programs use CUDA.

Deep Fritz 13 offers the opportunity to use up to 8 cores, but since I went a little cheap on my CPU, I only use 4 or 6 depending on how much I will be multitasking. I technically could use 8 cores but so far have been nervous to do this because I was trying to avoid a crash.

And as advice to anyone wanting to buy a system for this (besides getting a flagship CPU with 12 logical cores) would be to get a lot of RAM (in the ballpark of 12GB). I have always been an advocate of low capacity but very fast and high quality RAM, but now I am feeling my stupidity there because I have to constantly stare at my RAM widget to avoid hitting 100%. This means I have to watch how many windows I have open while browsing etc. Right now my RAM is hovering at about 85% and I am only using 4 cores on Deep Fritz. I am 72 hours into a calculation though and *I think* that more RAM resources get used when you are in 34+ ply.

As a datapoint, I am using 3-channel Corsair Dominator RAM (2G X 3). It has low latencies and a high clock rate, but the lack of capacity is really killing me here. And to repeat since my post was a bit long and dry, my overkill-expensive graphics system is completely worthless to me for this, and I would have been better off getting a CPU with 12 logical cores. 

However, if you don't use more cores than your computer can chew, you should always be able to compete with the top calculation for any position in "Let's Check", regardless of computer strength, if you are willing to wait enough time. That and if you don't accidentally touch your mouse wheel with Fritz open because that will reset the calculation...

I just wanted to share an amazing development, which is also an example of a previous question that has still gone unanswered.

I began analyzing the first move for white as a showdown between 1.e4 and 1.d4 (using 6 cores). I assumed that none of the other positions would be able to compete with these two so I cut out all the other candidate moves.

8 hours later, I wake up and 1.Nf3 replaced 1.e4 as a candidate move!! Now 1.Nf3 is tied with 1.d4 (with the same score). 

First of all, wow. I haven't been giving much credit to the computer analysis of these very early lines, but if 1.Nf3 keeps performing strong a few days deep into this calculation, then I will be considering it as a new main line (instead of 1.c4 or 1.d4). I am very sick of the Queen's Gambit, and so am rooting for the Zukertort here (which I always assumed was inferior).

Anyway, this brings up the question of how the engine is working or what is going on here? I had it analyze two candidate moves and it just replaced it? Does this mean that it is unecessary to analyze three or 4 candidates because the best one will surface? How did it surface?? Is it related that I was using 6 cores for only 2 candidate moves?

Hopefully Robert can shed some light on that one. Would be happy to hear anyone else's ideas though.