Best CPU for chess engine game analysis

bfn

Hi,

I'm thinking of upgrading/replacing my Windows 7 PC with a first generation quad core Intel i7 processor.  When I run the houdini engine to analyze games, it pegs all four cores at 100% usage.  Is this common or am I doing something wrong?  I just used the default setting from chessbase.  I also have 12 GB of RAM.  Is there a way I can improve performance (reduce CPU usage), or should I consider upgrading to a CPU with 6 or more cores?

mldavis617

I believe chasing multiple cores follows the law of diminishing returns.  Going from one to 4 cores only increases speed 60% in Fritz (according to Chessbase), so ditching a quad core for one with hyperthreading isn't going to help much.  If your cores are pegged, then hyperthreading isn't going to help much if at all.  I suspect processor speed is more important than number of cores, so if you go from a 2.2GHz CPU to a 4.0GHz CPU, you will come out better than just adding a couple of cores at a lower speed.

I set up my computer to analyze my old games overnight, usually with a two hour total time limit.  But then I'm not depending on speed...

mldavis617
FirebrandX wrote:

Hyperthreading merely splits the cores in half (i.e. half as much speed). The concept is completely useless for chess engines, and in fact is a detriment to them when used. Also, more actual physical cores does significantly help analysis time. A current generation Intel hex core outperforms my old quad core by a factor of 4 to 1 when running Houdini.

The ability of the engine to use hyperthreading would be dependent upon how the program was written and how the threads were utilized.  Obviously, chess is a multiple-threaded application, so more cores handle more threads.  Late generation Intel chips handle memory IO faster than older chips that relied on the older front side bus architecture, and yes more cores are able to handle more threads.  So there are several variables, including, as I mentioned above, CPU clock speed which has hit somewhat of a plateau unless you're willing to part with some big bucks or risk overclocking issues.

I'm not a chess engine programmer, so I'll take your word for the 4X speed increase, but I'm skeptical that going from 4 to 6 cores gives you that jump.  And what does that speed gain you unless your're playing online against another computer?

bfn
FirebrandX wrote:
mldavis617 wrote:
FirebrandX wrote:

Hyperthreading merely splits the cores in half (i.e. half as much speed). The concept is completely useless for chess engines, and in fact is a detriment to them when used. Also, more actual physical cores does significantly help analysis time. A current generation Intel hex core outperforms my old quad core by a factor of 4 to 1 when running Houdini.

The ability of the engine to use hyperthreading would be dependent upon how the program was written and how the threads were utilized.  Obviously, chess is a multiple-threaded application, so more cores handle more threads.  Late generation Intel chips handle memory IO faster than older chips that relied on the older front side bus architecture, and yes more cores are able to handle more threads.  So there are several variables, including, as I mentioned above, CPU clock speed which has hit somewhat of a plateau unless you're willing to part with some big bucks or risk overclocking issues.

I'm not a chess engine programmer, so I'll take your word for the 4X speed increase, but I'm skeptical that going from 4 to 6 cores gives you that jump.  And what does that speed gain you unless your're playing online against another computer?

Again, hyperthreading wouldn't help at all unless we're talking about not using all 100% CPU resources. Say for example you want to use 3.5 cores instead of all 4 cores, perhaps hyperthreading would allow that and still be slightly faster than 3 cores, but every chess programmers has always stated that hyperthreading does more harm than good when it comes to calculating positions.

With regard to the 4-6 jump, I should point out that each core was also a full Ghz faster. So instead of 4 cores at 3 Ghz, the current 6-core at 4 Ghz was indeed 4x faster at full load. Houdini's knps went from 6k to 24k on the spot. Compare this to the dual Xeon that jumps it up to well over 40 knps, though obviously the 16 cores are making up for that fact that Xeon CPUs are much slower than the single hex-cores CPUs.

What does all this benefit? The simple answer for me would be analysis time on ICCF where computers are allowed. Somebody with a hex-core could analyze in one night what would take my old quad 4 nights to do. In other words, my overnight analysis would only equal 2 hours of work on someone else's hex machine.

Do your CPUs still max out with 6 cores?  Also, does your system have traditional fans or is it liquid cooled?  I'm thinking of making my own system and have a budget of ~$2,000.  Any recommendations of a build your own system or from a reputable manufacturer?  It would also need to serve as a gaming machine too.  :)

pfren

The xeons are fantastic for dedicated jobs, but IMO not very suitable for chess. For much less money you can set a quad-opteron rig - the most expensive opteron 16-core CPU does not cost more than $ 1,000 (each) officially, although the street price is quite lower.

But you can get excellent results (easily some 14M nodes per second) with a cheap FX-8350 for just $190. Sure enough the FX-8350 is a weaker CPU than i7 3930k, but the difference when it comes to chess is too small to justify the huge price gap (the 3930 costs 2.5 times more).

The AMD FX series could be the ideal budget choice for chess servers, but unfortunately their architecture does not allow more than one CPU per rig (same applies for the expensive i7's).

bfn

Cool.  However, a little bit out of my price range (you didn't include tax, shipping, and the cost of an operating system).  Thanks.  I'll change my engine parameters, too.  I wish the engines provided better documentation on how to get better performance.

mldavis617

As a followup to the hyperthreading comments, you do know that the Intel i7 chips all use hyperthreading, with a very few exceptions for laptops.  As I mentioned, I do not write programs any more, but a chess engine is a prime candidate for the use of as many cores (real or virtual) as there are available.  My old quad core Q9450 is overclocked to 3.21GHz and runs all four cores if the engine is programmed to do so (Fritz 13, Houdini, etc.).  What should be done is to run one benchmark (for example the one that comes with Chessbase products) with all cores hyperthreaded, then disable hyperthreading and run it again.  Tell me what you find.  The benchmark on my quad core is 8916 kN/sec.  It is not hyperthreaded, but the newer i-series are for the upper level chips.

mldavis617

I'm not arguing.  You stated that hyperthreading does nothing for a chess engine.  I'm saying it depends on the program and how it's coded.  I run a lot of high end heavy load programs such as Adobe Photoshop, Microsoft Flight Simulator and X-Plane.  I also have a high end graphics card onto which much of the heavy lifting can be off-loaded from the CPU to engage the massive speed of the nVidia architecture.  If chess engine authors are not availing themselves of the potential speed increases of these available features, they are missing a lot of horsepower.  If in fact the engines are being limited in their use of available multithreaded and parallel processing power, then you are correct.  Perhaps that coding is beyond the abilities of amateur programmers.  It certainly works with Photoshop and Flight Simulator.  Did you run comparative benchmarks as I suggested?  What was the result?  I cannot do that on my old Q9450 quad core.  The question is the number of simultaneous threads that can be run, or the speed of the processor.  If hyperthreading is useless, then Intel is wasting their time and we need to send them an e-mail, OR there are programmers out there who are missing the boat.  Who to trust, Intel, or amateur chess engine programmers......

Kingpatzer

Something like a graphics application is not entirely comparable with a chess engine. FirebrandX is absolutely correct that hyperthreading does not help chess engines. Period. End of story. If you don't believe us, go ask about it in any of the chess programming forums. 

You are simply wrong on this point. 

It is not that hyperthreading is useless. It is that for some algorithms it is detrimental to performance, while for others it enhances performance. Chess is an example or a problem set where the best known algorithms for producing the desired result are hampered by hyperthreading. 

pfren

I can confirm that adding virtual cores to the engine pool does not improve performance. In the case of Houdini 1.5 (I don't have 3.0, nor I intend buying it) the only "gain" from the ante of threads from 2 to 4 (my best rig is an 8-year old Core2Duo 6600, which isn't even SSE4 compliant) is making the rig unusable for any other use, while Houdini is "thinking". But AFAIK this also applies for newer Intel CPU's, which fully comply to the SSE4 standards.

At least when the engine uses the two physical cores, you can still use the rig for some other light jobs...

mldavis617
mldavis617 wrote:

I believe chasing multiple cores follows the law of diminishing returns.  Going from one to 4 cores only increases speed 60% in Fritz (according to Chessbase), so ditching a quad core for one with hyperthreading isn't going to help much.  If your cores are pegged, then hyperthreading isn't going to help much if at all.  I suspect processor speed is more important than number of cores, so if you go from a 2.2GHz CPU to a 4.0GHz CPU, you will come out better than just adding a couple of cores at a lower speed.

Jeez, how did I get on the wrong side of these barbs?  Read my original post above.  And again, above, "...it depends on the program and how it's coded."  I also begged off on any knowledge of how chess engines are coded.  No argument on whether or not hyperthreading is helpful in this application or not.  Again, run the benchmarks and see.  I'll agree with whatever you find.  Smile

Kingpatzer

mldavis617 - we're TALKING about chess engines though, not about generic programs. Yes, when speaking generally without regard for the type of programs being run, hyperthreading in general increases performance. But chess engines are a specific type of application with specific needs and are the specific topic of this discussion. We aren't talking about general computer architecture questions devoid of application context. We're talking chess engines. With regard to chess engines, hyperthreading hinders rather than helps. Period.

Nordlandia

My quad i5-2500k work good on all tasks (gaming/video compression) and chess as well. Wounder if the tech is already outdated for chess performance sense.

Chess benchmark score:

Speed: 20,88

KNPS: 10024

http://www.liquidnitrogenoverclocking.com/vortex_f6.shtml

Xilmi

Don't know about Chessbase, but since it is in the settings for Arena 3.0, I would guess chessbase has something similar:

There should be a way to reduce the real-time-priority of the engine. So that it is only run at full-power when nothing else needs the processor.

This will reduce it's power only when you are doing something else at the same time, but at least doing something else works.

mldavis617

Fritz allows you to set the number of cores running - or as they call it the number of CPUs.  This is dependent of course on which engine you have engaged.  The basic Fritz 13 engine will only run on 1 CPU as the link shows but in fact it runs on all 4 cores on my machine, each core at 25% load.  There also seems to be a limit of just over 1000 threads handled at any one time on the quad core (Core 2 Duo) Q9450, so hyperthreading would not increase chess engine efficiency - I'll eat crow on that.  In comparing Houdini 1.5a 64x running 4 CPUs, that will run 4 cores at 100% load.  So the Fritz 13 basic engine seems to be the same as Fritz 13 Pro, dumbed down to 25% efficiency.  If you want the whole engine, you pay for what amounts to the key to unlock it.

From a practical standpoint, if I run Houdini 1.5a on "3 CPUs", I can let it run in the background and use my computer for more mundane tasks such as reading e-mail or surfing the 'net.

marcustheadore3

Hackers are able to take advantage of gpu's for brute force attacks now. I understand that what gpu's do is vector calculus. Dosen't that make them essentially math processors? Can our chess poeple please make use of this? I want more nodes for less dinero. More on topic though, the AMD FX-8150 overclocks to perform at what the AMD FX-8350 does on merely a Hyper 212 EVO (heatsink and fan). Therefore the AMD FX-8150 makes $500 chess super-computers available. 

See the below links for Tom's Hardware comprehensive fritzmark benchmark (excluding dual-cpu) and also for a link to professional and competitive overclocking.

 

http://www.tomshardware.com/charts/cpu-charts-2012/-22-Fritz,3160.html

 

http://hwbot.org/hardware/processors#key=am3_plus

mldavis617

I think the basic problem is that most chess engines are written by amateurs, a few of them being commercialized.  You'll usually see the programmer's name associated with any engine you find, including Houdini.  The problem with off-loading calculations to the GPU is that the program must have been written to accomodate that.  It isn't, as far as I know, simply a function of the CPU seeing all that GPU horsepower out there and sending the load away for faster computation.  The program must be written to utilize the GPU, and not knowing what GPU is available since it varies with the system, I don't think our engines are (yet) capable of speeding up with a powerful GPU onboard.

marcustheadore3
[COMMENT DELETED]