This might not be what you're looking for, but I'll throw it out in case you really want to get your hands dirty.
I often say that one of these days I'll try out the Cutechess interface, but I've never done it. So I can't help you with getting it running - You're on your own, unless someone else steps in. But I know that many hardcore testers use it for fast time-control tournaments.
The downside is that there is no compiled GUI yet; Compiles only exist for the command-line interface.
Talkchess thread for the latest version 0.8.2 (go to the end of the thread; Page 5): http://talkchess.com/forum/viewtopic.php?t=60178
Binaries available here: https://github.com/cutechess/cutechess/
Source available here: https://github.com/cutechess/cutechess/releases/tag/cli-0.8.2
Edit - I skimmed the help information, and most of it seemed clear enough. The only portion of the help that isn't clear to me is how you would create a json engine configuration file. (It's Greek to me.)
That looks like it's way too complicated for me.
Moreover, what's the fun in conducting engine tournaments if you can't see them play ;)
EDIT: OK, you replied before I posted, heh.
There have been discussions about why Rybka takes more time to reach higher depth than other engines, even though its strength was comparable to them. If it really was slower to reach the same depth, it would be much weaker.
So people believe that Rybka misreports the depth reached or simply that it works in a different way and that the "depth" means something different.
If I were you I would run a tournament where all engines use 2 threads, the same memory (about 256mb), ponder off of course and exactly 8 seconds for each move. Not sure about books or egtb, probably without those (?)
I want to see whether Komodo 10 or Stockfish 7 would win.
Can you reccomend an interface that allows me to control all that? Something other than Arena.
This might not be what you're looking for, but I'll throw it out in case you really want to get your hands dirty.
I often say that one of these days I'll try out the Cutechess interface, but I've never done it. So I can't help you with getting it running - You're on your own, unless someone else steps in. But I know that many hardcore testers use it for fast time-control tournaments.
The downside is that there is no compiled GUI yet; Compiles only exist for the command-line interface.
Talkchess thread for the latest version 0.8.2 (go to the end of the thread; Page 5): http://talkchess.com/forum/viewtopic.php?t=60178
Binaries available here: https://github.com/cutechess/cutechess/
Source available here: https://github.com/cutechess/cutechess/releases/tag/cli-0.8.2
Edit - I skimmed the help information, and most of it seemed clear enough. The only portion of the help that isn't clear to me is how you would create a json engine configuration file. (It's Greek to me.)