Create app to "see" board

Sort:
Avatar of thebioguy

It seems to me that a cell phone (or anything with a camera) could be used to "see" a board and (1) record moves, (2) be used as an input device for chess computers and (3) online chess servers.

Does something like this exist??

Avatar of mgx9600

Nothing I know of exists, but I've toyed with this idea before.  The main trouble is mounting the camera somewhere above the board.  I suppose for personal use, it isn't too big a deal.

 

Avatar of tambarskjelve

You mean something like this? You could even combine it with a chess engine for a more "capable" robotic chess player, like the Raspberry Turk. There are plenty of libraries/frameworks/APIs to help you develop an app with this kind of tech, like OpenCV and Apple Vision

Avatar of thebioguy

Yes, exactly like that! Pair it with something like this ( https://www.instructables.com/id/Phone-Scanner-Stand/) and it's a go!

Avatar of tambarskjelve

Placing the camera directly above the board shouldn't be necessary. You should be able to just point the camera towards the board from any angle as long as all the pieces are visible for the object tracker, although this might require some calibration beforehand. 

Avatar of rothaus

This requires machine learning, feeding the software with tens of thousands of hand-tagged pictures of real chess boards. There's a guy in Silicon Valley who did that, he even published his code. 

Avatar of zeitnotakrobat

Something similar already exists, see https://github.com/swenae/chesscam

But I don't think it is an app for a mobile.

Avatar of thebioguy

OK, now we're getting somewhere:

https://www.youtube.com/watch?v=5268q4ZBMLA

 

 

Avatar of rothaus
thebioguy hat geschrieben:

OK, now we're getting somewhere:

https://www.youtube.com/watch?v=5268q4ZBMLA

 

 

No. This assumes a set up board at the beginning of the game and only tracks from which field to which the human moves a piece. It does not have to identify which piece is actually is since it knows that already. It is therefor not capable of scanning a random position on a board.

Avatar of ilarinyman

I am working on it. Ideas & funding would be greatly appreciated.

Avatar of mgx9600

This isn't too difficult if you don't want 100% accuracy on the pieces. I did a prototype a while ago. I'm not as interested in chess currently; but if I get some time, I might wrap the thing up into something useable.


I started with wanting to add an automatic input to my analysis program where you can take a picture of a chessboard (both 3D and 2D) and get a partial FEN (the picture cannot tell you move count, castling, etc.) The program worked pretty well, getting 70%-100% accuracy depending on the image quality (e.g. computer screen and camera sensor pixel aliasing, wrapped pages in books, uneven shadowing on real chessboards, etc.).


I was working on improving each deficient area to boost the accuracy until I got distracted so the project was abandoned.

Based on my experience; I think writing a program to simply track a game's progress (like my first post in this thread) isn't hard at all and you can get 100% accuracy. All you need to do is track movements, color, promotion pieces and human hand is moving around. This seems a lot easier than what my program was doing (and that wasn't very hard).

[fixed formatting issue -- MS]

Avatar of chessroboto

MGX just broke the Internet. 

Avatar of jjupiter6

Why did he break the internet?

Avatar of chessroboto

Is it just me or is the formatting of last post by MGX messed up?