ChessFigRdrLite - Android OCR Program for Chess Figures

I have written an Android app which uses the  camera to recognize Chess Figures in books using OCR technology. It saves each recognized figure in a pgn file, and also puts the most recent one in the clipboard as a FEN string. You can then directly invoke a chess program like DroidFish, or SCIDOnthego directly to analyze the position or do a data base search for games that contain this position.

The video http://www.youtube.com/watch?v=H6NuOqUrV2M shows the app in operation and the Google Play link for the app is https://play.google.com/store/apps/details?id=com.kgroth.chessfigrdrlite

This is release 1.0 and is a demo only. It always shows the recognized figure, but only saves the first four figures. Thereafter the recognized figures are no longer saved but are still shown. I plan to sell a non-demo version if there is enough interest. Please look at the video and the help menu item to understand the program operation in more detail. Comments and e-mails are welcome.

Gerhard Roth, Ottawa, Canada

This app has been replaced with ChessOcr https://play.google.com/store/apps/details?id=com.kgroth.chessocr

(free version) and ChessOcrProKey (paid version). Release 1.2 can now be called directly from the chessapp SCIDOntheGo (see Google play page for my app). Also, there is a web site with more information called http://kgrothapps.com/ Do not buy the paid app until you are satisfied that the OCR works well enough on your books. The free app only allows 25 positions to be saved but it always shows the results of the OCR, so you can check how well it works for free.

Appears to be incompatible with my device.

You need a back facing camera with autofocus for the app to work and at least Android 3.0. Recognizing a figure means that you can analyse it (there are two free Chess programs on Android; DroidFish and SCIDOntheGo) or you can find all the games that contain that position using SCIDOntheGo and a free database of 2.4 million games on the app web site. So then you can play through the game on your phone/tablet.

I have a back facing camera, and android 4.1.  It still says incompatible, I also have droidfish & chessbase.

The camera needs to have autofocus or else in my experience the recognition is not succesful. Sorry about that. I have another app, called ChessOcrPict https://play.google.com/store/apps/details?id=com.kgroth.chessocrpict It does the same as ChessOcr but the input is a picture taken from any camera, or a screen capture in the gallery. This app does not require a camera and works with 4.1.

Gerhard, can you release an iOS version?

The newest version of ChessOcr, which is version 1.9 https://play.google.com/store/apps/details?id=com.kgroth.chessocr&hl=en now works with back facing cameras that do not have autofocus. I have tested it on my Samsung Galaxy Tab 3. Performance is good, but you need to hold the camera a bit farther from the figure. The app draws a green box around the chess figure, so once you see this green box, it will work. The easiest way to test the app is to display any pdf from the excerpts section of http://www.qualitychess.co.uk/pdf.htm on your monitor, and then try to recognize any of the figures using ChessOcr. Once this works you can try different books. Any questions or comments you can e-mail contact@kgrothapps.com

I prefer to work on more Android apps than to write an IOS version.

Impressive app ... I may have to buy an android device to enjoy it. If I can ask? What kind of algorithm does it use.

Sorry for the delay in responding. I do not login to this web site that often. I use some optical character recognition. However, the algorithms are modified to deal with the fact that the chess figure is not exactly head on, and the page is slightly warped, and the lighting varies. In my opinion, the app is unique, and I think it is difficult (but not impossible) to duplicate. By the end of this year I plan to have an IOS (Apple) version. 

The iOS version is great news Gerhard, I look forward to it.

Hear hear!

Hurray! I will buy an iOS version, for sure, as I've been wanting this functionality for ages. Compatibility with older (e.g. 5.x) versions would be a big plus, imo.

I played around with chess OCR a bit in MATLAB a while back. It's an interesting problem. Got as far as non-uniform lighting correction, thresholding, board edge recognition/corner location and unwarping, but did not tackle piece recognition. Gerhard, it looks like you did an excellent job on this.

Yes excellent OCR work by Gerhard. It is possible for iOS, I used to use a Sudoku app which did the same for printed puzzles.

Thank you for the complements. It sounds like fburton did quite a lot of work towards solving the problem. I spent a long time on the image processing, and some of it is complex. To do a good job more is required than what you have listed, but this is the correct start. I use OpenCV, and this package is ported to both IOS and Android, so the image processing part of the app will stay the same. The effort is in learning IOS. The app will work fine on IOS, but the interaction between it and the chess apps is not as flexible as in Android (there are no Intents in IOS as far as I know).

I'm sure there would be much more needed in the way of tweaking, the challenge being to make it really reliable with a range of inputs. My amateurish experiments with MATLAB highlighted what a wonderful image processor our eyes&brain are, and how dumb computers can seem in comparison.

Anyway, I wish you every success with porting to iOS and that you manage to get it to interact usefully with existing chess apps.

I'd also like to buy the iOS version. I have the Android version in my Samsung Galaxy S4 phone and it works great. But the screen of this phone is too small to use as an auxiliary board while reading chess books. I have an iPad Air that I use for most of my chess related activities and I have several paper chess books still to be read. But I feel lazy to manually setup the positions on a physical or on screen board.

Thanks for the motivation! I was just looking at what I need to do to create an IOS version of ChessOcr. I plan to have it done by the end of this year, maybe earlier if all goes well. However, IOS is completely different than Android, so it takes some time to learn. Best Wishes, Gerhard Roth