Who wants to help train my Machine Learning Model

Hi guys,

Not sure this is the right place to ask but I created a small iOS Swift library that analyzes a picture of a chessboard and renders the FEN notation of the pieces. I'm currently training my Machine learning model with a lot of different pictures of 2d top-down puzzles. If you are interested in helping me feel free to reach out to me. I'm looking for sharp photos taken from books or newspaper chessboard puzzles.

Andy

Since we have huge PGN databases in TWIC, could the model be trained by some software that would do it backwards - print out positions from the DB, in the meantime simulating bad quality, various fonts and deformations of the page?  I think some graphical filters when applied to the positions could help with that...

@timothyha I'm not sure I fully understand how you mean it. I can elaborate a bit more on how my framework is going to work.

you can take a picture of a chessboard like this from a book or newspaper. and then the image recognition together with the machine learning model is trying to analyze this position and renders the FEN. the next screenshot shows how its currently working. In the bottom, you see the analyzed picture and on top the rendered FEN. And what I need now is a lot of photos of different chessboards in books to be able to train my model. I, unfortunately, need actual photos to train the model. For now it only works with 1 book good and with some other books quite bad.

Yes, I do understand the training part, but in view of a lack of photos, especially with the corresponding correct FENs, why can't you try to simulate photos by using the PGN with correct FENs and print them out with low quality into graphical images?  And then, feed that into your ML network?

@timothyha How would you simulate photos of chess pieces? we don't need different positions to train the model we need different single images of chess pieces.  since they all look different.
the following images are samples of the training data for the model. So the actual position on the board doesn't matter only the picture and shape of the piece is what matters.

There are good TTF chess fonts (some come in the pgn4web software) that simulate the common book symbols of chess figurines.  And then you can damage the quality and feed that into the learning process, maybe?

oh nice. I didn't know that there are TTF chess fonts. that could actually be a good idea. let me see what I can find there.

I hope it works.  Some of your colleagues, by the way:

https://web.stanford.edu/class/cs231a/prev_projects_2016/CS_231A_Final_Report.pdf

and this, https://www.reddit.com/r/chess/comments/50b8kz/an_app_for_scanning_chess_positions_from_books/ - I forgot where the author had his webpage.  

I saw this app. I actually started for fun and wanted to write a framework that can be used by whoever and not focusing on an app since there are soooo many different chess apps. And now it gets better and better. so I hope I can publish this framework "soonish" and have some smaller apps integrate it. 

@timothyha I downloaded now a ton of fonts but I can only install 4 of them in OSX. The rest has errors and can't be installed. Do you have any experience with Chess TTFs on OSX? I unfortunately don't have a Windows machine around.

I didn't try printing PGN with these fonts, but pgn4web that I mentioned earlier has a set of them -https://sourceforge.net/projects/pgn4web/files/legacy/

Try the .zip archive - https://sourceforge.net/projects/pgn4web/files/latest/download
The fonts seem to be viewable in Mac OS Finder.

As for your framework, please ping me whenever there's something to play with!  Thanks for the work!

@timothyha thanks. I'm actually preparing a Testflight release for the "test" app and then it would be great if you can just take as many pictures of some books. It will upload those so I can see the result and optimize the model. In case you are an iOS user?

Yes, I'm on iOS 

You can use Pix2Pix to generate images from PGN.

https://www.tensorflow.org/tutorials/generative/pix2pix

I became more familiar with applying this after reading another article here on Chess.com ...

https://www.chess.com/news/view/update-on-regium-chess

Referenced: https://www.thispersondoesnotexist.com/

https://en.wikipedia.org/wiki/Generative_adversarial_network

That TensorFlow thing actually works pretty well for taking something like text (picture or screenshot) and turning it into sketchy images -- which you could then train another network layer to output to PGN, etc.

The ML stuff is getting pretty interesting in terms of what you can do with it and training networks to identify things.

@SJCVChess Yeah it's crazy where we are heading with technology. I saw the Pix2Pix as well a while ago. For now I'm trying to stay out of it. Looked quite complex  

Nope. Really simple. Follow the guide.

The most complex part was figuring out and finding reference for the input image format and such. After figuring out the stuff they don't tell or show you, it's very simple.

ok cool will definitely have a look. I first want to create my alpha version that I can distribute the capture app to some people. Since I only work in my spare time on this, the progress will not be the fastest  

@timothyha I sent you an invitation in a private message for the alpha version

Perhaps could try using a javascript library like roughJS or even doing some image manipulation to blur what you have and get more 'different' types of chess pieces.