I know this question is not about the API; it's about something the API doesn't provide and maybe never will. I also know that chess.com might not like seeing scraping in general. However, I'm not attempting to create any competing service or gain any money from this; on the contrary, I need to get a specific set of data that I could send back to chess.com staff and/or other interested members. Getting this data manually is infeasible; it would take an incredible amount of time.
What do I want?
I'm looking for a script/library, which would allow me to scrape all comments from a given finished vote chess game for a given move. Recommending a script which could scrape all comments from a given chess.com forum would also be very helpful, as I think I could modify it easily to scrape comments from a VC game instead. I haven't written any scraping code before and I guess some of you did, so I hope you could help me.
edit: I'm interested in scraping the html content rather than just the text, so that I can extract both the lines/continuations suggested as text and also the PGNs.
Scripting language
I'd prefer to use Python for this project but I'm afraid I might run into trouble with captchas. In that case, using Javascript might be better, as a bunch of JS code could just be pasted into the browser console, even though it wouldn't be comfortable - it'd still save me a lot of work.
Stress/load on the server
Clearly, throwing a bunch of requests against a server without thinking could put more stress on a server that a standard browser user would. I intend to slow the script down by increasing the time between two subsequent requests to 1-2 seconds, which shouldn't be much of a problem for the servers. I also won't be scraping all the VC games on the server, just a number of selected games.
Why do I want this?
I you want to know why do I want this, send me a PM.