Python

Since Python is such a popular, easy to use object-oriented language & many developers & amateur coders use it in their projects, it seems a good idea to have a separate topic for it.

Whenever I need to carry out some operation that involves a lot of heavy-lifting of data, Python is usually my first port of call. Many of my posts on this site that involve a large number of links, are compiled using Python scripts that I've written.

So I'd like this to be a place for sharing views on the language, ideas about projects & Q & A's for those facing problems in their coding: Everything from installation on your computer to starting up the Python shell, writing & editing scripts, through to debugging & more advanced concepts.

I sense that some people may be a little nervous about installing Python in the first place, so I'll leave a few following posts reserved to give a step by step guide on installation & some how-to advice.

(1)Installation

My advice to anyone without special IT skills is to keep things simple!

So visit the Python download page & use their online installer to download whichever version of the language you want. When I downloaded Python five years ago, the current version was 3.3 but the latest is 3.6 & that's the one I'd recommend. Remember that this is open-source software so doesn't cost anything to download!

I think version 2x is useful if you're running a lot of legacy programmes written for it but as a new user, definitely opt for Python 3.6.x. I found the download one of the easier ones I've attempted but that was in Windows7, so I can't speak for Mac machines or Linux.

The installer will create a new folder in your machine's root directory of the form: Python36 & within that, the application itself along with several sub-folders containing DLL & library files etc. It should be that easy but if you have problems, please ask (providing your computer's operating system & browser details).

If the download's been completed successfully, you'll be able to see the Python folder on your start menu. In Windows10 it looks like this (after I've opened the folder by clicking on the down-arrow):-

.
The option to start Python is the IDLE one but I'll cover that in the next post.

(2)Starting Python

Click on Start (Windows) or whichever operation displays your applications under your particular OS. If you can see the Python application folder, open this & you should be able to see an option titled IDLE (Python GUI) or something very similar. IDLE stands for Integrated DeveLopment Environment which is a form of graphical user interface. Click on that option to launch your version of the Python interpreter.

A new window should open that looks something like this ...

.
That's the Python 'shell' window. The interactive Python command line is indicated by the '>>>' symbols. You can enter simple commands or operations followed by Enter (/return) & you'll receive a response. For example, type 2 + 3 (enter) & the result, 5, will be displayed on the next line. Type 2 * 3 & the product of those numbers is returned, i.e. 6. 2 ** 3 returns or gives the cube of 2, 8.

However, if you want to do more than enter simple one-line commands, you'll need to access the Python editor window & that can be done by clicking on File & then New Window on the File menu. A new window opens that should look like this ...

.
At that point you can start to write your own programs, editing as you go. When you're satisfied that your script looks o/k & want to test it, you can either execute it straightaway or check it first. Do this by clicking on the Run menu option & the menu opens, then Check or Run your module (program). Python requires that your script is saved first & a dialogue box will open for this.

Any console output (i.e. to the screen) that your program creates will be printed in the shell window. Any errors involving the syntax of your script will raise an error dialogue box & the location of the error is highlighted in red in the editor window. Once you've identified the incorrect syntax (I have a habit of omitting colons), correct it & re-try.

Other types of runtime error will create an error report in the shell window. For example, if you forget to assign, or otherwise define, a variable before referencing it, you should see an error report similar to this ...

.
I'm assuming a certain basic knowledge of programming languages above but if terms such as syntax are unfamiliar, please ask for help.

Otherwise, happy coding 

.

.

.

Linux and OS X note:

Ubuntu 16.04.4 LTS (Long Term Support) ships Python 2.7 and Python 3.5. I imagine most other distributions will also have both available.

OS X El Capitan (about two releases old) ships Python 2.7, but 3.x is readily available as binary packages or can be installed from source if you like that sort of thing.

Further, I moved one (well, right now the one and only) Python script I have which accesses api.chess.com across from OS X to Windows 10, and after installing the "requests" module (which I heartily recommend) my script just ran.

Thanks for the info.

Your reference to api.chess.com reminds me that I need to start work on a method of dowloading & filtering out all players signed up to our new round of TMCL matches who have time-out ratios at or over 15%. We allow team admins to remove any such player from their rosters, so publishing a list of them might help admins with large teams (200 players isn't that unusual).

But it's the kind of task for which Python is perfect.

I've all the pieces I need for that ... but there is one problem: api.chess.com gives integer percentages, and despite my asking at least three times none of the developers will say if the numbers are rounded or truncated. 14.99% isn't 15% ... unless the TMCL admins say using api.chess.com integer percentage numbers is OK. (Plus there is caching. PITA. But if it says most are right and  you only have to check a few manually, you still win. Want the code, or want to write it yourself?)

I suppose I could even translate what I have to Python. Honestly, for most of the 100-150 line scripts I've written its six of one and a half dozen of the other. Python is more verbose; perl's syntax errors get weird when complex data structures are involved.

Perl's syntax for regular expressions I definitely prefer! Python has the same functionality though so it's syntactic sugar at best.

I'm fairly busy just now on a couple of projects, so anything you have would be most appreciated & I always give acknowledgements in the credits!

What's easiest - a message?

A PM with a URL, I think; unless it's entirely too slow to write I'll write it in Python for you and avoiding cut&paste with Python's requirements for precise indentation is probably a good idea.

Arguments as match IDs suitable? If you have a preference for something else that isn't web scraping of the TMCL forums let me know.

I'll try to make the per-team and per-match parts standalone subroutines so you can take the argument parsing code etc off in favour of something else without too much trouble.

Making anything an all-up module is probably overkill in either language.

I misunderstood there because I thought you had it in Python already. Please don't go to the trouble of re-doing all of that from scratch. Once I have the api downloads, it should be a simple enough job to splice out the details I need.

What would help me a lot is the Python code I need to make api requests. Up until now I've found it easier to copy & paste the text content of web pages & use that as my source material but api data means I can automatically fetch that information directly from the servers.

I obtained some Python code a while back that worked well enough for simple web pages but when I tried it on this site, I kept getting error messages.

For what you want, it's knowing the endpoints. No difference in complexity. What is going to be more difficult is playing some live bullet against a young enthusiastic member of one of the teams I'm an admin for. Wish me luck.

Minimum you can take what I provide and use it as an example if nothing else.

I've seen people refer to 'endpoints' a few times but I'm still not clear what they are. Surely making an api request for a particular set of data for a given page, only needs the URL & returns a file of data specific to that webpage?

I don't suppose it's possible to filter a request? That's to say request only player (user) names with their time-out percentages & nothing else?

It returns JSON. The API is currently read-only without authentication, so any filtering has to be done client side. It's pretty simple, but a lot simpler with an example, which I am writing to give you one and polish my Python.

Now, if I wasn't likely to have to shift notebooks (a dodgy "a" key is a bit of a problem) today would be going better. I have another notebook fortunately, but I like this one and Apple don't have a really enticing replacement for sale (performance/price/new release). Either the key will settle down or I'm about to start using my Windows notebook as my main machine. As it was bought as a backup for that very purpose, I can't complain too much.

I calculated once that I have typed well over one million words on this machine so for its keyboard to be a little worn is not that surprising. Age plus condition makes repair uneconomic I fear.

Later thought: postman or one of the interactive tools might be enough for you. I've not tried them. Another thing for the TODO list.

Edit: postman looks like a development environment; scratch that.

Code:

https://www.skelos.net/static/tmcl-player-timeout.py

Note: The non-core module "requests" is used. The current version at the time of writing is 2.18.4 and works with Python 2.6–2.7 & 3.4–3.7. Unless you have good reason, use Python 3, but not version 3.3 or earlier: to make the script work you'll have to choose a different HTTP/HTTPS module and adjust the code accordingly.

The script could use a nicer name. It could also have the ugly ".py" dropped if you're on OS X or Unix/Linux, but on Windows, you'll want the extension. 😈

It's pretty basic, purposefully since:

• That makes for a better example
• It needs to be functional more than pretty.

Edit: The script really should handle an exception nicely if a fetch from an endpoint fails; it doesn't partly due to an oversight on my part and partly that I haven't mastered what happens for different failure modes (404 for a bad match_id, presumably 5xx codes for network failures, and more ...} and how they appear as exception. If someone makes those changes and wishes to share them, I'll update this post and the online script with credit.

Usage is something like the following.

[Yes, the data is real as of the time of posting. It's all public information and it's not finger pointing; if I have to I'll obscure the names but knowing what is a username around here that isn't used can be tricky. @clubs I found after I wrote a bug in one script.]

$ ./tmcl-player-timeout 884868 884840
Match 884868 team 300-spartans
	37% @lalasdim

Match 884868 team boris-spassky-chess-club
	23% @cotopeny2009
	25% @driso
	22% @rickgere
	28% @sellig_1973

Match 884840 team --russia-central-federal-district
	16% @gidra_gidra
	24% @nataliyamoiseeva54
	26% @ssemafor
	39% @stodor

It could be prettier; human readable team names and mixed case player names for two.

I am not sure whether match IDs on the command line, or match URLs, or reading either from a file might not be a better user interface. @stephen_33 you'll be able to do whatever you want about that I'm sure.

[ Code from URL above originally posted but elided; it was only partly useful to one person and I have updated comments in the version linked to above. I'd rather not update two places if I update it again. --giles]

I notice in reading my previous post that the option handling is not used. Part of my boilerplate; I always have -v for verbose and -D for debug in my scripts. That code can stay. I'm more irritated about not handling an exception from a endpoint fetch. But the code is usable and someone else may apply any polish required. There at least is an example of handling exceptions around the option parsing: the benefit of starting with a template for any new code.

Update: For @stephen_33 particularly @andreamorandini says the integer vs decimal issue for timeout_percent is not forgotten, currently rounds (so 15 may be 14.55) and will be fixed he suggests within a week. The code above is likely to stay working, but I might tweak the comments and version number in it after the change goes through.

Thanks for the info' but I think I'd work around that rounding issue by adding a caveat to any list of excessive time-outs I posted, cautioning team admins to check any members below 16% before removing them. Not really a problem.

In fact I might just set the threshold at 16% & leave it at that. Still a major improvement on the situation we have now where some admins search for high ratios while most others don't have time or aren't concerned.