It's a matter of what you call a railway. I recall learning about the first railways at school (one of the few things from history that stuck) - the Stockton and Darlington railway, in Victorian times. I did have to look up the date. That referred to mechanically powered, public railways, as most people imagine them. Horses pulling trolleys on rails are not really the same thing.
Wikipedia answers the question you were considering: "The oldest known, man/animal-hauled railways date back to the 6th century BC in Corinth, Greece."
I have to say that from my limited experience of Greek railways in the late 20th century, they probably got less efficient over the intervening period. Being several hours late was the norm.
I can't remember being on a Greek railway but I did travel right across Anatolia to Erzurum from Istanbul, starting about Boxing Day 1975, which was an atrocious Winter. We were double hauled but even so got stuck on a mountainside for about 15 hours in a total white0-out and we had to wait for the blizzard to stop before the snowplough came and dug us out.
Most people in my compartment were from Kars, up near the then Soviet border beyond Mt Ararat. I've always had a good resistance to cold so I was ok but they systematically took the carriage apart to get all the wood and we actually had an open fire going in the middle of the compartment. They were very kind to me ... let me sleep in the netting luggage rack, which was an excellent hammock and shared their food with me because I had virtually none. As a result of many similar experiences in Turkey I love Turks in general.
First it was horse-drawn, as in France but then static engines became quite common. When I was a child, the working mine near the docks in Whitehaven used a static engine on a railway line right up to the mid 1960s to haul the empties back up the spoil heap to be filled. Nearly all the time they didn't need the engine because the full wagons coming down the slope pulled the empties back up via a loop hawser. You could stand literally one foot from where the full wagons came down over the chute and they knocked out the pins and the coal went into the ships waiting to take it to Ireland or the Isle of Man. Absolutely zero health and safety. Anyone could go into the docks, which were just a continuation of the street along the dock-side. No protective walls or fences so if you wanted you could fall down the gap between the ships and the dock walls and get crushed to death. I suppose as a result I had absolutely no fear of heights and became a half decent rock climber.