I've never had much trouble in railway situations, but I'm not a regular attender.
Plane-spotting, on the other hand, can be much more 'serious'.
I will confess to having been inside, but only for a couple of hours. I was either side of thirteen at the time and was apprehended at gun-point - in fact, bayonet-point, which I remember thinking seemed rather quaint. My father and I spent a couple of hours in military custody and were released without charges. The time in the cell was certainly less stressful than the time spent being initially searched and questioned by a very twitchy chap with an SLR. His two companions were rather more amenable. The twenty minute journey back to their base, over dirt roads, was very entertaining - with me being 'held hostage' and my father following on his motorbike, being covered by Mr Twitchy lying on the back bench of the Land Rover that by this stage had no floor left, due to the radio batteries leaking during the emergency run to capture us.
It didn't take long to convince the Interrogating Officer that we were not PLO terrorists, when he finally arrived.
I had a few more armed detentions, but only for as long as it took to explain oneself. The one above escalated because they had buggered their radio and so couldn't get guidance on what to do with us, and there were NATO allies in the vicinity who needed to be impressed by the level of security they were being afforded.