Discovered this article that describes fairly well how SF got the way it is now. Needs some updating (written when Gavin was mayor-in-chief, not Ed Lee), but the basic issues it discusses are still there.
[
www.sfweekly.com]
That said, Caltrain has always been run as something like a railroad, even under public ownership. SF is not fully in charge - SM and SCl counties have voices too. So Ed's Big Project to reroute Caltrain (and incidentally make it harder to run right - shades of the Muni!) may not be in for smooth sailing. We'll see.
Over and out.