"losing passengers to high-speed railroad lines"

That's exactly what HSR has done in Europe. It hasn't really put a dent in highway travel, it's taken business travelers off of short-haul airline flights. The "average Joe" does not use HSR like the American press wants you to believe - they use conventional trains or drive their car (which has dramatically increased in Europe in recent years).

As a result, much of the short-haul air market has dried up except for flights to places where HSR can't go to (i.e. islands).

And that's exactly what HSR will do here in the U.S. If built in California, a lot of the L.A.-S.F. air market will collapse. HSR could shut down the Horizon Shuttle from PDX-SEA. But in the Northeast Corridor, there's still multiple airlines (American, Delta and U.S. Airways) and now buses (i.e. Megabus) competing with Amtrak's less-than-HSR service. We have planners planning for HSR in corridors with little overhead air traffic (i.e. Portland-Eugene) - they will be sadly surprised when the traffic volumes don't materialize (because nobody wants to travel between those two points today, as evidenced by the lack of traffic on I-5, the lack of flights, and the lack of Greyhound buses).