We are comparing rather different things here. The NEC is a medium-distance operation, whereas Caltrain on the peninsula is short distance operation (more like an S-Bahn).

The requirements are therefore completely different. Accelleration is not as important in medium-distance operation, where a higher sustained speed and some flexibilty in train length is of higher importance. S-Bahn-style operation requires good accelleration, and many doors for short station dwell time. And that's something the current application simply can not provide, even if you have a Vectron or a TRAXX every four or so cars. Note that a Vectron or TRAXX can haul way more cars, even at speeds of 200 km/h, but it takes its time to reach that speed, and a lot of time is left getting from 0 to 80 km/h…

Therefore, the preference for EMU is not "bureaucratic", it is based on sound requirements.

For medium to long-distance operation, there are fewer advantages of EMUs over loco hauled stock, and to some extent it is also a bit a question of fashion (assumed that the loco-hauled stock operation is actually push-pull, otherwise, it gets rather expensive because it requires a higher number of locomotives than with push-pull operation (actually, the number of trains needed for a service pattern plus 2), and it will need more staff at the terminals.

Now, if the Caltrain cars are rather young, they still could be used on other services, or they would have a decent resale value. They most likely would not end up on the scrapyard.