If funds were unlimited, the Bakersfield-LA(ish) segment certainly should have been first. Some form of (Amtrak) connection exists for the rest of the run between LA & SF, and the San Joaquins often do manage to run at 79 (unlike the existing, more scenic Coast route) or even 80 (when cheating a bit, or just going down a gentle grade).

Unfortunately, the original bonds totaled $10B, probably closer to $5-8B(ish) after all the transit "connection" stuff is taken out (and not counting what was spent on the Valley). Add the federal money, and there was maybe $15B to spend. A minimum budget for Bakersfield-somewhere in the San Fernando Valley has been known since nearly the beginning of planning to be around $30B. So, something that could be done for <1/2 of that was chosen as Phase 1. Simple financial math.

On the plus side for your scheme, there's still no money (even with the mythical Biden funding) for BFL-LAish. So we can virtually flap lips to our hearts' content with no worry about actually affecting something. For commercial and transportation service reasons, I expect the Palmdale tweak will happen some day. It most likely will be well after many (if not most) of us who comment here are no longer around. If you believe in a literal afterlife, you might be able to observe things happen someday, as construction proceeds at geological speed.