Does the fact that most of your freeways cross fault lines make you not want to drive on them? In fact, some moderate quakes have caused freeways to fall down, sometimes killing people, and they take months if not years to recover fully (decades in the case of the Bay Bridge); railroads in those quakes either were not seriously affected or were rebuilt and reopened within days to at most weeks. Railroad shooflys can go in fast, like highway detours, and they don't waste time getting on with the permanent repair (such as daylighting the tunnel that got wrecked in Tehachapi 1952) either.

HSR engineering will probably be more like freeways than standard railroads due to the magnitude of structures needed, but engineers know how to deal with that. Most freeway structures designed and built in recent years do not in fact fall down in moderate to large earthquakes. Japanese HSR practice regarding design, earthquake warning, and response is well-developed, and can and should be adopted here at least for HSR if not for other things.

Finally, earthquakes big enough to cause damage or require at least temporary shutdown for inspection don't occur every day. Inspection might be triggered a couple of times a year counting the whole state, and damage much less often than that. CA's a big place, so a quake in LA or Tehachapi while damaging to disastrous there would not prevent operation of the rest of the system any more than it would require shutting down freeways in Sacramento. And most (again, CA is a big place) interesting earthquakes occur too far away from major populated areas or the proposed HSR alignments to be likely to cause damage there.

So yes, our active geology does need to be dealt with, because it can't be avoided if you want to build anything in CA. But it's a planning and engineering issue that IS being dealt with. There are many ways and reasons to attack HSR if you really don't like it, but geology is a trivial one - it's an engineering issue not one related to the concept.