All data/insn abort exception handlers are expected to run with
interrupts enabled, so forcing the virtual state off causes havoc with
the mm context switching code in SMP, which has particular
requirements due to ASID handling. Other handlers don't care for the
virtual IRQ state.

Would some code start caring about this at some point, a proper
emulation based on the hw IRQ state on entry would be required, not an
arbitrary root stall.