mmc: fix device_remove when HS400_ES is enabled

HS400_ES is missed when down grade to HS mode during
device_remove the mmc device

Signed-off-by: Ye Li <ye.li@nxp.com>
Reviewed-by: Jaehoon Chung <jh80.chung@samsung.com>