Yeah, if you remove the battery it will be fine. The problem is when a really, really low battery becomes so weak that it gives off a charge below the minimum 1.8V, this causes a lock-up state/Brown-out-detect. Here’s the Texas Instruments advisory on the issue regarding the chipset used in these sensors:
Basically 0.00V is okay, but 1.78V bricks it. It is technically ruined while the battery is super low, just before it is dead…
Therefore, if you remove a still functioning battery, the sensor will not ever feel the low voltage and will continue to function correctly whenever you reinsert a properly functioning battery again. Your plan will work fine. Go for it.