It might be okay to continue if the channel_manager restores even if all of the channel_monitors fail if that's what we want to do, at least then the channels will force close and the user will get their funds. But continuing without channel_manager seems problematic, since the user will not see their LN balance without any further information (I believe, haven't tested). The design also has UI for this case, which I think is not implemented:
Originally posted by @pwltr in #424 (comment)