I don’t mean to distribute, but to scan for myself to read digitally, am I allowed to do that legally and ethically when I buy a physical book or would that be totally unethical and illegal?
Legal? Likely not, as you’re turning one copy into two copies.
Would anyone pursue you? Likely not. No one is going to get a warrant to search your devices to see if you’ve photocopied your books before giving them away; unless you’re sharing the digital files publically.
Do not share the digital files publically, as that is definetly not legal.