The eyes are adapted to see daylight levels of light during daytime. If your ambient lighting and screen don't match this, then you're not using your body in the way it's evolved to function. What we need more of are screens with anti-glare and a wide range of brightness levels.
What might have a worse impact on your eyesight is the effects of Myopia, which is being exacerbated by basically every portable device with a screen. Anyone who has the technological literacy to enable dark mode (and advocate for it) is likely someone who gets above-average screen time and is therefore someone at a higher risk of myopia.
Light mode is lower contrast for me, and is more difficult to read text, especially at the end of the day.
Dark mode is a relief, and much easier to read for me.
I miss it in the situations when it is not available, like PDFs with small grey text on a white background, more so on a small screen like a phone.
Some people think dark vs light is opinion, but for some it is not.
more: https://www.nngroup.com/articles/dark-mode/
- dark mode may present some advantages for some low-vision users
- in users with normal vision, light mode leads to better performance most of the time.
- applications meant for long-form reading (such as book readers, magazines, and even news sites) should offer a dark-mode feature