It's been a while so I don't quite remember but I THINK- and someone correct me if this is wrong- with Setting, Video & Audio 4k or 1080p WITH HDR on, I was getting odd results when playing SDR content. I think that was causing the "dimming" you were describing.
Now your OLED TV should easily be able to handle HDR content, so I suspect the setting for you would be 4K or 1080p SDR but set the Match option to Match Dynamic Range (or whatever it is)... meaning that when you play something through AppleTV that is HDR, it will send it as HDR. However, when you are playing something that is SDR, it will send a SDR signal. The former might give you that sense of "dimming" but think of that as HDR kicking in, which basically promises "darker darks and lighter lights." So if like most movies the opening moments are mostly dark, it's going to look DARK. But when you have a brightly lit/daytime scene, the lights/whites will be extra bright (relative to SDR too).
So I think you should experiment to find the right combination. Try:
4K or 1080p HDR
Match Dynamic Range ON
...and play something that was delivering the "dimming" effect that was bugging you. If it doesn't dim now, you are probably done. If it does, try:
4K or 1080p SDR
Match Dynamic Range ON
That would basically be saying send HDR content to my TV only when it actually IS HDR content. SDR content would be delivered as SDR. Your TV will then respond accordingly.
For me, I have the former settings on my primary TV now and don't notice any dimming. But I think with another TV, I had to choose the latter settings to dodge the apparent dimming when playing SDR content.
While in Match, you might want to go ahead and turn on Match Frame Rate too. That means that when you are playing something other than AppleTV native frame rate, it will pass it through at whatever the frame rate actually is... and your TV will then display it and/or do any frame rate processing. For example, MANY movies are 24fps, so that would be passing them through at 24fps instead of converting them to a higher frame rate within AppleTV.
Both Match options are basically about passing through whatever you are trying to play. Generally, I believe it is very good to have BOTH on at all times. It's that first option 4K or 1080p SDR or HDR that is apparently trying to force one or the other. But, I haven't dealt with this in some time so I may not be remembering it exactly right. Someone else chime in to correct me if you know better.
I hope this helps you.