I'm not on AT&T, but I will chime in with my experience. No difference in battery life between having wifi calling on & off. Personally I live in an edge spot for TMo, and if I'm going to be on work calls, I turn off cellular to force it to wifi, which always results in better call quality for me (at my house; it is more hit & miss when I'm out & about).
If I were the OP, Id give wifi calling another try & give it a few days to see if it settles down. I can't imagine that it would impact the battery life in a significant way & wonder if something else was going on at the same time (system software, apps, music or podcasts downloading in the background, etc). If you do it for a week, and it's still giving you bad battery life, then I would turn it off, but I'd definitely give it another shot. Like one of the posters above said, you may not be able to turn it on during an outage & it's pretty helpful to have when you need it.