Return your XS and get an iPhone 8 then.
That doesn't help the OP.
One way to improve Face ID when you first set it up is to not hold the phone directly in front of your face at head height - lower it to your most comfortable position (say around stomach/chest height) where you'd typically want to unlock the phone as you take it out of your pocket or lift it off a table.
Have the face registration look at your face from below, if that's where you'll normally glance at it to unlock it.
Over time, Face ID will also improve itself, as it learns your face, but having the initial registration set up from a point of view you'll most commonly use to unlock it should help.
It's still won't be perfect as it's continually being refined, but I wouldn't give up on the phone too soon without giving Face ID as best a chance you can give it.