I have found this to always be the behavior. If you are opening the other door after this one on your phone while it connected it’ll switch it othe last door used next time. The real fix is multiple doors for the same location, they really need to implement that.