I‘ve set an automation (recurring daily) in the Home App to lower the volume in the evening and raise it again in the morning. Can’t set an exact volume, as there‘s only a measureless slider, but it still works okay for me.if I know I want to do/test Siri stuff and it’s late at night or early in the morning I say the above command before doing anything else.
Siri’s voice volume on HomePod is whatever the volume of the speaker is set to.