Hi gjp,
Well, the truth is ... it took me 10 seconds to get it (at least, I think) ^^.
I think you can understand it only by saying it out loud":
"Vous êtes d'où? Oui." is nonsense, it means "Where are you from? Yes."
BUT "Vous êtes doux? Oui." would be translated by "Are you gentle? Yes I am."
Both are pronounced the same way, but everything here is a question of spelling.
Hope I could help you properly ;).