Vex seems a slightly archaic word, one more suited to 19th Century English novels. It simply means to afflict or annoy which can be the effect on people who get questions about their age.
Different ages look on age differently. Ask a young boy how old he is, and he may respond with an enthusiastic “I’m seven and going on eight.” He can’t wait to get older. Give the same question to a mature man, who would be happy to put aging on hold, and if he responds, it may be a grumpy “60” and a quick change of subject.
Women seem to be more sensitive to inquiries about age than men. Years ago I volunteered to register people eligible to vote, On the registration form, in replying to the age question, some would write merely “21 plus.” Meanwhile, I tried to guess their actual age. Silently, of course.
Curiously, this emphasis on age secrecy lessens with the passage of time. It is not unusual for a very senior lady to take pride in her age. She is not only are happy to reveal it, but volunteers it with an uncharacteristic enthusiasm.