Besides the valid advantage for rogues, lets not forget the disadvantage for victims.
The example may be a bit unrealistic, but not more than the whole thing:
One ylian comes to ambush me, his description says ".. wears a brown leather suit". I see his face and he runs off escaping me. The next day I see an ylian who "wears a red velvet suit". My char could recognize his voice and his face with determination, but since the ylian is "Unknown" I got no chance.
Also, I can only ever have a certain person different to others if he wants it himself! That cant be true. Like discussed to extends I cant check descirptions of people constantly only to know whos around.
We have some plot being played at the moment, and I am not sure how can we continue it at all now. Secretly finding out details about persons and gathering knowledge about groupings becomes unrealistically difficult if not impossible now.
So: Add an option to tag people, or mark them, or give them an own description, an individual name!