This thread reminded me(surprisingly) of Fallout 2. In a city there, you had to put down your weapons before you could enter, and if you walked as a gun or sledgehammer-totting freak, the cops would shoot you to death.
But anyway, even though it is singleplayer, I only had guns when I know I needed them, because before doing any action, when you have a gun or melee weapon, the character unequips the gun temporarily(with the animation of it) and then do it, what becomes annoying after some time inducing you to not have weapons equipped when you don't need them.
What I am trying to pull is a simple game mechanic: that people totting their weapons around should have some limitations on the way they interact with others and with the world.
For example, your character shouldn't be able to make any friendly gesture(/greet , /salute, etc.) with a knife in hand, for obvious reasons, neither you could trade things while wielding a weapon, and so on. I hope this simple thing won't be dismissed, as it is much closer and easier to be developed than smart NPC guards that put you in jail or perhaps even kill you like those mentioned cops from the beggining of this post. And also it's a much subtler way to encourage people to not have their weapons always equipped around, and the subtler is usually better: a suggestion generates less grief than an order or rule.