Ten years ago, when my gig was running network security, I surprised my then-CISO after he made an off-handed comment about what might happen "if you leave." I calmly explained that my team would disable my SecurID token, an automated job would put my name on a list of people whose building badge would be shut off at midnight, and everything would be fine. He was sure that I would walk away with enough information to break into the company; I was sure that my job involved building and running a network in such a way that knowledge of how we stitched things together didn't constitute a key to the kingdom.

Fast forward to 2011: twice in the past quarter, I've encountered suppliers who resisted sharing basic hosting design information with me, their paying customer. These were suppliers that my company entrusted with its most sensitive data. One claimed their firewall rules -- even the client-specific ones designed to guard my specific set of hosted data -- were private and could not be shared, even under NDA. Another was slightly more open and agreed to share their security-related operational procedures under NDA, but not in electronic form. My question to each supplier was, when an employee leaves (it would have been plain mean to ask about role changes and internal transfers), do they change their firewall rules or their ops playbooks because of the departure?

Don't get me wrong: I think that withholding certain information is a useful, often necessary, secondary defense. We put locks on doors to keep the riff-raff out; but when that sketchy roommate finally moves out (and becomes the riff-raff), we change the locks: changing it is the logical thing to do to a valuable secret when someone who has had access to it becomes untrusted. It's why we still change our passwords periodically, even though forced change is, in this day and age, the weakest of the sundry defenses we put around passwords (more on that later). This, to me, is the easy litmus test of whether an obscurity defense is appropriate. If you're prepared to change the locks when someone who has had the keys moves out, having locks make sense. If you're not, your risk model is flawed and I as your security-conscious customer will be asking you why.