On Being the Security A-Hole

In a previous life, I used to be a web developer. At one point, I managed a team of developers at a web sweatshop. We would slog away at building websites for mom-and-pop businesses sold the sun, moon, and stars by the sales team with no idea what it took to make a shared scheduling calendar like Google, but swore we could turn it around and push it live by EOB Friday, no problem. It was pretty brutal, but after the dot-bomb of 1999-2001, you were grateful for work.

One of the top developers who was a bit of a bully and loudmouth told the VP of Technology that we needed to step up our security game and he was the guy to do it. The VP agreed. So they built him a desk way across the room (this office was built in a former bowling alley — the ball returns made great cable runs!) and he began his new career as The Security Guy.

He would walk around and if someone left their machine unlocked, he didn’t just make the desktop image a My Little Pony screen, he’d change your password for you. He didn’t tell you he did this, he did it and then watched you freak out when you couldn’t log in and locked yourself out for password failures. If you had your keys or wallet on your desk, he’d lift those. Again, he wouldn’t tell you, he’d just watch you freak out. He would snarkily tell you off for not being more aware of your surroundings or the dangers of leaving things unattended, but he liked seeing people squirm and loved making you feel stupid and have to beg for your stuff or your password.

One day, I got an email from this guy telling me to log into Customer X’s database and see what was written there. I logged in with my admin creds and saw a message telling me to look at Customer Y’s DB. I found a message telling me to go to Customer Z’s DB. There, I saw yet another message telling me to go to this other customer’s DB, then another, on and on. I didn’t know the point of this little fool’s errand, and I replied to the email asking why I was given this task. He didn’t reply but walked back over to where the developers were and stood by my desk. He loudly said, “I know you think I wrote those messages using admin access. I didn’t. Your team is writing s***tty code and they need to do a better job of securing it.” No one knew what we were talking about, only that the former developer turned Security publicly said the team writes bad code. I expressed my thanks and went back to what I was doing, trying to communicate silently that the scenario was at an end. He huffed something and went away.

The next day, as I was showering, I played the scene back in my mind, ignored the insult around the quality of work my team was doing, and focused on the fact that he hadn’t used admin creds to log into the various DBs and write me messages. So how had he done it?

When I got to work that day, I walked from my “lane” (which was like 12 or something) to where his desk was (this would have been lane 2 or 3) and asked him to explain what he meant by not using creds. Then he began to teach me about sanitizing and validating inputs and how he could access the entire DB and web server by abusing the code written by my team as they scrambled to hobble together the least viable functional code, just getting it up and out of the door. He showed me what he was doing, how it worked, and how to rewrite the code to prevent it. I hated to admit it, but he was 100% correct, and we were writing code with weaknesses big enough that you could taxi a 747 through them.

I began learning about what we now call AppSec and started training the developers to write better, more secure code. Again, I hated that he was right because he held that over me for years that I knew nothing of secure code and was managing others, and what a complete schande it was that this was so, especially when I eventually moved to InfoSec and ran into him at DefCon years later.

This reminds me of the point of this blog post. A few years later, I think it was 3 or 4, I was at a new company, again writing web code, and the world was just beginning to utilize API access to scale functionality on web assets (this is for a timeline point-of-reference). I was given the opportunity to move from web dev to Information Security and I’ve missed explaining something in the story, but at a much earlier time, I was a QA engineer, which I LOVED (I’ve since learned that I’m on the autism spectrum, which helps explain why I loved QA and still love InfoSec (much to the chagrin of my wife, I easily see things in binary — the product does what it’s supposed to or it doesn’t (and you need to fix it), or this or that thing is secure or it isn’t (and you need to fix it)). So I was given the opportunity to move to InfoSec, which I saw as a kind of QA (go back a line or two if you missed that part), and was the happiest clam you could imagine a clam being. There was only one problem — I had only one point of reference for how one becomes “The Security Guy” — you have to be an A-hole.

I didn’t *really* want to do that, but I was ready. I knew I would offend people and lose friends, but that’s part and parcel of the job. You become the enemy, and it’s you vs. them, and you’re going to drag them to a more secure place, kicking and screaming if necessary, and it’s all worth it in the end because then the code is better and the customers are happier and it’s a better world when you get there. I was struggling with this, but I was ready to get started. This transition happened in the middle of July, and my boss asked if I knew of any available resources to learn more about InfoSec (my boss knew Linux security, but not much more). I said I’d look around and started looking at social media and learned that two conferences were coming up, and they happened to coincide with one another, something called BlackHat (that sounded like evil hackers for sure) and DefCon (which I remembered was a term from Wargames describing the level of alert/threat around global thermonuclear war). Both of these conferences were happening in a week and a half. I told my boss about them, and he told me to go to the secretary and book a flight, hotel, a rental car, and to get a ticket for Blackhat (through which I could also buy my DefCon ticket — so the evil hacker conference was working with the national threat conference! Spooky!)

When I got to Vegas, as most of you may have experienced yourselves, I dislocated my jaw trying to get my mouth around the fire hose and spent a week in a daze, learning just how wonderful Hacker Summercamp (I learned I completely missed the other event, BSidesLV, but I’d fix that the following and subsequent years) is, and just how many rabbit holes you can find in InfoSec and how the depths of these rabbit holes have yet to be fully plumbed. I saw all kinds of people, heard all kinds of things said, heard other things yelled (I attended the panel talk where Anonymous members were called out and identified, then pursued through the halls by Federal agents), beheld antics in the halls and elevators (someone hacked the elevators in the Rio to lock out all floors not in the Fibonacci sequence unless you knew the code to unlock them (which was also the Fibonacci sequence, I later learned). But for the sake of this story, I was determined to attend a talk that sounded like it was right down my alley as I transitioned to being the Security A-Hole, “Steal Everything, Kill Everyone, Cause Total Financial Ruin!“

This was the last talk of the whole week, and because of a scheduling mixup, it took place DURING the closing ceremonies. But I was ready; this speaker would show me how to be the ultimate Security Guy. He stressed that we should remember the kittens (with laser beam eyes) and that he’s a good guy, he gets paid to do terrible things. He then showed us video of him walking through a hotel in pajamas, apparently lost, and accessing all kinds of things he shouldn’t be. He showed us how he could have burned the hotel down to the ground, by disabling the fire impression and alarm system, possibly injuring many if not all of the people in the hotel by performing these horrible deeds in the middle of the night. He said he could have done these things but didn’t, so we should remember the kittens. He explained how he does these things to show people how to make things better, and this sounded familiar (like the guy from my previous life), but he hammered home the message that we can’t make them feel dumb. We can’t call the employees stupid because they clicked links. We must see ourselves as missing the mark because we haven’t trained them properly. It’s not their fault they make security mistakes, it’s ours.

“We can’t be surprised the humans are the weakest link in our security posture. It’s not because they’re stupid regarding security, it’s because they have had the least investment put into their security training.” — Jayson Street

My mind was rocked. Seriously, there was a resonant dissonance going on in my head. I thought we had to be jerks, but Jayson was telling us that we should be supporting our coworkers and helping them to be better. When Jayson does these physical penetration tests, he *tries* to get caught so they can feel good they caught him, and then he emphasizes all the things they are doing right and how they’ve just got a few things they can do better. He wants them to feel good for having security holes exposed and articulated. Where’s the schadenfreude in that?

This made SO MUCH more sense to me. This sounded how I wanted to run security — not as a monster, but as an ally. I wanted to support my team and be a resource for them. I wanted to show by example and not with punitive insults. I’ve heard Jayson recently say a brilliant observation (which I don’t recall exactly, sorry Jayson), “We can’t be surprised the humans are the weakest link in our security posture. It’s not because they’re stupid regarding security, it’s because they have had the least investment put into their security training.”

As security professionals, it’s our job to make the secure choice the easy choice. We need to make the secure option the obvious one. As long as security is an opponent, an obstacle, an impedance, we will be hated. We don’t have to be jerks about any of it, but if we aren’t *helping* then we’re hurting. The organization’s culture can be greatly influenced by the security team. I want to discuss and explore this more, and I hope you’ll help with such discussions, reader. If we follow this intent, we can make the security team approachable as opposed to intimidating or scary, and then we can talk openly and comfortably about the security questions people have. And if we’re on the same team, they’ll feel better about the security of the org, they’ll trust you more to not make fun of them if they bring an event to you, and you’ll find the security team is almost talked about fondly. Almost.

If you made it this far, I appreciate your tenacity. I hope I didn’t waste your time. I don’t know how to make the point about looking at security this way without the whole story, but I agree it is a lot. Let us not be A-Holes, but support our teammates and encourage them to join our team (everyone is responsible for security) insofar as they can. Show them how they can look at the secure choices differently, and find a way to help them without slowing them down. That’s a big challenge, but it’s one we have to figure out to make our worlds better places. That’s probably the best way we could spend our time here (I may be talking about your day job, I may not).