Much like open offices and outsourcing in business, information security is subject to trends. One you probably saw in your vendor spam folder over the past couple of years is phishing awareness exercises.

The premise sounds simple – phish your employees before the bad guys do, monitor how they respond, and react accordingly.  In reality, people’s experiences have been more complex. There’s not much middle ground in the discussion of phishing exercises. I see either glowing articles praising their merits (most of which are selling something), or bemused cynicism about them from security professionals. In my experience, there really can be benefits to running phishing test exercises in a sensible way, but many organizations are not implementing them in a sensible way, so they end up pretty worthless.

When you’re setting up a phishing test program, you have the option of developing your own phishing exercise infrastructure and metrics collection toolkit, combining open source solutions like King Phisher or (SET), or purchasing one of many available commercial solutions. I won’t advocate for one brand over another in this blog – most will work (in the right configuration and conditions). A similar set of concerns exist, whether you develop a deployment and metrics solution, or you buy a commercial solution in a box. Let’s discuss how any and all of these tools are being used incorrectly.

 

Before spending money, or implementing anything

Develop a clear goal for your program with your senior leadership fully involved. This goal should not be, “stop employees from clicking on phishing messages”. That’s simply unattainable. Yeah, you want that number to decrease, but even top security professionals have fallen for well-crafted phishing messages. People click on things when they’re busy and distracted, and it theoretically only takes one compromised host to breach a network. A real attacker only has to get that one, inattentive click. If your senior management measures your success by phishing clicks reaching zero, you’ll ultimately find yourself dumbing down campaigns to look more successful. This won’t do anybody any favors.

A more realistic goal is improving the quantity and speed of reporting of suspicious emails. Detecting phishing with tech is hard. Most organizations spend a great deal of money on modern solutions to catch and alert on phishing messages, and even those can be circumvented. Your last line of defense against phishing and social engineering is a good relationship with end users who will promptly tell you they are being attacked. While it takes only one phish to compromise a network, it takes only one prompt report to security to shut an attack down.

Next, you should bring your HR and Legal teams into the conversation and discuss anonymity. There is no room for gray area here. You will either conduct phishing exercises anonymously or you will not. If you conduct the phishing exercises anonymously, you must develop the program in a double blind way where even network security can’t practically retrieve the names of people who clicked. You’ll still see an overall view of the health of your organization, but nobody can be pressured to provide identifying data, even by angry executives.

If you choose to not conduct exercises anonymously, I recommend that you clearly document any repercussions for clicking, and ensure they are uniform across your organization. Otherwise, your exercises could easily become a public humiliation game or end in unequal punishment by managers, putting you in hot water with HR.

 

A carrot, instead of a stick

Regardless of if you conduct your exercises anonymously or not, you may decide to provide extra security training to people who click on your test phishes. Frankly, a lot of security awareness training is pretty awful, “death by PowerPoint” stuff. If your users can fly through every slide and kludge their way through your multiple choice test, chances are it’s a waste of time. Try to have some empathy for how an end user is feeling when they click on a test phish and are routed to a long, mandatory training. They’re embarrassed, frustrated, and it’s very possible they clicked because they were already frantically busy. In their minds, you aren’t helping – they feel like you tricked them. There’s now hostility in your relationship, not a willingness to help “the team” stop attackers.

If possible, in-person training is a great option (snack bribery highly encouraged). Offer a lunch and learn, or a social hour with IT security. Offer this in lieu of traditional web-based training, and have a conversation with your end users. People are statistically more inclined to help somebody they have met in person and feel some connection to. You want to try to make your phishing exercises a positive thing that people want to improve, not a negative thing that people subconsciously associate with punishment or embarrassment.

If training has to be computer-based, try to make it quick, effective, and interactive. This is a space where you may wish to spend some money to get something good quality and enjoyable.

Be clear about what you’re trying to accomplish with phishing exercises and why they are important to your organization. Ensure you give credit to people who report phishing and help your team improve more than you punish people who make genuine mistakes. It’s better to provide measures to protect victims and help them learn, rather than encourage them to circumvent your security team.

 

Who should you phish?

Establish the scope of your exercises. Must certain employees be exempt for legal reasons? Are multiple languages spoken in your organization which will require separate exercises? Will your exercises be conducted across global business hours and all shifts? Have you done some OSINT to generate a list of exposed users and email addresses that require special attention?

I highly advise against phishing everybody at once. The only things that travel faster than light in workplaces are rumors. Once one person realizes he or she has fallen for the phishing exercise, it’s nearly impossible to contain the “helpful warnings” to neighbors and friends. This is good, but won’t necessarily give you accurate metrics about individual performance.

 

Designing your phish

Security teams everywhere look forward to this part with glee. I must remind my blue team friends of a lesson that successful red teamers learn early in their careers: your job is not to “get” your target for the laughs. Your job is to educate your target and improve their security. You are on their team. Yes, you can phish nearly anybody with a well crafted message and insider knowledge. Conversely, you can produce excellent metrics by selecting an absurdly easy phish. Neither results in any significant security training.

Your phishing exercises are a scientific experiment, and a good experiment has as few variables as possible. The variables that do exist must be well quantified, and should include the difficulty of the phishing message, which is easier said than measured. Comparing clicks on an excellent phish with perfect grammar and a timely topic to one that applies to few employees and is written in poor English is apples to oranges. If you want to change the variable of phishing difficulty, do not change the variable of employee selection or time of day, and vice-versa.

If you’re having trouble with this, look to your phishing awareness training. Most commercial training programs list warning signs of a phish. When developing your messages, choose a specific number of these specific warning signs to include.

 

Avoiding phishing-related divorces, and other unpleasantness

Writing a phishing email seems fun and easy. You copy one you’ve seen in your filters, or use a common phishing theme, and send it out with a link or attachment, right?

Or not.

Bad guys have it a lot easier than us, as defenders and pen testers. Bad guys can emulate any public company or person they want in their phishing messages, and abuse any emotion. While we want to make test phishes as realistic as possible, there are good reasons why we have to put more thought into ours.

The reaction of a human being to a phishing email depends on a lot more factors than just their corporate security training. They’re also influenced by their outside security education, their biases and experiences with the content of the message, and their emotions. Imagine a phishing test email that uses the classic “payment received” scam, ostensibly from some real online payment firm. Some people will look at the phish, see it for what it is, and report it appropriately. Others will Google the payment provider and report the phish to them instead; a black eye (or even a blacklist) for your company. In a worst case scenario, an employee could receive the message and apply a personal context, forwarding it to their spouse as ‘proof’ they’re hiding money.

You must try to keep your phishing exercise contained. Remember, you are handling live lies. Not only could forwarding of your test message alter your metrics, but it could also result in more dire legal or ethical consequences if it should leave your network perimeter. Ensure you thoroughly prevent this, and clean up after your exercise as soon as possible once you’re done.