Office Attack Simulator – Part 2

This post is part two in a series looking at Microsofts Office 365 Attack Simulator.

Part one looks at the ability Office 365 administrators have to be able to run three types of attacks on their users. These attacks are:

  • Spear-phishing
  • Brute Force
  • Password Spraying

For part one, click here.

Part two will focus on high-level mitigation’s for these types of attacks.

If you wish to jump to a related section, click one of the below links:

  1. Spear-phishing
  2. Brute Force
  3. Password Spraying

Spear Phishing

Spear phishing is an email-spoofing attack that targets a specific organisation or individual, seeking unauthorised access to sensitive information. In most cases, that confidential information is the user’s password to give attackers a platform on which to expand their efforts.

Spear-phishing attempts are not typically initiated by random hackers but are more likely to be conducted by perpetrators out for financial gain, trade secrets or other specific information you, or your company, have access too.

This type of targetted attack means that the perpetrators have usually investigated your company sufficiently to be able to pinpoint the exact user on which they wish to focus. That increased effort makes it harder to spot a spear-phishing campaign from a regular one that is generic to thousands of recipients. Often, attackers will use language and terminology the target can relate to, overcoming the trust barrier.

This focused effort makes identifying spear-phishing campaigns from a technology perspective is very hard. With most non-targetted phishing campaigns the following best practices from a technology perspective apply:

Mail Filtering – use a 3rd party mail filter in front of your email servers. This recommendation follows the defence in depth methodology, meaning, if you are using Office 365 Exchange Online Protection, consider adding additional services in front of this such as Mimecast. The more filters emails pass through before reaching the recipient, the higher the chance of full sanitisation before it arrives. Obviously having ten mail scanning applications would offer fantastic peace of mind. After the third and fourth application, however, the benefits would start to become minimal from each additional product. The cost of such an approach should also be considered when investigating.

Reputation-Based Web Filters – These can be on your firewall or installed directly on your endpoints (usually from anti-virus products). These will add an additional layer of protection between the user and the internet. Reputation-based filters will review websites against a known bad list to assess whether it is safe to allow the user to visit them. That bad list can be for multiple reasons similar to what traditional proxy servers manage. However, they can also prevent users from accessing sites they have detected as phishing. Once a user clicks on a known phishing link, the application intercepts the browser traffic and displays a notification to the user telling them they cannot go to the link provided, and why.

Unfortunately, the above two methods usually won’t stop a spear-phishing campaign against your organisation. Both systems rely on multiple triggers to alert them to problems, as well as large amounts of information. If a malicious actor targets your company, there is a high chance that you are the only company receiving the email aimed at you, and the URL will be unique. This unique method efficiently makes reputation and analysis based investigated almost redundant (in this scenario).

It is now commonly accepted that attacks of type will get through your defences. There is currently no technology that guarantees you 100% of phishing emails will be cleaned before being delivered. That leaves a single object between your company and a possible breach. The user.

This is why training your users to understand this type of attack is your best chance of defence against it. There are some excellent guidelines online, however, in general users should be looking for:

  • Correspondance from users who are asking for things out of the normal.
  • URL’s to click that are misleading (text reads, URL is
  • Emails explicitly saying don’t get in touch, I am unavailable but process this while I am out of reach
  • Emails that are suddenly urgent that you were unaware of until the email arrived.

An excellent way of measuring this type of awareness is training courses through companies such as knowbe4. They offer online courses and a mini quiz to ensure your users are fully versed in the potential hazards out there.

And, of course, test your users yourself, which is were the Spear-Phishing attack in Office 365 comes in. If you keep your users informed and regularly tested with real-world trials, this will keep awareness raised ensuring users are less likely to fall prey to this type of attack.

Gamification is a perfect play for this. Start to reward staff, create a leaderboard of sorts for people who managed to identify and report phishing campaigns of any type correctly. It could be added to monthly meetings with a small prize for the best one discovered that month, or the most discovered. All of this helps keep users interested and vigilant.

Note – Symantec recently released their “Internet Security Threat Report” for 2017. This shows that Spear-phishing is accountable for 71% of the overall attacks being conducted against organisations. That’s an incredible statistic and highlights the rapidly growing area of concern. It also shows a year on year increase of this attack vector.

Brute Force Attacks

Brute Force attacks have been around for possibly as long as the internet. This type of attack involves finding an exposed authorisation point and then continually trying passwords against a known username (or, sometimes, rotating through commonly used usernames).

Thankfully, unlike Spear-Phishing, this is something that technology has developed numerous solutions to discover, investigate and prevent unauthorised attempts of this type.

As the Attack Simulator focuses on Office 365, this document will highlight how to discover and thwart these attacks specifically in Office 365.

Multi-factor Authentication (MFA) – This is the process of having an additional layer of security after correctly entering your password (or, in some cases, before you input your password). Enabling MFA for users is a guaranteed way of preventing brute force attackers from successfully gaining access to the environment that provides MFA, should a password be guessed. However, this does mean that you have exposed user credentials, as once a malicious actor discovers that the password is accepted, they can then start to look for exposed applications that do not support MFA. Clearly, the scope of protecting Office 365 would be covered, but if the attacker finds a Remote Desktop Server that does not have MFA enabled, they then have a gateway into the environment. This drawback means MFA should not be relied upon solely.

Account Lockout Policies – There are a couple of ways of addressing this. The first, if you are on managed logins (not using Active Directory Federation Services (ADFS)), then Microsoft goes some way to watching out for Brute Force attacks. They also prevent users from password recycling from lists have previously been exposed through hacked website leaks.

For more information on password recycling and some cool tricks around it, click here.

If Microsoft detects that one of your users is being brute force attacked, they will lockout the account, as below:

This works in conjunction with MFA, which is perfect.

The next method is used to address organisations that are using AD FS, and again there are two primary ways to tackle this problem.

The first is Active Directory. Having a lockout policy in place in Active Directory will undoubtedly stall brute force attempts on your users. If brute force scripts only get to try three passwords every thirty minutes that pushes up the average time to find a password up exponentially.

It will also be highly visible, as, if a user is always being locked out, someone would be alerted to the fact. Unfortunately, this is also the downside to this method. The companies users are what drives the business, so having them locked out every ten minutes can drop productivity through the floor. And, you know it will be at the worst possible time for the user.

Investigating why accounts are being locked out also takes significant time as it could be one of (sometimes) hundreds of different systems that are locking out the account, either internally or externally.

This brings us to method two if you are using AD FS, Extranet Lockout.

With the extranet lockout feature in Windows Server 2012 R2, an AD FS administrator can set a maximum number of failed authentication requests (ExtranetLockoutThreshold) and an ‘observation window’s period (ExtranetObservationWindow). When this maximum number (ExtranetLockoutThreshold) of authentication requests is reached, AD FS stops trying to authenticate the supplied account credentials against AD FS for the set time period (ExtranetObservationWindow). This action protects this account from an AD account lockout, in other words, it protects this account from losing access to corporate resources that rely on AD FS for authentication of the user. These settings apply to all domains that the AD FS service can authenticate.

So, setting the ExtranetLockoutThreshold below the Active Directory lockout threshold alleviates the initial Active Directory lockout policy problem and allows quicker discovery of what is causing the problem. It does this by enabling the user to continue working internally. However, they would have problems accessing the external portals provided by AD FS, as they would be locked out.

Log Monitoring or Cloud App Security – Both of these items amount to the same thing. However, Cloud App Security is automated, whereas monitoring the logs is usually a manual process. For those fortunate enough to have Cloud App Security checking for Brute Force attacks should be easy.

The portal allows the creation of alerts via email and text message to administrators should certain conditions be encountered.

The image below shows and invalid password attempt registered by Office 365:

When the task is expanded, select the “New policy from search” button, to open policy editor.

Complete the top section, then under policy filters and alerts, configure as below:

Obviously entering your own 24/7 details into the email address and telephone number. 

This will ensure your on-call staff will be alerted should any user start to get locked out from your cloud services. 

A less streamlined way is log checking. This can be done manually or through automated scripts (such as PowerShell).

We won’t cover how to do that in this post (one for the future) however guidance can be found elsewhere on the internet. A good starting point to investigate is:


These are currently a little tricky to test due to the fact you should be using MFA for your logins however an internet search should assist here. As mentioned, we’ll cover all of this in another blog in the future.

If you wish to review the logs manually, you can do so by:

  1. Log in to Office 365 as an administrator
  2. Navigate to Admin Centres, Azure Active Directory Admin
  3. Select Users
  4. Select Sign-Ins

This page will allow you to traverse the log files adding filters for failed logins etc.


Password Spraying

Password Spraying is a difficult topic to address and protect against, mainly due to the secrecy necessarily people have around discussing passwords. How are users ever going to discover they have the same password without knowing what other people are using?

Unfortunately, it’s also one of the hardest to defend against as most people who are performing spray attacks will understand that you have account lockout procedures and complexity requirements on your passwords. This knowledge means, they will only try spray attacks at intervals to not lock-out the targetted accounts.

There are always multiple ways to harden against any security risk, however, this post will cover the following options:

  1. Password complexity and reset options
  2. Help-desk awareness
  3. Password recycling evasion

Password complexity and reset options A basic defence against to attempt to ensure users are not using the same password is the complexity requirements. As complexity requirements increase the likelihood of users using the same passwords decreases. Unfortunately, this adds overheads as, if you have to renew passwords every 45 days, this makes it difficult for users to manage,

One thing to remember about password complexity requirements and password reset times is they should be used in conjunction with each other. As you relax the complexity requirements, your users should be resetting their passwords more often. Increased complexity requirements give you the ability to extend the time they use their passwords for.

To understand this, it’s often worth thinking of how long your password requirements would take for a malicious actor to crack using standard brute force techniques. If your passwords are good for two hundred days, setting your passwords to expire after one hundred and eighty days (or every six months) would still leave you protected without inconveniencing your users too much.

An excellent resource to test your base requirements is by Dash Lane. The below image shows how long brute force would take using a typical password generator to crack [email protected]

This password meets all of the default Windows complexity requirements, but allows attackers to gain access within nine hours. Not great. Increasing the requirements to twelve characters, rather than eight, shifts this time exponentially and therefore would allow your users to change less often.

Help-desk awareness

Another great way of avoiding Password Spraying attacks is to train the help-desk staff to generate unique passwords every time a user requests a password reset or a new account is created.

For example, a malicious actor could follow the below to attempt to discover a crafted password spray:

  1. Malicious actor calls the help-desk with a known user account
  2. Help-desk staff reset the account to “Welcome01” and set the account to not require a reset at login (as the user is external only)
  3. Malicious actor now knows there will be multiple accounts that users haven’t reset as they have no facility to do externally.

The malicious actor now has a perfect Password Spray candidate that is more or less guaranteed to work. The actor doesn’t necessarily need to go as far as calling the help-desk, they could discover it sat in Starbucks or talking to any employee.

If help-desk staff are trained to be aware of this problem, they can generate individual passwords (possibly using a password generator) which ensure if a malicious actor does get a remote user credentials, it is limited to that one remote user allowing easier discovery and lock-down after the subsequent incident is discovered.

Password recycling evasion

This technique requires significantly more investment than any previous option. To understand password recycling, and get basic information on how to test for it, click here.

One way of implementing this in a real world scenario is adding checks into the 
Windows Password Filtering.

The Windows Password Filter feature allows you to add additional checks when a user attempts to pick a new password. The password first goes through the domain’s password complexity check. After passing the complexity check, it then gets passed through your custom check. You can have multiple checks if you’d like. The image below shows the process.


This allows organisations to deploy mechanisms to users that allows password recycling to be checked prior to the password being reset. 

This decreases the chances of password spraying attacks being successful however can be difficult to implement without an internal development team. Thankfully, as this problem is increasing in significance, vendors are becoming more focused on addressing the issue and there are a variety of free and pay to use products that will allow your company to implement these filters without writing your own.

As with everything else, if there is anything on this page you would like more information on or assistance with, get in using the form below.







Send a Message

One thought on “Office Attack Simulator – Part 2

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.