Office 365 Attack Simulator

Brief Outline

This is the first of a two-part post on the new Office 365 Attack Simulator. This post covers accessing the Simulator and running attacks. The second post is available here.

To jump to a section to continue, click on one of the links below:

  1. Overview
  2. Requirements
  3. Simulating Attacks
    1. Display Name Spear-Phising Attacks
    2. Brute Force Password Attack
    3. Password Spray Attack


When it comes to the security of your environment the best line of defence is one that is reactive versus one that is proactive; however, if you are lucky enough to not currently be target my malicious actors (or even script kiddies) how do you know how you will stand up to a security incident?

Attack Simulator is designed to put organisations that use Office 365 in a position where they understand their weaknesses and in a situation where they can update their policies and procedures with real-world data collected during the simulation. With Attack Simulator companies can run realistic attack scenarios against themselves, to help you identify and find vulnerable users before a real attack impacts your bottom line.

Office 365 recently ran a public preview to allow security professionals (or the people who look after security) within organisations to run multiple attacks internally to discover weaknesses in their environments. The public preview was requested so many times that Microsoft pulled the application a week after the announcement and moved straight to general availability.

Currently, three kinds of attack simulations can be carried out.

  1. Display name spear-phishing attack
  2. Password-spray attack
  3. Brute-force password attack


Unfortunately, and probably due to the rushed release of this product, it can’t currently be discovered in the menu of the Security and Compliance centre of Office 365. It will come eventually, but for now, there are a few hoops that need to be jumped through to gain access to Attack Simulator.

The first major obstacle is licensing. Attack Simulator requires an E5 license. The cost for these licenses are obviously quite high, so hopefully, in the future, Microsoft will add another licensing category just for this.

If you don’t have any E5 licenses, head over to the trial page and grab yourself some for testing.

Another requirement to use Attack Simulator is that the administrator has Multi-Factor Authentication (MFA) enabled. If you don’t, you should, so now would be a great time to complete this.

The next hurdle is permissions. To be able to complete the setup, you will need to be a member of all the available permission roles. Again, hopefully, this is due to the rushed release but Microsoft currently doesn’t make a designated list of required permissions available (even internally), so we need to be in all roles.

To do this, follow the below:

  • Head over to
  • Click Create
  • Give your Role Group a name. I used “Test Threat Permissions” to easily identify it when Microsoft releases the correct guidelines
  • Click Next, and then select “choose roles.”
  • Click Add
  • Highlight ALL roles (select the box next to Name)
  • Click Add, then Done
  • Click Next, and then select “Choose members.”
  • Click Add
  • Place a check next to the ADMINISTRATOR account that will be used for Attack Simulation
  • Click Add, then Done
  • Click Next
  • Click “Create role group.”

Once completed, you should see a group that looks similar to the below:

Once the above is complete, you will have to wait for the permissions to propagate through Office 365. I waited about three hours however Microsoft tells me this could be anything up to a day.

Once you have waited, head to:

This link will take you directly to the Attack Simulator page. It’s a good idea to bookmark this until Microsoft add it to the Threat Management sub-menu of the Security and Compliance section of Office 365.

Once here, you will be greeted by an option telling you further configuration is required. Click the setup now button.

If it fails and comes back with the below, you need to wait more time for the permissions to propagate.

Simulating Attacks

As previously mentioned there are currently three kinds of attack simulations can be carried out.

  1. Display name spear-phishing attack
  2. Password-spray attack
  3. Brute-force password attack

These are covered in the following sections.

Display Name Spear-Phishing Attacks

Selecting attack details on this explains the why of spear phishing.

Phishing is a generic term for a broad suite of social engineering attacks. Social Engineering, in the context of Security, is the art of manipulating people into performing actions or divulging confidential information. This type of confidence trick is often used for information gathering, fraud or computer system access. If successful, Phishing attacks can often lead to a more complex attack. The end objective generally falls into one of three areas: Financial Gain Espionage Reputation There are a multitude of Phishing variants, however most Phishing attacks have a common structure: Source, Payload and Target. 


Display Name – Spear Phishing

To carry out this attack simulation, select “Launch Attack” on the main page. This will bring up a context menu to navigate you through the process.

The first section is naming the attack (for later reference) as well selecting a template to use.

Currently, there are only two templates to select from, which are:

  • Prize Giveaway
  • Payroll Update

You don’t have to select one of these, but for testing purposes I chose the “Prize Giveaway”

The next section allows administrators to select the target users for this campaign. Remember, multiple campaigns can be run in order to attack different people with different approaches.

Once the targets (or victims) are selected, we move on to how the message will appear to the users.

The only thing modified here is the From email address and the Custom Landing Page.

Office 365 makes several “Phishing Login Servers” available to allow you to select one that is close to the original purpose as possible. I would imagine as the number of templates increases, so will the number of URL’s that are available for selection.

For this demo, I’ve chosen to send the email from “[email protected]” with a landing page of (which is merely the WordPress short link for this page).

One thing worth noting here is the simplicity of the From email address. Having used several other programs which do very similar jobs to this, Microsoft has made sending it from any email address exceptionally easy.

Typically, due to email monitoring programs and policies the sender’s email address must be correctly entered and have SPF records correctly assigned, or the mail will be dropped as Spam at the border. As this system works inside the edge, it allows any From email address to be entered.

After this, it is simply a matter of confirming you wish to run the campaign.

After a few moments, your target users will receive your crafted email:

Navigating to Attack Details for your campaign (or, going to this URL will allow you to monitor the progress of your attack.

The user experience

As with most phishing techniques, certain things will give away the fact that things are not what they seem.

The first is the URL in the link

If you sent the email from this wouldn’t be so bad, but as my test came from Instagram users should be looking to be directed to one of Instagrams servers. Unfortunately, with most legitimate mailing campaigns, this isn’t always the case.

The next is clicking on the URL brings you here:

Which, as you can see is a HTTP (not secure) link asking for credentials. It looks like the Microsoft portal login (which, Office 365 users will be familiar with) however the URL is still “” which would never appear for a Microsoft website.

Here, if you test it yourself, enter your email address but put a RANDOM password in (it’s insecure, and the wrong website anyway!) and Microsoft will move you onto the redirected URL you selected earlier (in my case, the WordPress short link).

Going back to the Attack report, you will see one of your targets has been silly enough to click through the link and expose their credentials.

Incidentally, should you use this in production and a user does fall for it, please ensure they change their password immediately. It has just been sent un-encrypted over the internet!

One thing to note, which was discovered during these tests, was the fact that Firefox is automatically tagging the website we used as deceptive. If your primary browsers within your company are Firefox, you may need to add an exclusion or risk seeing something like this:

which would give the game away completely.

Note, I forgot to screenshot the deceptive site page, so the one above is from a google search, and not the website Microsoft redirected us to.

Brute Force Password Attack

Selecting attack details explains the why for brute forcing passwords.

Password cracking techniques are used to guess the user’s password by trying many variations with a computer. Since most companies with substantial numbers of users, or complex application ecosystems end up running a hybrid or federated domain model, an on-prem account and an Office 365 account will have the same credentials and will represent a single weak point around the services accessible by the affected user. Once the attacker has the username and password for the user, they will generally be able to find a way to authenticate and interface with Office 365 as if they were the end user. Brute force attacks work by calculating every possible combination that could make up a password and testing it to see if it is the correct password. As the password’s length increases, the amount of time, on average, to find the correct password increases exponentially. This means short passwords can usually be discovered quite quickly, but longer passwords may take decades. 2 types of attack typically exist, a dictionary attack using a well-known dictionary list of passwords and an exhaustive attack, where combinations are tried sequentially.


Brute Force Password Attack

Once you select the Launch Attack button for this type of attack, as before, a context menu will appear guiding you through the rest of the process.

The first option is entering a friendly name for the attack.

Secondly, we target the users we want to investigate.

Once we have our list of targets, we specify what passwords should be attempted against their usernames.

This gives us two options.

  1. Enter passwords manually
  2. Upload a password list

At this moment in time, even if you upload a password list, you must still manually enter at least one password. To do this, enter the password and then press Enter. The password will be added to the list (indicated by dots replacing the actual text).

If you also wish to add a password list, I would recommend head over to Daniel Miesslers secList github

For this test, I grabbed the file and copied to my workstation, then uploaded to Office 365.

Then, confirm you are happy to proceed with the testing.

The attack can take a couple of minutes to several hours to complete. Once it does, you will be able to click on “Report Available” within the Attack Simulator portal to see if any users were compromised:

This attack is not visible to the users.

Password Spray Attack.

Selecting attack details explains the why for password spraying attacks.

A password spray attack against an organisation is typically used after a bad actor has successfully enumerated a list of valid users from the tenant, utilising their knowledge of common passwords used. They attempt one(1) carefully crafted password against all of the known user accounts (a one to many attack). If the attack is not successful at first they will typically try again utilising a different carefully crafted password, usually waiting longer period between attempts so as to not trigger any policy based account lockout triggers.


Password Spray Attacks

This attack is very similar to the brute force attack however utilises the one-to-many methodology explained earlier.

As with the brute force attack once you select the Launch Attack button you will be given a context menu that guides you through the rest of the process.

First we need to enter a friendly name to remember this attack.


Next we select the users. Remember, this is supposed to be a one-to-many attack so select multiple users.

Next, we enter the password we wish to test against.

If you have a standard new starter password of something like “Welcome1”, this would be a great start to discover if users have changed their passwords since. Passwords like this are the primary focus of attackers when users this type of attack.

As with the other attack types, you must confirm the settings before starting the attack.

As with the brute force attack, users will not see anything that makes them aware that this attack is being carried out.

Once it is complete, you will be able to view the report from the Attack Simulator portal.


It’s worth pointing out that there are a number of very good paid alternatives to the attack simulator which give a lot more options for individual attacks however Microsoft’s integration into Office 365 ensures these features are easy to use and accessible, providing the you meet the licensing requirements!

As always, if you need any help with any of the items discussed here, don’t hesitate to contact us.

One thought on “Office 365 Attack Simulator

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.