Office 365 Secure Score

Here at Complete Cloud Solutions we’ve long been fans of the Office 365 Security and Compliance settings. They offer a great overview of options to investigate and protect assets your company has in the cloud.

Todays blog is to cover an often overlooked aspect of the Security and Compliance page, namely the Office 365 Secure Score.

If you’ve never heard of it, below is Microsoft’s take on Secure Score

Ever wonder how secure your Office 365 organization really is? Time to stop wondering – the Office 365 Secure Score is here to help. Secure Score analyzes your Office 365 organization’s security based on your regular activities and security settings and assigns a score. Think of it as a credit score for security.


Secure Score Introduction

So, what this means in real terms is Microsoft produce a large list of best practice scenarios and aligns how your organisation is doing in terms of these best practices.

When opening Secure Score the first thing you will probably notice is your ‘score’. This is displayed alongside your predetermined maximum  score. It is possible (and in fact probable) that your companies maximum score is different to the one shown below as this score is decided by the security baseline Microsoft align your company too.

Below your score is a sliding bar that allows you to set your companies target, followed by a list of recommendations to assist you to achieve your target. As you slide the target to the left or right, the list of recommendations changes in order to facilitate the best approach to your desired score.

The list of recommendations is ordered by:

  1. Highest is the items that affect the score the most, with limited user impact.
  2. Lowest is the items with least impact on the score, but higher user impact.

The right hand panel has a couple of important items too. 

The first is a Risk Assessment that your organisation faces while at your current score.  Each item displayed as a risk can be selected to allow you to easily understand what problems you are facing.

The second item is a score comparison chart for organisations Office 365 believes are similar to your own. This allows you to track your score against other companies to see whether your security standing is better or worse.

So, after reviewing a single page I now know:

  1. My current security standing is approximately 10% of what it could be
  2. I need to perform 22 actions in order to reach my ‘goal’ of 299
  3. I am doing better than the other companies similar to me, as well as the average Office 365 tenant
  4. I’m currently at risk of Account Breach, Privilege Escalation and Data Ex-filtration

So, not a great start, but not terrible either. So, what exactly does Office want me to, to increase my score and just how hard are these on busy organisations.

Let’s look at the first four, and understand what business impact it will have.

So, this tenant currently has Global Admins that don’t have Multi-Factor Authentication enabled for them. 

This is only complaining about Global Admins at this point, people who have privelages to do anything in Office 365. These are generally dedicated IT accounts (and should not be assigned licences) and only used when changes to the tenant are required. If these are configured correctly, the only impact this has is to the IT user who needs to occasionally (or even regularly) log in to perform one task or another. 

Impact on normal, everyday users. None. Therefore, impact on business should also be none.

And, as the information displayed informs us, this will assist with our two main problems of Account Breach and Privilege Escalation.

OK, so next up is:

Multi-factor authentication for all users. Experience tells me this one is harder for many businesses to stomach however the advantages outweigh any potential user adoption problems. As shown, this also helps with Account Breaches and Privilege Escalation. 

As you will probably see, in the environment I am currently looking at there is 6 user accounts which will increase my secure score by 30 points, meaning 5 points per user. This is also another way to assist you to track the adoption of MFA as this score will increase as each user is activated.

The next one covers the Data Ex-filtration problem:

This one is a little more complicated to understand and implement however usually this would again generally not have any impact on standard users.

What this alert is actually complaining about is the ability for users to create a rule that automatically forwards their emails to external addresses. For example:

  1. Dave works as a finance analyst
  2. Dave receives an email from the CFO with a list of bonuses that people received over the last financial year
  3. As the email comes in, a rule in Dave’s mailbox forwards the email to [email protected] 
  4. Dave is systematically offloading all emails with attachments to a none authorised email address, outside of the company

Usually, there is no reason for users to do this for legitimate purposes and it’s generally regarded as a good idea to automatically prevent it globally, then add exceptions to users that have a valid business requirement.

Enabling this setting will create a transport rule that will stop external messages leaving your Tenant, that are of the type AutoForward, mitigating the use of Client created external mail forwarding rules and malicious Remote Domain entries as a data exfiltration vector.

  1. If The Sender is located ‘Inside the organization’
  2. If The Recipient is located ‘Outside the organization’
  3. If The message type is ‘Auto-Forward’
  4. Reject the message with the explanation ‘External Mail Forwarding via Client Rules is not permitted’

 OK, so, lets look at the last one:

This one seems obvious however is another topic that is often overlooked. Simply by regularly reviewing what are classed as ‘risky logins’ increases your score by 45 points. That’s 1.5 times the average Office 365 score!

These Risky Logins show:

  • Users with leaked credentials
  • Sign-ins from anonymous IP addresses
  • Impossible travel to atypical locations
  • Sign-ins from infected devices
  • Sign-ins from IP addresses with suspicious activity
  • Sign-ins from unfamiliar locations

 Which every organisation should definitely be tracking on at least a weekly basis. I would also recommend enforcing MFA for users that are discovered as ‘risky logins’. This has the possibility to disrupt some users however will give you maximum protection should you not have MFA enabled.

So, enabling these in my test environment took a grand total of ten minutes. Real world will probably take longer however most of the time would be dedicated to change controls and business adoption. 

These four topics should increase the score from 36 to 181, with only MFA being a user affecting item. This is achieved from:

  • Enable MFA for all global admins gave 50 points
  • Enable MFA for all users gave 5 points per user
  • Enable Client Rules Forwarding Block Advanced Action gave 20 points
  • Review signs-ins after multiple failures report weekly gave 45 points

Once they are done , you will the see your score increase after approximately 24 hours.

One thing to note is the score took approximately 5 days to update. You’ll see within the chart the last update time was March 13th, 2018. The previous update time was March 9th. 

The Microsoft documentation says this score will update every 24 hours, however, real world experience is generally a longer than this.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.