Update 29.3.2020:
With this update, end-users will now be able access phishing email within Quarantine. However, only admins, not end-users, will be able to access High confidence phishing email within Quarantine. (Message center MC200232)
Microsoft has added a category named “High confidence phish” (MC193076) and now after this change (MC200232) they are finally allowing the self service quarantine management for the phish category. Change should be rolled out by now. This solves the issue originally described in this post.
CHECK OUT MY NEW BLOG SERIES!
Result of this series as three minutes Youtube video: Data from home – Info Display
Details for the display: Controlling Samsung Signage QB13R-T based on Unifi Protect
Data from home – setup and preliminary results (Part 1)
Data from home – wireless water meter (Part 2)
Data from home – energy analyzer, requests against webUI (Part 3)
Data from home – district heating, computer vision (Part 4)
Data from home – alerting and water shutoff (Part 5)
I hope you enjoy!
Microsoft 365 has a bunch of ways to protect the users from unwanted or even malicious email. In this article I will write mostly about the Office 365 ATP. I guess pretty much every one using email, wants to see only relevant and safe messages. For administrators, the burden is even heavier since you are trying to protect your end users. This might be a quite large group of people with varying capabilities on secure email handling.
Security automation includes balancing between how well you can filter and how much false positives you get trying. What comes to help in context of email is a self-service quarantine. This means, that in specified cases user won’t get the email to his inbox but instead gets a notification that something got filtered out. Usually (and the same applies also for Office 365) this notification is not delivered for every message but instead for example once per day.
For Office 365 you have a different kind of categories of filtered messages (phish, spam, bulk, malware and high confidence phish) for which you can configure the desired action. It might seem easy and safe to deliver possible false positives (not high confidence results) to the users junk folder, so those messages are available immediately. On the other hand, junk folder is somewhat problematic and quarantine notification might seem in a way to be easier for the end user to handle. This is were the problems start.
Quarantine notifications are easy to configure. Testing it with SPAM should be easy with GTUBE test. But did you remember to test what happens with phish category? That one should be possible with GTPhish. Unfortunately I have been having issues using these tests.
Based on Microsoft’s documentation and FAQ “End users can access their own phish, spam, and bulk mail.” from quarantine:
Q. What messages can end users access in quarantine?
A. End users can access their own phish, spam, and bulk mail. End users can’t access their own malware, high confidence phish, or messages that matched a mail flow rule (also known as a transport rule); these are only available in the admin quarantine.
The truth seems to be that phish category is not available through the self service quarantine but those messages end up to the admin quarantine instead. That means the original recipient won’t be notified in anyways. This comes to be even more problematic with the fact that Office 365 ATP Phishing protection seems to be quite aggressive.
Below you can find a Powershell snippet to report a relative amount of different types of quarantined messages in your environment. Remember to connect to Exchange Online before running this.
# Preset limit for max result size. Caused by pagesize this will be rounded to next multiple of 1000
$maxResults = "50000"
$quarantined = $Null
# Get-QuarantineMessage gives out at max 1000 results per page so we want to DO - WHILE
$currentPageNumber = 1
Do {
$currentPageContent = Get-QuarantineMessage -Direction Inbound -Page $currentPageNumber -PageSize "1000" | select-object type
$quarantined += $currentPageContent
Write-Host "Reading page $currentPageNumber, objects found $($currentPageContent.count)"
$currentPageNumber++
} While (($currentPageContent.count -gt "0") -and ($quarantined.count -lt $maxResults))
# Writes output formatted to show percentage of each type found
$quarantined | group-object -NoElement -Property type | sort count -Descending | Select @{l="Type";e={$_.Name}},@{l="Count";e={$_.Count}},@{l="Percent";e={“{0:p2}” -f ($_.count / $quarantined.count)}}
Results might look like something below. “Phish” and “High Confidence Phish” are reported separately. Never the less the end user experience shown here is the same for both types.
Type Percent
---- -------
Phish 83,93 %
Spam 8,99 %
High Confidence Phish 6,69 %
Malware 0,38 %As an example here is a view from the admin quarantine. This message was sent to me. Make a note, reason is marked as a “Phish”, not as a “High Confidence Phish”.
Based on the documentation the messages with “Phish” categorization should be available from the self service quarantine. This is a view from my self service quarantine.
Are you getting my point? In the worst case most of the filtered contents (including most of the false positives) might end up in the admin quarantine. Then its up to your processes, local legislation and your company policy if there is anything you can do for those. The biggest problem is that it seems not to be working as documented.
Remember to test and understand your settings and how they affect your users and business!
Just because I/1 can 