Dozen new Azure AD/Office 365 roles have been added

While we still haven’t got any traction on the custom roles for Azure AD/Office 365 front, a bunch of new admin roles was introduced recently. The actual number seems to be 12, as listed below:

  • B2C User Flow Administrator – Can create and manage all aspects of user flows. That is Azure AD B2C lifecycle flows, not related to Microsoft Flow ūüôā
  • B2C User Flow Attribute Administrator – Can create and manage the attribute schema available to all user flows.
  • B2C IEF Keyset Administrator – Can manage secrets for federation and encryption in the Identity Experience Framework (IEF).
  • B2C IEF Policy Administrator – Can create and manage trust framework policies in the Identity Experience Framework (IEF).
  • External Identity Provider Administrator – Can configure identity providers for use in direct federation.
  • Compliance Data Administrator – Creates and manages compliance content.
  • Security Operator – Creates and manages security events.
  • Kaizala Administrator – Has full access to all Kaizala management features and data, and manages service requests.
  • Search Administrator – Can create and manage all aspects of Microsoft Search settings.
  • Search Editor – Can create and manage the editorial content such as bookmarks, Q and As, locations, floorplan.
  • Printer Administrator – Can manage all aspects of printers and printer connectors.
  • Printer Technician – Can manage all aspects of printers and printer connectors.

The two roles on the bottom have the same description, that’s not a copy/paste error on my end ūüôā

As you can see, we now have roles dedicated to managing Microsoft Search, as well as Kaizala. We also have scoped down roles for parts of the functionalities exposed in the new Security and Compliance centers. And yes, printer connector related roles, whatever that might be.

The four B2C roles are already available in the Azure AD blade, where you can get more detailed description on them, as well as granular list of role permissions. The same is true for the External Identity Provider Administrator role, which is probably the most interesting one. It has the following description:

This administrator manages federation between Azure Active Directory tenants and external identity providers.‚ÄĮWith this role, users can add new identity providers and configure all available settings (e.g. authentication path, service id, assigned key containers).‚ÄĮThis user can enable the tenant to trust authentications from external identity providers.‚ÄĮThe resulting impact on end user experiences depends on the type of tenant: (1) Azure Active Directory tenants for employees and partners:‚ÄĮThe addition of a federation (e.g. with Gmail) will immediately impact all guest invitations not yet redeemed. (2) Azure Active Directory B2C tenants: The addition of a federation (e.g. with Facebook, or with another Azure Active Directory) does not immediately impact end user flows until the identity provider is added as an option in a user flow (aka built-in policy).‚ÄĮTo change user flows, the limited role of ‚ÄúB2C User Flow Administrator‚ÄĚ is required.

In addition, there is now a default Guest User role, which all guest users in the tenant are assigned to. This ensures that such object have access to only a limited subset of the information stored within the directory.

Posted in Azure AD, Office 365 | Leave a comment

Files restore is finally available for SharePoint Online libraries

An year has passed since we got the Files restore feature for OneDrive for Business. Initially announced back at Ignite 2017, it took Microsoft several months to release it in the beginning of 2018. While there were some rough edges, the feature worked as advertised and is very helpful in addressing certain scenarios. You can find my initial impressions on it here, or read the documentation here.

At least year’s Ignite conference, Microsoft announced that Files restore is coming to regular SharePoint Online document libraries, and now we finally have it available. There’s really not much to cover about the feature, as it works in the same manner as its OneDrive sibling, so I wont go into any details here, apart from sharing the news that it’s already available in at least some tenant. If you need additional information, make sure to check out the documentation.

Posted in Office 365, SharePoint Online | Leave a comment

Bug in the EAC rules UI causes erroneous “duplicate object” prompts

Everyone that has been using Microsoft’s cloud offerings is well aware of the fact that quality has drastically degraded over the past few years. The examples of features that don’t work as advertised or are just broken are numerous, and it often makes you wonder just how much time is spent on testing and QA. Anyway, without further ado, here’s the latest annoyance I’ve run into.

The issue is evident when you create a new mail flow rule, or edit an existing one. All you need to do is include any condition, exception or action that involves some kind of recipient to be designated. It seems that some changes were made recently to the UI, which now passes the Display Name attribute to the underlying PowerShell engine. And since for every tenant with more than a handful of users it’s expected to have at least few objects with matching Display Names, selecting one such object results in the following error message:

Using the “Show cmdlet logging” dialog reveals that the cmdlet being passed to the PowerShell engine references a custom object, something like the below:

Command: Set-TransportRule -ModerateMessageByUser @([Microsoft.Exchange.Management.ControlPanel.PeopleIdentity])

While the full object cannot be retrieved and investigated, it doesn’t seem like it resolves down to an uniquely-valued entry, but passes something like the DisplayName value instead. Which in turn results in PowerShell complaining about the ambiguity of the input, as it should. So the issue basically boils down to how the selected object/parameters are being passed down from the UI to the backend. Pretty much every action that takes an user object as input from the picker dialog (invoked via can trigger the issue, which is in reality the majority of cases for which you would be creating Transport rules.

If you want to reproduce the issue yourself, all you need to do is create or edit a transport rule and within its settings, point to any recipient that has a matching Display Name value with another object in your tenant. If you need help locating such objects, you can use the following PowerShell cmdlet:

Get-Recipient  | select Alias,DisplayName | Group DisplayName | ? {$_.Count -gt 1}

Count Name                      Group
----- ----                      -----
2 First group               {@{Alias=firstgroup; DisplayName=First group}, @{Alias=Firstgroup1; DisplayName=First group}}
2 Vasil Michev              {@{Alias=vasil; DisplayName=Vasil Michev}, @{; DisplayName=Vasil Michev}}

As seen from the above output, in my tenant there are two instances of duplicate Display Names, which is perfectly normal and acceptable, even expected in any reasonably-sized tenant. What’s also expected is for the Exchange UI to be well aware of this fact, and for it to use an uniquely-valued attribute to identify the objects instead, but alas.

Anyway, once you have identified a matching object, all you need to do is use it within a transport rule. The example below shows such a rule being created with the “Includes any of these recipients in the To or CC box” condition, and with one of the “duplicate” objects selected:

The rest of the settings don’t matter, you can select anything for the action. The moment you press the Save button on the Rule dialog however, you will be presented with the error shown on the first screenshot above. The good news is that Microsoft is aware of the issue, and it should be fixed soon, hopefully.

In the meantime, if you run into this issue you can simply resort to using PowerShell as the workaround. If you insist on using the UI, you can instead type in the object’s primary SMTP address in the “Check names” control.

Posted in Exchange Online | Leave a comment

How Microsoft stores Windows Activity history and Cloud clipboard data inside your Office 365 mailbox

Over the past few years Microsoft has been releasing more and more cloud-enabled features and integrations for Windows 10. Activity history is one such feature, designed to synchronize information about apps and services you use, browsing data, media files played and more, across all your devices. Activity history in turn powers the Timeline, which you can find under the Task View button, or by pressing Win+Tab. With the 1809 release of Windows 10, we also got the Cloud Clipboard, another example of cloud-reliant feature, which syncs your clipboard history across devices.

Such features are often turned on by default, so inevitably some sensitive data ends up stored in the cloud, which in turn raises privacy concerns. Even though Microsoft introduced controls to disable gathering of activity information, users have reported cases of such information still being gathered with all the controls toggled off.

The question on how and where exactly this data is being stored is an interesting one to explore. To get an overview of the various type of activity information Microsoft is storing, you can use the Activity history page of the Privacy portal: The dashboard is depicted below:

An important note is due here ‚Äď the Privacy portal can only be accessed via a MicrosoftID. In effect, if you have a device that’s Azure AD joined and only the organization account is configured, the dashboard will not be accessible. Yet, even in such cases the activity information is being displayed on your timeline. So where is the data stored then? The answer is ‚Äď in your Office 365 mailbox. You might not see any indications of that, as it’s all hidden under the covers of the non-IPM subtree.

To access and work with the activity data stored within your mailbox, you can use some low-level utilities such as MFCMAPI. It’s beyond the scope of this article to cover the basics of using MFCMAPI, so if you need additional instructions check out the gallery of articles available here.

After you’ve opened the message store with MFCMAPI, expand the Root Container node and select the ApplicationDataRoot entry. This folder is the “standard” placeholder for any application to store their own data in, within the confines of the user’s mailbox store. Storing data inside the Non-IPM subtree isn’t something new and various applications have been doing it for years, often without following any guidelines or schemas, as evident from the numerous other top-level folders you can find within the store.

If you expand the ApplicationDataRoot node, you will see a list of various GUIDs, each corresponding to a given application. For example, the 501d6dd3-1f78-4bff-9115-ad2c6375b205 entry corresponds to the Events from email feature, b669c6ea-1adf-453f-b8bc-6d526592b419 is where Focused Inbox does some of its magic, MyAnalytics uses 3c896ded-22c5-450f-91f6-3d1ef0848f6e and so on.

In our case, data related to user’s activities is being stored in the d32c68ad-72d2-4acb-a0c7-46bb2cf93873 container, more specifically under the Activities subfolder. As you can see from the screenshot below, over 55k items are currently stored in said folder in my mailbox, detailing my activities over the past year or so. Each of those items corresponds to a single activity, with the activity details stored in a JSON format inside the RawJSON property:

The JSON string itself stores information about the user performing the action, the device and platform, the application used and resources accessed, timestamp and more. In the example above, an URL is visible, indicating that the activity in question was captured from one of my browsing sessions. As another example, here’s an activity captured for an action performed in Outlook, with the JSON expanded and trimmed to show only the more interesting properties:

From top to bottom, we have the ObjectId of the user performing the action, followed by the TenantId. Application information is gathered next, as well as the DeviceId, start and end timestamps, and so on. By fetching all the individual messages corresponding to events for a given date, or for the past 30 days, one can reconstruct the timeline presented by Windows in the Task View.

Apart from activities information, another interesting type of data stored under the d32c68ad-72d2-4acb-a0c7-46bb2cf93873 container is the content of your Cloud clipboard. Indeed, if you browse the ClipboardItems subfolder, you will see items with similar set of properties to the ones detailed above. As before, the RawJSON property contains the interesting bits of information, with the actual copied item being stored in encoded form in the Payload property. Here’s a snapshot of one of the ClipboardItems stored in my mailbox:

To get to the actual copied content, we can use PowerShell to invoke the corresponding .NET methods in order to convert the Base64-encoded string to a human-readable one. This in turn will output another JSON string, decoding the content property of which will give us the final result:

As it turns out, the example item represents some text I’ve copied out of the sponsor chapter of the Office 365 for IT Pros book, probably while making changes to it last month.

Of course, clipboard items are not always just plain text content, so using the method outlined above might not generate acceptable results every time. The general idea still applies though ‚Äď you enumerate each item in the ClipboardItems subfolder, get the RawJSON property and decode the Payload value to get the content of the clipboard item created at the given time and date.

So, now that you are aware that the Activity history and Cloud Clipboard data is being stored inside your mailbox, does this make you feel safer or?

Posted in Exchange Online, Office 365 | 2 Comments

NonIPMRoot items returned in searches in Exchange Online

Everyone with some experience with Exchange knows that there’s a lot more to a mailbox than what we see in clients such as Outlook or OWA. The RecoverableItems subtree is one such example, storing items placed on hold, audit items, calendar logs and versioning, etc. Generally speaking, email clients work with just the IPM subtree, while low-level tools such as MFCMAPI or EWSEDitor can be used to explore the Root, or Non-IPM subtree.

In the Exchange Online world, the Non-IMP subtree is quite larger, as over the years different Office 365 applications have been using (and abusing) mailboxes to store and process data. Delve, MyAnalytics, Teams, all have their own folders or folder trees inside the Non-IPM root. Nowadays, you can even spot a whole new subtree parenting such application-specific folders, named ApplicationDataRoot. If you are interested in obtaining more details about such folders via PowerShell, check my previous article on the subject.

What I want to focus on in this article is the fact that items from the Non-IPM subtree are now returned in search results in Outlook. An easy way to check this is to perform a search for items received on a given date, and change the scope to include all items in the current mailbox. Here’s an example of one such search I did recently:

Apart from using the “received” keyword, the only thing I’ve done is to add the “In folder” column, in hopes of understanding what the few hundred such “empty” items were. As you can see from the above screenshot, I have 31 items in a folder called ClipboardItems, 561 items in folder called Activities and so on. None of those folders were created by me, which prompted me to investigate what’s going on.

So, turning to PowerShell, I could easily locate those folders as part of the Non-IPM subtree. It’s actually interesting to compare the current state of my mailbox to the snapshots I took an year ago with the same cmdlets. The good folks at Microsoft has certainly been busy dumping more and more data into the mailbox – the total number of folders has grown to 321, and the size is now a whooping 23.51GB.

$folders[0] | select FolderPath,ItemsInFolderAndSubfolders,FolderAndSubfolderSize

FolderPath ItemsInFolderAndSubfolders FolderAndSubfolderSize
---------- -------------------------- ----------------------
/                              279291 23.51 GB (25,239,132,610 bytes)

And here’s the current breakdown by 10 largest folders:

$folders | Sort -Property @{Expression={[double]((($_.foldersize).split(" "))[2]).trimstart("(")}; Ascending=$false} | select FolderPath,foldersize,ItemsInFolder -First 10

FolderPath                                                                                                                                          FolderSize                     ItemsInFolder
----------                                                                                                                                          ----------                     -------------
/AllItems                                                                                                                                           7.955 GB (8,541,124,958 bytes)         59540
/Top of Information Store/MVP DL                                                                                                                    3.163 GB (3,395,854,617 bytes)         23009
/Top of Information Store/MVP Only DL                                                                                                               3.044 GB (3,267,951,982 bytes)         19192
/SubstrateFiles/SPOOLS                                                                                                                              1.837 GB (1,971,942,903 bytes)         32446
/Top of Information Store/Inbox                                                                                                                     940 MB (985,650,804 bytes)              6523
/Unified Inbox                                                                                                                                      928.6 MB (973,702,527 bytes)            6438
/Finder/OwaFV15.1AllFocusedAQMkAGU2MWM5NzU0LWY3MmQtNGI3OS1hNDVlLTBkMzI3OWVlADViM2YALgAAA6EpIkCGWdRMge0pKyjfROEBAEg98MzHIÔ£Ņ9FiFjwzzGYA9UAAAIBDQAAAA== 736.2 MB (772,009,689 bytes)¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬† 4584
/Finder/OwaFV15.1ToOrCcMeAQMkAGU2MWM5NzU0LWY3MmQtNGI3OS1hNDVlLTBkMzI3OWVlADViM2YALgAAA6EpIkCGWdRMge0pKyjfROEBAEg98MzHIÔ£Ņ9FiFjwzzGYA9UAAAIBDQAAAA==¬†¬† 533.5 MB (559,374,034 bytes)¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬† 4165
/Top of Information Store/Forums                                                                                                                    523.7 MB (549,174,242 bytes)            4694

As well as the folders with most items:

$folders | Sort ItemsInFolder -Descending | select FolderPath,foldersize,ItemsInFolder -First 10

FolderPath                                                           FolderSize                     ItemsInFolder
----------                                                           ----------                     -------------
/AllItems                                                            7.955 GB (8,541,124,958 bytes)         59540
/ApplicationDataRoot/d32c68ad-72d2-4acb-a0c7-46bb2cf93873/Activities 446.3 MB (468,014,414 bytes)           54069
/SubstrateFiles/SPOOLS                                               1.837 GB (1,971,942,903 bytes)         32446
/Top of Information Store/MVP DL                                     3.163 GB (3,395,854,617 bytes)         23009
/Top of Information Store/MVP Only DL                                3.044 GB (3,267,951,982 bytes)         19192
/Finder/NoArchiveTagSearchFolder8534F96D-4183-41fb-8A05-9B7112AE2100 961.4 MB (1,008,126,877 bytes)          8219
/Calendar Version Store                                              433.1 MB (454,166,321 bytes)            7340
/Recoverable Items/Audits                                            29.8 MB (31,247,493 bytes)              6726
/Top of Information Store/Inbox                                      940 MB (985,650,804 bytes)              6523
/Unified Inbox                                                       928.6 MB (973,702,527 bytes)            6438

Back to Outlook now. Why such items are being returned is not clear, but it seems to be a side-effect of the new server-driven search. The behavior doesn’t seem to depend on the Outlook build, assuming it’s a version that uses the server-driven search, as I can reproduce it with the several different versions, across different channels. Looks like a clear bug to me, but perhaps there is a reason for it…

And it’s not just Outlook exposing items in the Non-IPM subtree, eDiscovery and Content searches also suffer from the same issue. The same search query performed against my mailbox via the SCC Content Search UI returns the following:

While it might not be immediately visible from the screenshot, the selected item is one of the items stored in the application folders. More important is the number of items found by the search, and the breakdown by folder we can obtain by exporting the search report:

As you can see from the above, the bulk of returned items are stored in the Activities folder, same as what the Outlook search reported. If you with to examine any of these items, you can simply press the “Download original item” link in the UI. You can decipher the content of such items via MFCMAPI, but that’s a subject for another article. Believe me, there is some interesting data stored inside ūüôā

Posted in Exchange Online, Office 365, Outlook | Leave a comment