Self-service PowerShell throttling policy relaxation for Exchange Online

Many vendors, Microsoft included, enforce strict throttling policies on their services in order to make sure that resources are not being hogged by a single user/process. In the Exchange world, throttling policies were first introduced in Exchange 2010 and have been present in each server version since. With the move to the cloud, they become even more vital, given the multi-tenant nature of the service.

It’s understandable why Microsoft wants to enforce such limitations, and one can even argue that the decision to remove the ability to even see the current throttling policy values that are in effect in Exchange Online was justified as well. However, it’s not that uncommon for organizations to have a legitimate need to go beyond the limitations enforced by throttling policies, for example when performing a migration from on-premises or third-party solution. In such scenarios, a process was made available for requesting temporary relaxation of the throttling controls, although Microsoft still had the last word in terms of the duration and actual throttling values.

Few months back, Microsoft decided to make this process a bit easier, by releasing a self-service experience for checking the current settings and updating the configuration to a more relaxed set of values. At the time, I decided to skip covering this new experience, as there’s not much you can say about it and even less in terms of actual configuration, and frankly I’m not a fan of that stupid “AI” support assistant. Since then, Microsoft has added additional scenarios, so I guess I have to play along 🙂

One new scenario in particular is relaxing the throttling controls for PowerShell cmdlet execution. To request this, you need to again open the Microsoft 365 Admin Center, then go to Support > New service request. In the “Need help?” pane, enter “PowerShell throttling” or similar keywords, and you will be presented with the following:

Here, you are given the option to Temporarily update throttling policies for a migration or Update throttling policies to align with your tenant size. Pressing the Run Tests button for either of these will initiate a (not so) short check and once the results are back, you will be presented with the option to Update settings, provided the automated diagnostic tool determined there is room for improvement. All you need to do is select the consent checkbox and press the button.

Overall, the process is pretty much identical with the EWS throttling scenario, and fairly straightforward, though I would much prefer to have this exposed on an actual page instead of having to go through the “Need help?” pane. Anyway, the end result is what’s important here, and being able to perform this without having to open a support case (or provide justification) is a good step forward. Nowhere in the process you will be presented with the actual values though, so the experience remains as convoluted as ever.

Do note that this is not a “golden ticket” and you should still add appropriate anti-throttling handling within your migration and reporting scripts. Same old recommendations still apply, at least until Microsoft releases a full set of REST-based cmdlets or their corresponding Graph API endpoints.

Posted in Exchange Online, Microsoft 365, Office 365 | 3 Comments

Common issues when sending mail via PowerShell in Office 365

Lately I seem to run into such questions quite often, so I figured I’d put a short article outlining the most common issues one might run into when using PowerShell to send email messages via SMTP AUTH in Office 365. The article is by no means intended to be an exhaustive resource for all possible errors, just some of the common things I see over at the different communities.

One issue that seems to be overlooked in particular is the fact that PowerShell by default uses on older/insecure protocols. More correctly, it uses the system default values, which even for modern (desktop) OS versions are a bit relaxed. By now, you should probably be well aware that within Office 365, Microsoft has more strict requirements, and generally speaking you should be using TLS 1.2. This in turn creates problems with PowerShell, as connections negotiated via older/less secure protocols will get blocked. The error message received simply says something like “the server requires a secure connection” but doesn’t directly tell you what to do:

Send-MailMessage: The SMTP server requires a secure connection or the client was not authenticated. The server response was: 5.7.57 SMTP; Client was not authenticated to send anonymous mail during MAIL FROM [VI1PR08CA0220.eurprd08.prod.outlook.com]

So, one thing to try, and make habit of is to configure the Security Protocol value and set it to TLS1.2. This can be done by invoking the corresponding method, as follows:

[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12

Once this is done, you can retry running the Send-MailMessage cmdlet and if the credentials specified are correct, it should work OK. Granted, there are several other things you might need to check on or configure before you can make it work, which we will cover next. But first, let’s answer another question that pops up – how can we make sure the changes to the Security Protocol value persist (they’re per-session ones). An easy way to do this is to add the line above to your PowerShell profile. Remember that profiles can depend on the host as well, so consult the above article to pick the best location for your scenario.

If the above suggestion doesn’t solve the issue, one other thing to check is whether SMTP Auth is disabled for your organization or the particular user. As the article explains, per-mailbox settings take precedence over the organizational config, so if needed you can add exceptions, either by using the UI or PowerShell:

Set-CASMailbox -Identity user@domain.com -SmtpClientAuthenticationDisabled $true

Another thing to remember is that the current implementation of SMTP Auth uses basic authentication, and thus is considered insecure. Newer PowerShell versions might even warn you about this when using the Send-MailMessage cmdlet:

WARNING: The command 'Send-MailMessage' is obsolete. This cmdlet does not guarantee secure connections to SMTP servers. While there is no immediate replacement available in PowerShell, we recommend you do not use Send-MailMessage at this time. See https://aka.ms/SendMailMessage for more information.

Microsoft has already announced support for using OAuth with SMTP, but that doesn’t mean the already existing clients/libraries will be able to leverage this automatically. They still rely on and will try to use Basic authentication, which by now should be blocked across most Office 365 tenants. The Security defaults feature is one example on how this block can be enforced, and depending on the SKU you might be able to leverage Conditional Access policies or Exchange Authenticated policies for more granular control over this. So make sure to check your appropriate controls and exclude any accounts that need to leverage outdated methods to send email, such as using the Send-MailMessage cmdlet in PowerShell.

Of course one also must make sure that the credentials are correct, and for best results, use an account that has a corresponding Exchange Online license assigned. Should you need to impersonate other users when sending messages, the Send-MailMessage cmdlet is probably not the best approach (use EWS instead), but it might work in some cases, granted you make sure the relevant Send As permissions have been assigned.

Posted in Exchange Online, Microsoft 365, Office 365, PowerShell | Leave a comment

Even quicker way to list Mailbox permissions in Office 365

While I was looking into some details for preparing my next Practical 365 article, I spotted a change in the way Exchange Online PowerShell returned mailbox permissions. Whereas previously we got a long list of “system” permissions, now only the relevant user-added entries were returned, making the list of permissions much shorter. Here’s an example:

Get-MailboxPermission sharedtest@michev.info

Identity             User                 AccessRights
--------             ----                 ------------
sharedtest           NT AUTHORITY\SELF    {FullAccess, ReadPermission}
sharedtest           NT AUTHORITY\SELF    {FullAccess, ExternalAccount, ReadPermission}
sharedtest           vasil@michev.info    {FullAccess}

And that’s all the details being returned currently, whereas few months back one would get the following, quite longer list:

Get-MailboxPermission sharedtest

Identity             User                 AccessRights
--------             ----                 ------------
sharedtest           NT AUTHORITY\SELF    {FullAccess, ReadPermission}
sharedtest           NT AUTHORITY\SELF    {FullAccess, ExternalAccount, ReadPermission}
sharedtest           vasil@michev.info    {FullAccess}
sharedtest           EURPRD03\Administ... {FullAccess}
sharedtest           EURPRD03\Domain A... {FullAccess}
sharedtest           EURPRD03\Enterpri... {FullAccess}
sharedtest           EURPRD03\Organiza... {FullAccess}
sharedtest           NT AUTHORITY\SYSTEM  {FullAccess}
sharedtest           NT AUTHORITY\NETW... {ReadPermission}
sharedtest           S-1-5-21-15893167... {ReadPermission}
sharedtest           PRDTSB01\JitUsers    {ReadPermission}
sharedtest           EURPRD03\Administ... {FullAccess, DeleteItem, ReadPermission, ChangePermission, ChangeOwner}
sharedtest           EURPRD03\Domain A... {FullAccess, DeleteItem, ReadPermission, ChangePermission, ChangeOwner}
sharedtest           EURPRD03\Enterpri... {FullAccess, DeleteItem, ReadPermission, ChangePermission, ChangeOwner}
sharedtest           EURPRD03\Organiza... {FullAccess, DeleteItem, ReadPermission, ChangePermission, ChangeOwner}
sharedtest           EURPRD03\Public F... {ReadPermission}
sharedtest           EURPRD03\Exchange... {FullAccess, ReadPermission}
sharedtest           EURPRD03\Exchange... {FullAccess, DeleteItem, ReadPermission, ChangePermission, ChangeOwner}
sharedtest           EURPRD03\Managed ... {ReadPermission}

The highlighted dozen or so entries all correspond to different Microsoft-managed accounts and background processes, and are of no relevance to us as “consumers” of the service. While being able to peek behind the curtain does add some value in few cases, most often this additional data can be regarded as useless. In fact, it was very common practice to filter out those entries entirely, including the default “NT AUTHORITY\SELF” one. Now, this is no longer needed.

But the improvements don’t stop here. While checking some additional mailboxes for permission entries, I noticed something odd in the output. Let’s see if you can spot it as well:

Get-MailboxPermission gosho

Identity             User                 AccessRights
--------             ----                 ------------
gosho                NT AUTHORITY\SELF    {FullAccess, ReadPermission}
Gosho2               NT AUTHORITY\SELF    {FullAccess, ReadPermission}

So what do you think? We get two “NT AUTHORITY\SELF” entries, but that’s not uncommon – in fact the first example above also has two. What is different this time is the Identity value, which points to a different mailbox object altogether. So either the output is borked, or the cmdlet is returning entries across multiple mailboxes. YES!

In other words, the Get-MailboxPermission cmdlet now supports wildcards, which in turn makes it possible for us to return any and all permissions entries across the tenant in a single run, without having to use the pipeline! Meaning, the following is now possible:

Get-MailboxPermission *

which is effectively the analog of:

Get-Mailbox | Get-MailboxPermission

but should in theory run much faster. Well, as another example of theory meeting practice – this is currently NOT the case. Even if we ignore the fact that the wildcard variant seems to always error out on some entires (you can add -ErrorAction silentlycontinue to account for that), it’s actually taking a lot more time to complete. In my tests, it took nearly twice as much time as the pipeline variant.

Looking at the actual output though, you will notice that the wildcard variant returns tons of “system” mailboxes, such as the ones used to hold organizational settings, quarantine, arbitration mailboxes and what not. A short sample of those is shown on the screenshot below. As evident from the output, you will probably never care about these entries, so including them is probably an oversight on Microsoft’s part. Then again, it’s another opportunity to peek behind the curtain and see just how many services within Office 365 use Exchange Online mailboxes:

In turn, the total number of entries returned is four times larger compared to the “sanitized” variant you get when using the pipeline, which contributes to the perceived slowness. Add to that the delay caused by paging and whatever that BackEndLocatorException error is supposed to signify, and we have an explanation as to why this method takes more time to complete. Hopefully, these shortcomings will be addressed in the future, and we will be able to use a single call to Get-MailboxPermission in order to list all relevant permission entries within the tenant, much like we can do currently with Get-RecipientPermission (which also returns all the system mailboxes btw).

Now, currently I have no information as to whether using the Get-MailboxPermission in such manner is actually a supported scenario, so I would hold off updating my scripts just yet. As mentioned above, there are some obvious issues with the current implementation anyway, so probably it’s a good idea to wait for Microsoft to address those. Lastly, I wasn’t able to use the same method with the Get-EXOMailboxPermission cmdlet, so no idea whether it’s coming to the REST-based cmdlets.

Once I get more information, I will make sure to update this post and the good old how-to:

Quickly list all mailboxes to which a particular user has access

Posted in Exchange Online, Microsoft 365, Office 365, PowerShell | Leave a comment

Microsoft authentication flows now feature snazzy gifs

A recent change to the authentication flow for Microsoft accounts has added some additional “illustrations” that might help better guide users over the login process. The process itself hasn’t changed, it simply has been enriched with said illustrations (in gif format), which looks something like this:

That’s of course a static image, as I’m too lazy to capture the whole thing as a gif, but the central image in there will be animated (sample below). Here’s another example in the case of “number” prompt:

For the time being, it seems that the change only applies to Microsoft accounts, but will probably be hitting OrgIDs later on. Again, the actual process hasn’t changed, but you might want to spread the word and update your training materials just in case. The gifs themselves seem to be semi-static, so you should be able to link to them. Or just grab a copy, here’s an example one:

https://logincdn.msauth.net/shared/1.0/content/images/authenticatorinfo_eb760e89c7b67b355b023e5af32a7542.gif

Posted in Azure AD | Leave a comment

Exchange API permissions missing

Few days back, Microsoft announced deprecation of v2.0 Outlook REST API. Important detail was omitted from that article though, and only added to the republished post over at the EHLO blog. Here is the changed paragraph:

Going forward, we will not be making any further investments in the capabilities or capacity of the Outlook REST API beta or Outlook REST API v2.0. This will result in two changes for developers. First, we will retire the OAuth Sandbox by December 31, 2020. Additionally, we have removed the “Exchange” app permission from the Azure portal. We have updated our documentation with additional information regarding creation of EWS apps.

The bolded part in particular is what’s most relevant to the Exchange crowd out there, as it relates to the way we grant permissions for any EWS OAuth-based app, as well as for enabling certificate-based authentication for the Exchange Online V2 PowerShell module.

Note that this doesn’t mean that the aforementioned functionalities are no longer available. It simply makes the process of adding the necessary permissions a bit more convoluted. And as the relevant documentation articles still use the “old” instructions (at least the V2 PowerShell module ones do), I figured it might be beneficial to publish a quick “how to”, reflecting these recent changes. So here goes.

Follow the instructions up to the point where you need to add the necessary API permissions, all the earlier steps should be the same. Once you end up on the API permissions page in the Azure AD blade, you will notice that at the bottom of the Request API permissions pane you will only see a single entry under Supported Legacy APIs. Where previously you would find both Azure AD Graph API and Exchange (as depicted on the top section below), now only the former is visible (bottom):

Instead, to add the relevant permissions you now need to go back to the top of the Request API permissions pane, click the APIs my organization uses tab and search for Office 365 Exchange Online. Note that searching for just Exchange will not yield any results, as the search functionality in Azure AD blade remains quite poor.

Once you locate the Office 365 Exchange Online entry, click on it and proceed with adding the necessary permissions. The steps from here on remain the same, so in most cases you would need the Application permissions entry, and the relevant set of permissions therein (such as full_access_as_app for EWS OAuth, Exchange.ManageAsApp for CBA). Select the relevant entries, hit the Add permissions button and consent to the changes as needed, and you’re good to go.

An alternative approach to achieve the same task is outlined in the documentation article cited in the blog post above. This involves modifying the manifest of your Azure AD application, which can be a bit of annoyance thanks to the JSON format used. Personally, I’d stick to using the UI method outlined in this article, unless Microsoft decides to totally disable this approach.

Posted in Azure AD, Exchange Online, Graph API, Microsoft 365, Office 365 | 2 Comments