We often hear stories about the wonders of the DevOps model and how it has revolutionized the software industry. It’s really great in theory, bringing the processes of development, testing and running the service closer together, and providing benefits such as faster release cycles, improved reliability, and whatnot.
That’s the theory alright, but as its often the case, in the real world things look a bit different. More often than not, DevOps has turned into an excuse to ship a minimum-viable version of a product, and to force customers to wait for years until all the promised features are delivered. Another thing that has become painfully obvious over the past few years is the fact that the role of QA or “tester” has been continuously neglected, to a point where software is launched with blindingly obvious bugs or issues. We’ve seen it with recent Windows releases, we’ve seen it with Office releases, we see it every day with Office 365 and Azure. Slowly but surely the customers are turning into beta testers, and it’s no wonder that DevOps is sometimes mockingly referred to as “testing in production” (not in a good way).
Anyway, after this rant-y preamble, let me introduce you to yet another obvious bug in the Office 365 Admin Center, which could have been easily avoided if the engineers simply bothered to test their code before releasing it in production. Actually, there are series of few interlinked bugs, and it all boils down to the way the Office 365 Admin Center UI handles admin role assignments.
Bug #1 was originally reported on the Azure AD forums over at MSDN, and can be summarized as “Office 365 Admin Center UI doesn’t recognize any additional Azure AD roles”. The steps to reproduce it are simple enough: go to the Azure AD blade in the Azure portal, select a user and assign any non-Office 365 role to him, for example “Application administrator”. Confirm that the role was successfully assigned and head to the Office 365 Admin Center, select the user and check the list of Roles on the user details pane. The Application administrator role will not be visible there, nor when you press the Edit button next to Roles. One can say that this is the expected behavior, as the role doesn’t “exist” in the context of Office 365. After all, we don’t see any Azure roles in the Office 365 Admin Center, right? I might actually let that one slide, even though Azure AD is vital part of Office 365, and so are any roles that give permissions on Azure AD objects and entities are of great importance.
That’s not all there is to the issue though. If you happen to press the Save button now, even though you haven’t made any changes, the role assignments for the given user will be overwritten with what’s currently displayed in the UI. And as the UI doesn’t list any of the Azure AD roles, in our scenario this means effectively removing the Application administration role assignment from the user. You can confirm this by switching back to the Azure AD blade and refreshing the roles page. This is essentially bug #2, and the short video below illustrates this behavior:
Very similar issues can be observed when you have more than one “standard” Office 365 role assigned. Although in most cases the Admin Center UI should handle such scenarios, one important exception is the Global administrator role. If you have assigned the GA role to a user, the Office 365 UI will not let you assign another role, despite the fact that such scenario is supported and can be enabled via the Azure AD portal or the Azure AD/MSOnline PowerShell module. If you use any of these endpoints to assign another role to a given GA user, say the Billing administrator role, and head back to the Office 365 Admin Center, the role assignments will be correctly displayed in the user pane. However, when you press the Edit button, once again the UI will not reflect the fact that the user has the GA role assigned. More importantly, if you click the Save button, the GA role will be stripped! Video below:
As illustrated above, there are obvious deficiencies in the way the Office 365 Admin Center handles role assignments. Whether this was a deliberate design decision in the quest to simplify management, or something that was overlooked, we can only guess. The good news is that the relevant teams at Microsoft are aware of the bugs described above and are already working on fixing the experience. Given the scale at which Office 365 runs, it might take few days before the Admin Center UI is updated with those fixes.
In all fairness, Microsoft’s reaction was very fast, as it should be in a DevOps world. However, this does not excuse the fact that the engineers neglected to thoroughly test the relevant UI controls in the first place, before releasing them to millions of Office 365 customers. When even critically important functionality such as assigning admin roles is clearly not given enough attention, one can only wonder whether a fundamental change in the way code is developed, tested and released might be necessary.
P. S. In case you are wondering, the same behavior was present in both the old and the new Office 365 Admin Center, as in fact the same UI bits are utilized in both.