Unified audit log in Office 365 continues to disappoint

Office 365 has been around for over 9 years now, and a lot has changed since it was officially launched. While the initial state was one of “loosely coupled” workloads, over the years Microsoft introduced tighter and tighter integrations between the products. This applies to not just “end user” features, such as say cloudy attachments, but also on admin controls, security and compliance features and the overall architecture. A set of “unified” functionalities have replaced workload-specific features such as audit logs, DLP, labels, etc.

The Unified audit log in particular has been around for 5 years or so, and while in theory it sounds like a great thing to have, with all relevant audit events being funneled into a single “data mart”, the execution leaves a lot to be desired even years after the initial release. While its span has grown well beyond the initial workloads and event types, and some nice improvements have been released to both the UI and the PowerShell cmdlets, some annoyances and deficiencies continue to plague the overall experience.

Take for example the ingestion of Azure AD Sign-in events. One can argue that authentication events are one of the most important entries to have, yet to date the process of ingesting those into the Unified audit log continues to fail on a semi-regular basis. While the documentation will happily inform you that such entries can take up to 24h to appear, in reality they can be missing for days and weeks at a time. Moreover, for the majority of cases where certain event types have failed to ingest you will receive no notification and no entry will be shown in the SHD either. The most annoying part is that even after complaining about this issue for year, Microsoft is yet to address it.

As an example, a search against the Unified audit log for any sign-in events between Oct 28th and Nov 5th resulted in zero entries returned. The same query against the Azure AD sign-in logs, executed from within the Azure AD blade returned few hundred results, as expected. Trying the same query against a different tenant resulted in no events being returned after Oct 31st when using the Unified audit log, while again the events were visible when querying the Azure AD sign-in logs directly. This is hardly the first time this has happened and usually events start backfilling, but just goes to illustrate how unreliable the Unified audit log can be at times.

Now, an argument can be extended that since the corresponding events can be found in the Azure AD sign-in logs, the issue detailed above is a cosmetic one. Sadly, this is more than just an inconvenience, as similar behavior has been observed for audit events sourced from other workloads, and such that cannot be accessed by any other means. And, it is more or less a regular occurrence, as a result of which the usefulness of the Unified audit log is greatly diminished. After all, what’s the use of an auditing system that only works from time to time.

What’s even worse is that delayed ingestion is hardly the only issue the Unified audit log is suffering from. I alone have published half a dozen post detailing other aspects of the experience that make it harder for us as end users to consume, and others have pointed out multiple similar issues. One of the most common one is with “nonsense” data being presented, such as the “unknown” user principal events I talked about a while back. One would think that after years of reporting similar issues Microsoft would take steps to improve the way data is presented as part of audit events, but alas this still remains an issue as illustrated below:

Obviously there’s little value in using values such as “NOT-FOUND” for any type of data, and while there is usually some additional information you can get out of the remaining/unprocessed properties, it would be that much easier if Microsoft simply used a proper identifier. Same goes for GUIDs, “unknown” principals, “Not available” entries, and so on. In addition, the fact that products such as MCAS can surface details not otherwise obtainable, either via the Unified audit log search experience or via PowerShell/Management APIs, remains an issue to date.

Which brings us to the next point, “premium” audit events, only accessible behind an E5 paywall. I’ll refrain from talking about the numerous issues we faced back when the first of these events, MailItemsAccessed, was introduced (in case you are interested in that, Tony Redmond detailed the issues in few articles). The more important aspect here is that even basic stuff such as Send events is only made available to tenant that subscribe to the E5 SKU or the standalone add-ons. And yes, you do get more bang for your buck, with the extended retention for audit events being particularly useful, so I’m not saying youshouldn’t be paying the extra. Given the amount of useless events one can find in the audit logs though, it’s infuriating to see something that practically each tenant out there would appreciate made only available for E5 SKUs.

And since I mentioned “useless” or “noise” events, these continue to plague the Unified audit log. It’s still very common to see questions of the “what the hell is this event” type over at the different Microsoft communities/forums, with the majority of them corresponding to some internal/background process performing an action against data stored within a mailbox, user’s ODFB or whatever. Most such events are not clearly marked as “internal” and it usually takes some additional effort before you can dismiss them as harmless. Sometimes they get removed/hidden from the Unified audit log after customer complaints, but that’s not always the case.

Overall, the benefit of having all the audit events in a single place continues to be diminished by various issues, which Microsoft has failed to address in a proper manner for years. Similar arguments can be extended to the Azure AD sign-in logs as well, where it’s also not uncommon to see events corresponding to internal processes, baffling application/resource names and so on. One can only hope that all these will be addressed in the future.

This entry was posted in Microsoft 365, Office 365. Bookmark the permalink.

2 Responses to Unified audit log in Office 365 continues to disappoint

  1. David A. says:

    Hi,
    I’m facing the same issue since a couple of weeks. It was working fine before. I’m unable to find “userloggedin” operations when using “search-unifiedauditlog” cmdlet. I can find other operations, like Teams logins or message deletions, but not the sign-in action in particular..
    I opened a case, just in case, but I think we’ll need to use the cmdlet Get-AzureADAuditSignInLogs.. ,and just wondering when it will be GA.
    Thanks for your blog, very interesting !

    • Vasil Michev says:

      I can still see sign in events, they’re simply delayed by few days… And I’m tired of having to report such issues, so instead I chose to go with a rant post this time around 🙂

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.