Some ramblings around Continuous access evaluation, support for Graph and service principals

Before we start, a fair warning – this article mostly summarizes my recent experiments around Continuous access evaluation, there’s probably nothing of value to be found in it. Just another example of “I blog so I don’t forget” 🙂

To set the stage, I was doing some random tests with the Graph explorer tool, when I noticed that the access token it uses seems to have an extended validity of 24h or so. Of course, I was aware of the Continuous access evaluation feature and the fact that using it results in issuing such “long-lived” tokens. What I was not aware of was the fact that the Graph as resource supports CAE, apparently. And the Graph explorer as a client also seems to be CAE-aware, as it adds the required xms_cc claim, though it does not behave exactly as expected upon token revocation.

Further observations seem to confirm that there is some support for CAE on the Graph resource layer, as blocking the user in Entra ID resulted in the eventual revocation of the access token (with some caveats, as we will discuss later). On the other hand, operations taken on the Graph explorer service principal that should have resulted in invalidating the access token did not seem to have any effect. Even deleting the service principal object made no change – I could still issue Graph requests for up to 24h after the original token issuance, so almost a day after the service principal object was deleted. And according to the documentation, this should NOT be happening.

So what’s happening? My initial thought was that I might be seeing yet another unfortunate effect of Microsoft’s convoluted licensing policies, as in the CAE enforcement for workload identities (service principals) likely being tied to the Workload ID premium SKU, which I did not have in my tenant. The documentation however does not mention any such requirements.

The next direction of my investigation was to confirm whether this session was indeed CAE-capable. The token lifetime was an indirect evidence of that, and so was the presence of the xms_cc claim. On the other hand, the troubleshooting guidance found in the official documentation advises us to verify the value of Continuous access evaluation/Is CAE Token field within the Entra ID sign-in logs (if looking at the “raw” logs, this is represented by the value of the “Is CAE Token” key within authenticationProcessingDetails).

Interestingly, the sign-in logs did not list any entries marked as CAE-capable, at least when looking at the interactive singins tab. As my tests were done in private browser session, my expectations were that all relevant signin entries would be shown under the interactive tab, which turns out is not the case, and resulted in me overlooking some entries. The documentation does warn you about the fact that multiple entries will be generated, and that some of them will instead be found under the non-interactive signins tab, so make sure to check both:

There are multiple sign-in requests for each authentication. Some are on the interactive tab, while others are on the non-interactive tab. CAE is only marked true for one of the requests it can be on the interactive tab or non-interactive tab. Admins must check both tabs to confirm whether the user’s authentication is CAE enabled or not.

My tests did not stop here though. As I wanted to understand how things work on the client side as well, I went though the steps of requesting a CAE token. You can find the relevant details in the official documentation. To implement this example in PowerShell, I leveraged the MSAL binaries installed as part of the Graph SDK for PowerShell. Here’s the sample code:

#Leverage the Graph SDK for loading the base DLLs
Connect-MgGraph

#And the MSAL one as well
Add-Type -Path "C:\Program Files\PowerShell\Modules\Microsoft.Graph.Authentication\2.17.0\Dependencies\Core\Microsoft.Identity.Client.dll"

#Set the client capabilities var
$var = [System.Collections.Generic.List[string]]::new()
$var.Add("CP1")

#Proceed with defining the scopes as usual
$Scopes = New-Object System.Collections.Generic.List[string]
$Scope = "https://outlook.office365.com/.default"
$Scopes.Add($Scope)

#Make sure to include the WithClientCapabilities() method
$app = [Microsoft.Identity.Client.PublicClientApplicationBuilder]::Create("xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx").WithClientCapabilities($var).WithRedirectUri("http://localhost")
$app2 = $app.Build()

#Request the token
$res = $app2.AcquireTokenInteractive($Scopes).ExecuteAsync()
$res.Result.AccessToken

Note that the resource we’re issuing this request against is Exchange Online this time. We’re again doing this for the purpose of illustrating something, bear with me. The important thing is to make sure you pass the xms_cc claim, with the CP1 value, which in the case of using the MSAL methods is done via WithClientCapabilities(). If doing a “raw” request instead, you need to add the following:

&claims={"access_token":{"xms_cc":{"values":["CP1"]}}}

The example above resulted in obtaining a long-lived token for the Exchange Online resource, as illustrated on the collage below. The sign-in logs contained two entries about this specific example: one interactive sign-in, marked not CAE-enabled, and a non-interactive sign-in, marked as CAE-enabled. Both entries are stamped with the same Correlation ID value of 9117fbec-e1a3-4857-8a4a-ab088814660d. The access token received (shown in decoded form in the top-right insert) has the unique token identifier from the non-interactive sign in entry, and thus is CAE-enabled, with 24h+ validity and the xms_cc claim present.

In effect, we’ve shown that every (public) client can request, and receive a “long-lived” CAE-enabled access token, provided the resource in question supports CAE. Which is why I deliberately choose to use the Exchange Online resource. In turn, by replicating the same example for the Graph resource, we get yet another evidence that Graph supports CAE now. What’s interesting though, in the case of manually obtaining CAE-enabled access token for the Graph, no interactive sign-in event was generated at all, and we only got the one non-interactive one. Talk about consistent behavior…

Speaking of inconsistencies, according to the documentation CAE should be supported for service principals as well. What is not clear is the type of auth flows supported, as there is quite the difference between a token issued in the context of a user or one issued via the client credentials flow. The API documentation is quite vague about this and only mentions the need to add the xms_cc claim. Other bits explicitly mention that adding the client capabilities claim is done by a call against the authorize endpoint, which again limits this to delegate permission scenarios. My attempts to add said claim to token request via the client credentials flow were all in vain, so at this point I am uncertain what support for workload identities entails.

Moreover, I want to again repeat that deleting the Graph explorer service principal object did NOT result in revoking the CAE access token, as already mentioned above. The documentation specifically lists the Service principal delete operation as supported, so I am unsure as to whether the observed behavior is because the Graph as a resource does not (fully) support CAE, or only tokens obtained in the application context are invalidated. Or only first-party apps are supported? Or perhaps authentication needs to happen with a SP-specific credential? As the access token itself usually only contains the GUID of the parent application object, and not the service principal objectID, perhaps properly correlating the deletion event fails?

Whatever the reason, the fact remains that a CAE token can be reused even after the service principal for which it was issued is deleted, up to the extended 24h validity period. Which in turn makes it quite difficult to address scenarios where a “rogue” service principals has been compromised. In fact, the only reliable remediation I can think of is to immediately revoke tokens for all users, unless you somehow can obtain a list of all the users that had grants associated with the service principal in question, before deletion happened. The audit logs will give you that information, and you definitely want to cover more than the 30 days worth of events included by default. You might also filter any sign-ins corresponding to the SP in question, though with the API not currently returning non-interactive signins, that’s a challenge in itself.

In summary, here are my observations from this exploration into Graph support for Continuous access evaluation:

• The Graph resource (00000003-0000-0000-c000-000000000000) seems to now support CAE. In turn, applications such as the Graph explorer or the Azure/Entra ID portal support CAE as well. The Graph SDK for PowerShell does not currently send the xms_cc claim though.
• You can filter the sign-in logs for any CAE events via the following:

GET https://graph.microsoft.com/beta/auditLogs/signIns?$filter=authenticationProcessingDetails/any(x:x/key eq 'Is CAE Token' and x/value eq '1')

Unfortunately, there currently seems to be an issue with retrieving non-interactive signins via the endpoint above.

• CAE tokens are NOT revoked immediately, expect delays of few minutes at least. I believe something like 15 minutes was mentioned in one of the documentation articles.
• Not every client behaves well when tokens expire however, i.e. the Graph explorer seems to end up in a loop.
• There also seem to be some replication delays, as token blocked for one query was working fine when hitting a different endpoint, likely due to hitting different farm/datacenter on the backend.
• Support for workload identities/service principals is shady/unconfirmed (might require additional licensing?)
• You can use any app to request a “long-lived” token, by simply adding the xms_cc claim (subject to support on the resource side).
• CAE can potentially be a problem when “rogue” service principals are encountered. Remediating such scenarios does not seem to work as advertised, as even deleting the service principal object does not result in invalidating any issued CAE tokens for it. Instead, you should consider revoking sessions/blocking access for all users, unless you can confidently list the ones that were leveraging the SP in question.
• Alternatively, you might consider outright blocking the CAE feature itself. Not that this will save you from rogue SPs 🙂