Reporting on Entra ID application registrations

After updating the scripts to report on Entra-integrated applications (aka service principals) last week, it is time to take a look at the updated scripts to report on application registrations. While one can argue that this scenario is less important, due to its primarily internal focus, it should not be overlooked. More and more administrative or even pure business tasks nowadays require additional application registrations, especially when you want to combat “consent creep”. For ISVs, securing access and ensuring your applications are not overly permissioned is an important scenario. An even if applications are only used by internal teams, you probably want to monitor secret expiration and adhere to best practices. Don’t forget that by default, every user within the company is allowed to create new app registrations!

So, what’s new in this version of the PowerShell scripts? First of all, it is scriptS now, plural, as we’re introducing an Graph SDK based version. Said version has been optimized to run with delegate permissions (i.e. in the context of a user), but as always feel free to update it to use your preferred method of authentication. Next, some additional data can now be collected and included in the report: Entra recommendations, service principal sign in activities, and the Application credentials sign in activities report data. The latter unfortunately seems to have stopped returning any data, but hopefully Microsoft will fix this soon. The Entra recommendations data leaves a lot to be desired, but we will talk about this later.

As including these new data points comes at the expense of additional permissions requirements for the scripts, two new  parameters have been introduced to control things. You can find the list of supported parameters, their default values and the corresponding permission requirements below:

• IncludeSignInStats – specify whether to include data from the service principal sign in activities report as well as the application credentials sign in activities one. Comes with the requirement for AuditLog.Read.All permissions. Default value is $false.
• IncludeRecommendations – specify whether to include data from the Entra recommendations resource. Requires the DirectoryRecommendations.Read.All permission. Default value is $false.
• Verbose – used to provide additional details during the script’s execution. Default value is $false.

Few additional helper functions have been introduced in order to handle parsing of the newly obtainable data or the various permission requirements. The most complex of these is the parse-Recommendations helper function, which is used to “map” the impactedResources property of each recommendation object against the matching application object, and surface any relevant information as needed. With the help of the prepare-RecommendationOutput function, those are then surfaced in the output as part of the Recommendations property.

Fair warning – the data exposed in said recommendations is often times of questionable value. For example, the “adalToMsalMigration” recommendation, which is supposed to signal that a given application is still leveraging the Azure AD Graph, was surfaced for only one of the apps within the test tenant, whereas no less than four of them had some Azure AD Graph permissions listed. As another example, the “applicationCredentialExpiry” recommendation returned an empty impactedResources value, even though at least three of the apps had expired credentials. In addition, the “staleAppCreds” recommendation did not return any data as to the credential’s last usage timestamp, requiring you to blindly trust it. The observed behavior is likely due to the limited timeframe covered by the recommendations, and will hopefully improve going forward.

Similarly, the application credentials sign in activities data is all missing currently, at least in any tenant we were able to test with. Which is a pity, as it represents one of the most impactful data point that you should consider when gauging usage of applications. Until the above mentioned endpoints get fixed, you can only rely on the data from the service principal sign in activities report, which gives you a breakdown of the last usage timestamps for the service principal object representing a given application for both delegate and application authentication flows, in the confines of the current tenant, that is.

Before we move to specific examples, few more words about the inner workings of the scripts. The first step is of course to handle authentication. The Graph API version of the script leverages the client credentials flow, so make sure to fill in the required variables (lines 233-247). The Graph SDK version on the other hand uses the delegate permissions model, so it will try authenticating in the context of a user. In both cases, feel free to update the relevant bits to use your preferred auth method instead. And, do make sure that you take care of the required permissions as well, as follows:

After obtaining an access token, the script will retrieve the set of application objects within the organization and their properties. If any of the optional parameters is invoked, additional data will be collected, as requested. Lastly, the script will iterate over each application object and prepare the output. Here, we do actually make one additional call per app in order to fetch the set of Owners. While technically we can get this via the original LIST request, it requires the use of the $expand operator, which is known to have few issues. Feel free to change the behavior here, if you believe $expand performs to your expectations.

Without further ado, here are the links to download the Graph API based script and its Graph SDK variant. Review the set of requirements and then run the scripts by leveraging one of the examples below (all examples use the Graph API version, replace the script name with app_reg_Permissions_inventory_GraphSDK.ps1 for the Graph SDK one):

#To run the script
.\app_reg_Permissions_inventory_GraphAPI.ps1

#To include service principal and application credentials sign-in stats
.\app_reg_Permissions_inventory_GraphAPI.ps1 -IncludeSignInStats

#To include Entra ID recommendations
.\app_reg_Permissions_inventory_GraphAPI.ps1 -IncludeRecommendations

#Use the -Verbose parameter for additional information about the script's processing
.\app_reg_Permissions_inventory_GraphAPI.ps1 -IncludeSignInStats -IncludeRecommendations -Verbose

The script will write its output to a CSV file within the working directory. As with the service principal scripts, some of the fields will contain multiple values, usually semicolon-separated. Additional information is often presented within [] or () brackets. For example, the permissions-related fields will designate the resource (“API) within square brackets, followed by the list of individual scopes/roles:

[Office 365 Management APIs]:ActivityReports.Read,ServiceHealth.Read;[Microsoft Graph]:User.Read,Mail.Read

Similarly, credentials will be presented in a semicolon-concatenated list, with additional information presented within brackets (such as the validity period of the key). If Entra ID recommendations are retrieved, they will also be presented in a semicolon-concatenated list, with the recommendation’s “type” enclosed in square brackets, and additional information added as relevant. The example below shows an entry with two recommendations: one showing that the application still leverages Azure AD Graph and one designating a credential as not used:

[StillUsesAdal]:True;[UnusedCredentials]:c7a08d41-d3bd-4bcb-99cc-b0fe235f7edd(password)(Last active -> N/A)

As mentioned above, do not expect all the data surfaced out of Entra ID recommendations to correspond to reality and take them with a grain of salt. The rest of the fields are presented “as is”, and should be the basis of your analysis. Depending on the parameters used, the following fields will be part of the CSV file:

• Application Name
• ApplicationId, aka the clientID of the application.
• Publisher domain and Verified give you information about the publisher verification process, whereas Certification field gives you the Microsoft 365 Application certification URL for the app, if any.
• SignInAudience shows the type of Microsoft accounts supported as per this table.
• ObjectId is the GUID of the service principal.
• Created On gives you its creation timestamp, i.e. when was it provisioned in your tenant.
• Owners gives you the UPN(s) of any owner(s) assigned to the application.
• Permissions (application) gives you the list of Roles configured on the application.
• Permissions (delegate) gives you the list of Scopes configured.
• Permissions (API) is the set of “internal” API scopes.
• Allow Public client flows shows whether the application supports less secure authentication methods, such as the ROPC flow.
• Key credentials gives you the count of certificates configured for authentication, as well as an overall “status” – whether any of them has expired, or has an excessive validity (defined as over 180 days, you can adjust the value on line 95). The KeyCreds field lists each key credential’s Id and status, whereas the Next expiry date (key) field gives you the date and time at which the key will expire.
• Similarly, we have the same set of fields for client secrets. Password credentials gives you the count of client secrets configured for authentication and the overall status. The PasswordCreds field lists each client secret’s Id and status, whereas the Next expiry date (password) field gives you the date and time at which the secret will expire.
• App property lock gives you the status of the App property lock feature, as detailed for example here.
• HasBadURIs signals whether any of the Redirect URIs configured on the application is considered unsecure (i.e. uses a wildcard, http:// or urn: prefix or “localhost”).
• Redirect URIs gives you the list of Redirect URIs configured on the application, separated by semicolon.
• Last sign-in is newly added field, exposed when invoking the -IncludeSignInStats switch. It gives you the timestamp when the Service principal corresponding to the application was last used, i.e. the date and time of last authentication in your own tenant. It is the “most recent” out of the other four newly added timestamps: Last delegate client sign-in, Last delegate resource sign-in, Last app client sign-in and Last app resource sign-in. Because the data comes only from SP usage within your own tenant, this is not a reliable representation of the application usage!
• If we are able to obtain the Application credentials usage report data, the Last credential sign-in field will reflect the corresponding timestamp. This field is more appropriate for gauging whether a given application is active… if only the corresponding endpoint actually returned data.
• UsesAADGraph is a “calculated” field, signaling whether any permission associated with the Azure AD Graph resource (“00000002-0000-0000-c000-000000000000”) has been found on the app.
• Recommendations gives you the set of Entra ID application-related recommendations, mapped against the set of application objects within the organization.

As it is hard to fit all of the above fields on a single screen, the screenshot below presents a curated view of the CSV file generated by the script. It has been filtered out to show only multi-tenant applications and has many of the “empty” columns hidden or simply not displayed. Even this limited view can still be used to highlight some bad practices: credentials with extensive validity, expired credentials, bad URIs, use of the Azure AD Graph and so on.

Our recommendation would be to filter the report based on few key properties, starting with credentials – do make sure that you use secrets with validity of maximum 180 days, and rotate them periodically. Keep the set of secrets per app to a minimum, and where possible, use certificates instead of client secrets. Review the permissions on the application periodically and remove any unused ones. Verify and optionally certify your public-facing applications if you are an ISV. And overall, review the properties periodically and make a habit of enforcing a good hygiene on your application objects!

As always, feel free to modify the scripts to better suit your own needs and environments, and don’t forget to send feedback.