New Maester Capabilities Added Recently

The Maester project is a PowerShell-based “test automation framework” to check tenant configurations to highlight potential issues for administrators to deal with. When I first covered the Maester project in April 2024, the initiative seemed like an interesting example of how the technical community can come together to build something of obvious value to Microsoft 365 and Entra ID administrators. By October 2024, Maester had added the ability for tenants to add custom tests to extend coverage to basically anywhere that the Microsoft Graph API could reach. In December, the developers published V1.0 of the Maester PowerShell module.

A glance at the latest Maester documentation shows just how much work has been put into its development. I was especially taken by the methods enabled to monitor Microsoft 365 tenants using Azure DevOps Pipeline, GitHub Actions, Azure Automation, and Azure Container App Jobs, including the ability to notify administrators through email or messages posted to Teams or Slack channels. There’s lots of value to explore here.

Running Tests Against User Accounts

The Maester documentation has examples of writing custom tests. If you want more, Clayton Tyger has created a GitHub repository for custom tests. Most of the current tests cover missing properties for Entra ID accounts, like phone numbers, city, department, hire date, employee identifier, and so on. Checking for missing properties isn’t difficult and given the importance of fully-populated accounts for components like the Microsoft 365 user profile card, it’s a good thing to do.

Venturing into tests for user account properties introduces a level of complexity over many of the other Maester tests. Often, a standard test checks for the presence of a setting which is either enabled or disabled, like who are allowed to create guest accounts in a tenant. Many conditional access policy settings are reviewed in tests to ensure that a tenant is well protected, and so on.

These kinds of tests can be completed quickly. Processing tests fast is important when Maester might run 120 or more tests to check a tenant configuration. You don’t want to get bogged down with waiting for details of 20,000 user accounts to be fetched for checking.

Maester uses a function called Invoke-MtGraphRequest to fetch data from Graph resources. My assumption is that the function is a developed version of the standard Invoke-MgGraphRequest cmdlet from the Microsoft Graph PowerShell SDK that adds functionality like automatic pagination and support for consistency headers for advanced queries. As such, Maester tests have no problem fetching large quantities of user objects to check, if you have time to wait.

Identifying Human Accounts

But then we get to the really difficult problem: how to identify “real” user accounts that should have values in all their properties? Finding all Entra ID member accounts isn’t a good way to proceed because Entra ID member accounts are created for room mailboxes, shared mailboxes, and other purposes. In addition, Entra ID creates member accounts for accounts synchronized from other tenants in a multi-tenant organization (MTO). Failing a Maester test because the account used by a shared mailbox doesn’t have its City property populated is probably not a valuable outcome.

The normal approach is to apply a filter to find user accounts with licenses on the basis that non-human accounts probably don’t have licenses. The only problem is that the accounts used for some shared mailboxes are licensed to allow the mailboxes to have archives or a higher storage quota. A variation on the theme is to filter user accounts with a specific service plan that isn’t usually assigned to non-human accounts. Finding such a service plan becomes the issue here. The service plan can’t be an Exchange plan, yet it must be assigned to all user accounts. The Teams service plan might be a possibility.

Another solution is to use one of the custom properties available in Entra ID to mark accounts. This approach allows a precise filter to find the set of Entra ID accounts used by humans at the expense of the overhead needed to mark accounts by updating the selected custom property.

The Alternative

An alternative is to use the Get-User cmdlet from the Exchange Online management module to fetch Entra ID accounts with user mailboxes. This approach works if everyone in the organization has a mailbox and it’s easy to check the accounts for missing properties (Figure 1).

Fetching a bunch of accounts to check their properties won’t be fast in large tenants, so this is a good example of processing best left to periodic Azure Automation jobs rather than the kind of on-demand test like those used by Maester. Functionality that works splendidly when processing just a few objects often struggles to cope when asked to do the same thing for thousands.

Insight like this doesn’t come easily. You’ve got to know the technology and understand how to look behind the scenes. Benefit from the knowledge and experience of the Office 365 for IT Pros team by subscribing to the best eBook covering Office 365 and the wider Microsoft 365 ecosystem.