Knowing what normal looks like for every account the SOC monitors
The ESH-SOC is integrating behaviour profiles into its monitoring of sign-ins to Microsoft’s cloud. The method comes from a study by TU/e and the SOC, tested on five months of sign-in logs from one of the SOC’s customers, a Dutch IT service provider, and checked by two of its senior IT managers.
In short
- To spot a stolen or misused account in the sign-in logs of Microsoft’s cloud, the ESH-SOC needs to know what normal looks like for each account. Many accounts have too little activity to tell.
- The SOC is integrating a method from its own research: it learns behaviour patterns from all accounts together, using knowledge of how Microsoft Entra ID and the organisation work, and gives every account a profile, also accounts with little history.
- In the study, IT managers at the SOC customer whose logs were used recognised most profiles and found the view useful. The profiles also exposed a service account that was being used in a way it should not have been.
The idea. Single sign-in events on the left, the behaviour profiles learned from them on the right. The log lines are illustrative; the profiles are among those found in the study.
Watching cloud sign-ins for the organisations the SOC monitors
Many organisations manage their accounts in Microsoft Entra ID (formerly Azure Active Directory). Every time a person, an app or an automated script signs in to Microsoft 365 or another company service, Entra ID writes a line to its sign-in log: who signed in, to which app and resource, when, and from which network.
A security operations centre (SOC) is the team that monitors an organisation’s systems and investigates anything suspicious. For the organisations it monitors, the ESH-SOC watches these logs for misuse, such as a stolen account or an account doing things it was not meant to do. To spot what is unusual, an analyst first needs to know what is usual, and that differs per account. A developer, a shared administrator account and a backup script all sign in in very different ways.
One model of normal behaviour for a whole organisation misses those differences. A separate model per account needs plenty of history, which many accounts lack. In the data studied, a quarter of the individual administrator accounts signed in fewer than six times in five months, and administrator accounts are among those an attacker most wants to take over.
This question came up in the SOC’s daily work, and the ESH-SOC took it to research. The result is a way to build a behaviour profile for every account, which the SOC is now integrating into how it monitors cloud sign-ins.
How the profiles are built
The method was developed by Gelareh Hasel Mehri, Emmanuele Zambon and Nicola Zannone at TU/e, together with the ESH-SOC. It builds on earlier work by the team on hospital access logs, and works in three steps.
First, every sign-in event is described by a small set of features, such as the app used, the time of day and the network it came from. Second, a clustering algorithm groups similar events together without being told what to look for. Each group is a behaviour pattern, for example “syncing the company directory from the data centre” or “opening Teams chat from a home network”. Third, each account’s profile is the set of patterns that make up a meaningful share of its sign-ins (at least 5% in the study).
Because the patterns are learned from all accounts together, an account with only a handful of sign-ins still gets a profile: its few events are matched to patterns that are well established by everyone else’s activity. Profiles built from all accounts at once largely matched those built from each account type separately, and fully for the automated bot accounts.
What makes the profiles sharp is knowledge that a SOC brings to the data. The researchers added it to the features step by step (Figure 1).
Figure 1. Each step adds features that describe sign-ins in more detail. The bars show the silhouette score, a standard measure of how clearly events separate into groups (from −1 to 1, higher is better), averaged over the five months. The paper also tests a fifth variant of step 3 with a different encoding.
The log fields alone give poorly separated groups. Counting how often things happen helps. The largest step comes from knowledge of Entra ID itself: apps get permission to reach a set of resources, and renew that access in the background. Reconstructing, for each account and app, which resources it reaches gives a much sharper picture. The last step uses what is known about the organisation’s staff: some always work from the office, others move between client sites or travel abroad. Counting the networks and countries an account signs in from captures that. Profiles built with all four steps also agree best with the known account types.
The data came from one of the ESH-SOC’s customers, a Dutch IT service provider: 1.26 million successful sign-ins over five months of 2024 by 402 accounts, from regular users to administrators, service accounts and bots, anonymised before analysis.
1.26 M
sign-ins from 402 accounts over five months
0.16 → 0.54
how clearly sign-ins separate into groups, from raw log fields to all features
0.77 → 0.85
agreement between profiles and account types (F1 score, 0 to 1)
2
senior IT managers reviewed the profiles
Checked with the customer’s IT managers
Before a SOC relies on profiles, the people who run the systems have to agree that they make sense. The researchers turned each pattern into readable rules, such as “accesses the directory but not Microsoft Graph”, and showed them to two senior IT managers at the customer, both certified Microsoft architects with more than ten years of experience.
The managers could tell which kind of account most profiles described. They recognised, for example, the bot that keeps the company’s on-premises directory in sync with the cloud, and administrator accounts by the security and device-management tools they used. They pointed out that the regular users include external guest accounts, and the profiles showed that guests mostly use Microsoft Teams, while internal staff use a broad range of apps. The profiles also separated finer groups that the managers had not anticipated, such as one made up only of team leads.
One profile stood out. A service account had been set up to handle the mailbox for job applications. Its profile showed that developers also used it to sign in to the Power Platform admin centre and make changes (Figure 2).
Figure 2. A profile records what an account does, which is not always what it should do. Here, review by the experts turned a learned pattern into a finding.
“As a bad account! Because it goes against the rules we set up. It’s good that you found it.” (Senior IT manager, IT service provider)
The managers described the profiles as understandable and “logical”, and said they made them look at the logs across all accounts at once. As one put it, “This is a much more holistic view.” They saw detecting unusual behaviour, in particular for automated accounts, as the main use.
What the capability brings to the SOC and its customers
As the ESH-SOC integrates the profiles into its monitoring, its analysts get a description of what each kind of account normally does, against which a new sign-in can be judged. This also covers accounts with little history, such as rarely used administrator accounts, where a per-account model has too little to go on.
The profiles also tell analysts what kind of account they are looking at. That matters when something looks wrong: an incident with an external guest account needs coordination with the guest’s own company, and a deviation by an automated account, which normally does one task from one network, stands out clearly.
The service-account case shows a second use for the organisations the SOC monitors. Profiles describe what is common in the logs, including habits that should not be there, so they should be reviewed with the people who manage an organisation’s accounts before they are used as a baseline. Done by the SOC together with those people, that review is also a check that sign-ins happen as intended: that service accounts are used only by the systems they were set up for, that administrators work from their own accounts so their actions can be traced, and that guests reach only what they need. Sign-in habits change over time, and the paper recommends rebuilding the profiles regularly. The review can be repeated each time.