Trimming Unwanted Versions Stopped by Retention Policies and Labels

Last month, I wrote about the introduction of Intelligent Versioning for SharePoint Online. I think this is a great feature because its automated management of versions created during editing sessions reduces the storage quota consumed to store file versions. The advent of AutoSave for Office increased the number of versions created for files, and keeping 500 or so versions for a file, when some versions only include minimal changes, is effective but expensive.

Microsoft allows tenants a default storage quota for SharePoint Online that’s consumed by items stored in sites and Loop workspaces (containers). If a tenant exceeds their SharePoint storage quota, they must buy more from Microsoft or use Microsoft 365 Archive to move the storage consumed by inactive sites to cheaper “cold” storage.

As I noted in the article, the big issue with the current implementation of intelligent versioning is that it doesn’t work with Purview Data Lifecycle management, aka Microsoft 365 retention policies. If SharePoint Online sites come within the scope of a retention policy or individual documents have retention labels, then the requirement to retain information about files trumps the desire of intelligent versioning to remove unwanted versions for those files.

Checking Expired Versions Trimmed by Intelligent Versioning

Microsoft’s documentation explains how retention works with document versioning. I decided to check out what happens when versions expire for documents in a site with a retention policy in force. On November 6, I noted that several versions were in an expired state (Figure 1).

The next day, the expired versions were gone from the list. In one respect, this is what you might expect to happen. A background SharePoint Online job detected the existence of expired versions and removed them, which is what intelligent versioning is all about (the process is called trimming).

But the retention policy applied to the site set a five-year retention period and the document had a retention label with a ten-year retention period. The document is a source file for the Office 365 for IT Pros eBook, and you can never be too careful with source material. The longest retention period wins, so SharePoint Online should retain the file for ten years. However, no trace could be found of the removed versions.

Microsoft’s documentation says that versions for items subject to a retention hold are not automatically purged. In addition, users cannot delete versions from the Version history. When intelligent versioning trims versions in a site without retention policies, the files bypass the recycle bin. This didn’t apply, so it seemed like the site preservation hold library is the logical place to look. However, nothing was found in the preservation hold library except the copy of the file containing all versions prior to the implementation of intelligent versioning in the tenant.

Reappearing Versions

Then the removed versions reappeared in the version history complete with a new expiration date (Figure 2). Interestingly, SharePoint Online adjusted the expiration date for some other versions to make sure that full coverage of changes to the file is available.

After chatting with Microsoft engineering, I understand that the observed behavior is quite normal. The expired versions are removed by a background job, only for retention processing to detect that the removed versions are still within their retention period. This causes SharePoint to add a week to the previous expiration date for each version and make the versions available again. The cycle then repeats until the retention period for removed versions lapses to allow SharePoint Online to permanently remove the unwanted versions from its store.

More Intelligence in the Future?

It’s unfortunate that a clash exists between storage management and retention. Microsoft’s current approach is probably the best that can be done for now. I’m sure that they have an eye on the potential to extend intelligent versioning to interact with retention processing better. One possibility is to allow organizations to decide if selective version trimming is permissible, perhaps at a less aggressive level. For instance, it’s OK to remove versions that only contain formatting changes but not OK to remove any that contain text additions or deletions. Perhaps some storage savings are possible without compromising retention. It’s a hard nut to crack.

Insight like this doesn’t come easily. You’ve got to know the technology and understand how to look behind the scenes. Benefit from the knowledge and experience of the Office 365 for IT Pros team by subscribing to the best eBook covering Office 365 and the wider Microsoft 365 ecosystem.