Hi folks,

I need to apply a Slowly Changing Dimension (SCD) using Azure Databricks with data from an on-premise Business Central database. I want to build a medallion architecture.

I am bringing the tables to the bronze level by selecting data incrementally using the field _systemModifiedAt. For building the silver level, I want to perform a Merge using the field _systemId. Is this possible? Specifically, is _systemId a unique identifier for every record in all the tables of Business Central, allowing me to apply a Merge for SCD?

When a record changes in Business Central, does it produce a new record in the database table with the same _systemId but with updated information?

Many thanks!

Jorge.

​Hi folks, I need to apply a Slowly Changing Dimension (SCD) using Azure Databricks with data from an on-premise Business Central database. I want to build a medallion architecture. I am bringing the tables to the bronze level by selecting data incrementally using the field _systemModifiedAt. For building the silver level, I want to perform a Merge using the field _systemId. Is this possible? Specifically, is _systemId a unique identifier for every record in all the tables of Business Central, allowing me to apply a Merge for SCD? When a record changes in Business Central, does it produce a new record in the database table with the same _systemId but with updated information? Many thanks!Jorge.  Read More

​