Split Pipeline json into separate files for every individual Pipeline activity (Code ILM)
Azure Data factory Pipeline
Currently a adf pipeline is stored a one big JSON file.
In order to copy a customized activity to a second pipeline in another branch, you need to edit the file, copy the exact lines code belonging to a activity, copy the code to the other pipeline, paste in exactly the right spot and re join the activities.
I propose to create a folder per pipeline and create nested folders for each activity. (similar to the file storage in Power bi, pbip file format) This also enables different developers to work in parallel on different activities the same pipeline.
Azure Data factory PipelineCurrently a adf pipeline is stored a one big JSON file.In order to copy a customized activity to a second pipeline in another branch, you need to edit the file, copy the exact lines code belonging to a activity, copy the code to the other pipeline, paste in exactly the right spot and re join the activities. I propose to create a folder per pipeline and create nested folders for each activity. (similar to the file storage in Power bi, pbip file format) This also enables different developers to work in parallel on different activities the same pipeline. Read More