Az.Synapse Set-AzSynapsePipeline - Ignores literal nulls (value = null) in json definition

[mwc360]

Description

This module has a bug when using the Set- cmdlets to publish json definition objects to Synapse.

If the JSON source file contains elements specified as literal nulls when the definition is published to Synapse it ignores the JSON elements that were literal nulls (i.e. value = null). This causes the use of the 'Treat as Null' Stored Procedure activity to not be publishable from the json files. I was able to work around this via directly using the Synapse Rest APIs, the same files published via the APIs do not strip away any literal null json element.

Issue script & Debug output

n/a

Environment data

5.1

Module versions

I tested via versions 0.1.0, 0.7.0, 0.18.0, the bug occurs with all versions

Error output

No error output from Powershell, the bug is that the published result to Synapse will ignore any json element set to equal a literal null (i.e. value = null)

[dingmeng-xue]

Thank you for your reporting. Tagging and routing to the team member best able to assist.

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @wonner, @zesluo.

Issue Details

---

Description

This module has a bug when using the Set- cmdlets to publish json definition objects to Synapse.

If the JSON source file contains elements specified as literal nulls when the definition is published to Synapse it ignores the JSON elements that were literal nulls (i.e. value = null). This causes the use of the 'Treat as Null' Stored Procedure activity to not be publishable from the json files. I was able to work around this via directly using the Synapse Rest APIs, the same files published via the APIs do not strip away any literal null json element.

Issue script & Debug output

n/a

Environment data

5.1

Module versions

I tested via versions 0.1.0, 0.7.0, 0.18.0, the bug occurs with all versions

Error output

No error output from Powershell, the bug is that the published result to Synapse will ignore any json element set to equal a literal null (i.e. value = null)

Author:	mwc360
Assignees:	-
Labels:	Service Attention, bug, question, customer-reported, Synapse, CXP Attention
Milestone:	-

[wonner]

Hi @mwc360 , Thanks for your feedback. I can reproduce this problem. I would like to know will it cause any problems if it strips away the literal null?

[mwc360]

@wonner Yes, it undoes the 'Treat as Null' feature (checkbox on the parameter mapping pane) for the Stored Procedure and other activities where you map values or expressions to parameters. If the 'Treat as Null' checkbox was checked as true in ADF (value = null in JSON) it results in the box being unchecked and no value being passed into the parameter, however since the parameter is still defined it will result in the activity failing as you cannot reference a parameter but not map any value into the parameter.

[wonner]

Hi @mwc360 , We are looking into it and will get back to you as soon as we have an update. Sorry for your inconvenience.

[arcticdba]

Can confirm this is STILL an issue in version 3 (the one currently in the cloud shell in Azure).

The impact is rather severe: every pipeline that has an activity with parameter will fail to be created the correct way, making running the pipeline impossible. Without resorting to the mentioned workaround with the REST API, it is impossible to create working pipelines from code.

I have raised this on the internal MVP distribution lists as well.