facing this below error how can i solve this

Question

ErrorCode=UserErrorColumnMappingNotAllowed,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Column mapping is not allowed for copy between 'AzureBlobStorage(DelimitedTextFormat)' and 'AzureBlobStorage(JsonFormatV2)'.,Source=Microsoft.DataTransfer.ClientLibrary,'

Answer 1

Hi ,

Thanks for reaching out to Microsoft Q&A.

Seems you are attempting to copy data from a DelimitedText (CSV) file to a JSON file in Azure Blob Storage.

Cause

• Column mapping is not supported when copying data between these two formats.
• JSON (V2) expects hierarchical JSON structures, whereas DelimitedText is a flat format.
• The default behavior of JSON (V2) expects to take all source columns as an object or an array rather than mapping them 1-to-1.

Option 1: Remove Column Mapping

1. In Copy Data activity, go to the Mapping tab.
2. Clear all mappings (Remove explicitly defined column mappings).
3. Save and rerun the pipeline.

Option 2: Use JSON Object Mapping

• If you want to structure the data properly in JSON format, modify the sink settings:
  1. In the Copy Data activity, select the Sink tab.
     1. Under JSON Format Settings, set File Pattern to 'Array of Objects' (if you want an array format).
        1. Try using Single Document Output instead of multiple row-based mappings.

Option 3: Use 'JsonFormat' Instead of 'JsonFormatV2'

• If possible, switch the sink format to JsonFormat instead of JsonFormatV2, as JsonFormat may allow column mapping (I am not sure on this, need to be checked)

Option 4: Convert CSV to JSON Manually Using Data Flow

If structured JSON is required:

1. Use a Data Flow instead of Copy Data.
2. In Data Flow:
   • Use Source transformation to read the CSV.
     • Use a Derived Column transformation to structure data in JSON format.
       • Use a Sink transformation with JsonFormatV2 for hierarchical JSON.

Option 5: Convert to Parquet First (Workaround)

If JSON (V2) does not accept direct CSV transformation:

• Convert the CSV to Parquet format first.
• Then use Parquet to JSON (V2) conversion.

Recommendation

• If your JSON structure is simple, removing column mapping should work.
• If you need a structured JSON format, consider using Data Flow with transformations.

Please feel free to click the 'Upvote' (Thumbs-up) button and 'Accept as Answer'. This helps the community by allowing others with similar queries to easily find the solution.