Wildecard in ADF copy activity

Question

Hello ,

Is there a way in wildcard pattern in ADF copy activity to pick files from folders with certain naming convection.

For example I have a following folder directory under bucket Myfolder

                      **Main Folder**                                         

                                            **SubFolder1**

                                                      **Sealing**

                                                         abc1.json

                                                         dcsh2.json

                                                      **Painting**

                                                           *hdcsve.json*

                                                           *vgfvgf.json*

                                                      **Sketching**

                                                           *dccrvrt.json*

                                                           *vdfdfv.json*

                                                **SubFolder2**

                                                        **Sealing**

                                                             yththt.json

                                                             gttgtr.json

                                                        **Painting**

                                                               *fgvrvr.json*

                                                               *dcedfc.json*

                                                        **Sketching**

                                                                 *dccrvrt.json*

                                                                  *vdfdfv.json*

                                                      

In the above example I want the files from Subfoler1 and SubFolder2 be picked which are under the folder name "Painting" or "Sketching "to be copied.

Answer 1

Hi @Shweta Chavan

Greetings & Welcome to Microsoft Q&A forum! Thanks for posting your query!

In Azure Data Factory (ADF), you can use wildcard patterns in the copy activity to filter files based on naming conventions. For your specific case, you can set the wildcard Folder Path to target the "Painting" and "Sketching" folders under both "SubFolder1" and "SubFolder2".

Wildcard paths: Using a wildcard pattern will instruct the service to loop through each matching folder and file in a single source transformation. This is an effective way to process multiple files within a single flow. Add multiple wildcard matching patterns with the plus sign that appears when you hover over your existing wildcard pattern.

For additional information, please refer: https://learn.microsoft.com/en-us/azure/data-factory/connector-azure-blob-storage?tabs=data-factory#source-transformation

Additionally, to delete the files after copying, you can use the Delete activity in ADF. You would configure the Delete activity to point to the same paths you used in the copy activity, ensuring that it targets the files you just copied.

I hope this information helps. Please do let us know if you have any further queries.

---

If this answers your query, do click `Accept Answer` and `Yes` for was this answer helpful. And, if you have any further query do let us know.