columns description in datacatalog

Question

can we add datacatalog columns description automatically through python. suppose there are 100-200 columns i want to add description for those columns is there a way to automate it?

Answer 1

Hello @Prasad Lotankar ,

Thanks for the question and using MS Q&A platform.

My understanding is that you would like to know how to bulk update the column descriptions in Microsoft Purview using Python. Please correct if my understanding is not clear.

Option1: The available option is by leveraging PyApacheAtlas (https://github.com/wjohnson/pyapacheatlas) as described here: https://github.com/wjohnson/pyapacheatlas/wiki/Azure-Purview-Tips#update-an-existing-column

Option 2: PyApacheAtlas is a very convenient way of just getting massive amounts of data into Purview via an Excel interface and a little bit of Python!

Here is a video on how to do bulk uploads with an excel file: https://www.youtube.com/watch?v=27jRUydL6qE

You'll need the following:

1. Install PyApacheAtlas and either get a service principal access to Purview or use the Azure CLI
2. Generate the excel template using the python -m pyapacheatlas --make-template ./purview.xlsx command
3. You will add the information to the BulkEntities tab.
   • Each row of the spreadsheet represents one column
   • Need the qualified name of the columns (e.g. mssql://server/db/schema/table#columnName)
   • Need the type name of the columns (e.g. azure_sql_column if it’s a column on an azure sql table)
4. For the bulk entities spreadsheet, you’ll need to add a column called “description” and fill in the descriptions for the given column.
5. you can then modify and run the sample script: https://github.com/wjohnson/pyapacheatlas/blob/master/samples/excel/excel_bulk_entities_upload.py
   • Please note that you should comment out lines 76 and 78 and update the file_path variable

Hope this helps.

---

• Please don't forget to click on and upvote button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how

Answer 2

For bulk updates I used this sample code to create an export. I updated the export using whatever tool I liked and then performed an import again. It would be usefull to have a look at this approach.

according to this sample https://github.com/pietrobr/azure-data-catalog-rest-python you can also directly access the data catalog from python. Unfortunately I do not have hands-on experience with that.