Synapselink Audit table source files

Question

Hi.

We have Synapselink for dataverse that send audit table data. The link creates audit and audit_partitioned tables automatically. Because of need to update spark pool there was pause in automatic table syncing. Now Audit_partitioned is missing some rows that are in Audit table. How does the synapselink create the audit table, are the rows stored in the csv or parquet files created by the synapselink? Because i would need to access them and store the missing rows into another sqldb.

Answer 1

Hi ,

Thanks for reaching out to Microsoft Q&A.

Synapse Link for Dataverse creates the Audit and Audit Partitioned tables automatically, storing their data in Delta format (Parquet files) within ADLS. The Audit table captures all audit log entries, while the Audit Partitioned table is optimized for performance by partitioning data. Since your Spark pool update caused a pause in the automatic syncing, the Audit table continued receiving new records while Audit Partitioned did not, leading to missing rows. These missing records still exist in the underlying storage files of the Audit table, and you can retrieve them by querying this table directly in Synapse. To restore the missing data, you can compare both tables, extract the missing rows from the Audit table, and insert them into your SQL database. Alternatively, you can access the data files in Azure Data Lake, process them using a Spark or Synapse pipeline, and load them into your desired destination. To prevent future issues, ensure that Synapse Link syncing is fully operational after updates.

Please feel free to click the 'Upvote' (Thumbs-up) button and 'Accept as Answer'. This helps the community by allowing others with similar queries to easily find the solution.