RAI checks when training a fine-tuned model in Azure AI

Gabriela Ayala 0

Hi,

Our organization has approval to modify content filters, which we can apply when deploying a model. However, when first training a fine-tuned model, the job fails because some of the training data triggers RAI checks:

status : Validation file: Preprocessing Summary: The provided data failed RAI checks for harm types: violence (70). Please fix the data and try again. Details - Samples of harmful lines per harm type: violence: Line numbers -->

These training data points are essential to our classifier application so removing the flagged data points is not a useful option. If we already have authorization to set content filters, is there a way we can modify RAI checks when training fine-tuning jobs specifically?

Thank you for your support!

kothapally Snigdha 1,620 Reputation points Microsoft External Staff

2025-02-25T23:55:56.1933333+00:00
HI Gabriela Ayala,

Azure OpenAI Service conducts automatic RAI checks on training data to detect harmful content, such as violence, hate, and self-harm. These checks are part of the service's safety features and are applied uniformly across all fine-tuning jobs.

While you have authorization to set content filters for model deployment, these filters do not apply to the initial training phase. The RAI checks during training are predefined and cannot be customized by customers.

You might need to preprocess your data to comply with the RAI checks. This could involve masking or transforming sensitive content in a way that still allows your classifier to function effectively.

Consider using different AI models or services that offer more flexibility in handling sensitive data. However, this would require evaluating the capabilities and limitations of those alternatives.

If possible, implement your own content evaluation process before feeding the data into the Azure OpenAI Service. This could help you identify and adjust problematic content proactively.

kindly refer these document https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/fine-tuning?tabs=azure-openai%2Cpython-new&pivots=programming-language-studio

I hope these helps you. Thankyou!
Gabriela Ayala 0 Reputation points

2025-02-26T18:43:21.13+00:00

Thank you for your answer, very helpful! We will look into your suggestions!

Share via

RAI checks when training a fine-tuned model in Azure AI

Your answer