Refining a language service setup

Question

Refining a language service setup

Green, Jim 75

Trying to setup my first CLU model. I am using Language Studio, not LUIS.My application has an object called an Eligibility that a user can define. An eligibility consists of either an age and service pair, or a single "points" value. For the former, a user might say "Create an eligibility of 55 and 10" (age=55; service=10). For the latter, it would be "Create an eligibility of 80 points" (points=80). I have defined entities for Age, Service, and Points and the above utterances for each (with Age, Service, and Points marked appropriately). When I test the model deployment on "Create an eligibility of 80 points", it shows both Service and Points with a value of 80 and confidence of 100%.Is there a way to provide training data that indicates that the appearance of the word "points" indicates a point value?

AI suggested that I create a "pattern" like "Create an eligibility of {Points} points". Where in Language Studio do I define this? I've seen no mention of patterns in Language Studio so far.

Thanks.

SriLakshmi C 3,250 Reputation points Microsoft External Staff

2025-03-12T05:43:55.3433333+00:00
Hello Green, Jim,

To improve entity recognition in CLU, you can utilize various features since Language Studio does not support direct pattern-based rules like LUIS. One approach is to use Utterance Variations, where adding more diverse examples helps reinforce correct entity extraction. Another method is leveraging Phrase Lists, which allow you to boost specific words, such as "points", to strengthen their association with the intended entity.

Custom Features can be used to define specific conditions for entity extraction, ensuring more accurate and consistent results. By incorporating these techniques, you can refine the CLU model to better recognize and classify entities effectively.

Steps to Improve Entity Extraction:

Add More Training Utterances

To enhance entity recognition, provide additional diverse utterances that clearly differentiate Points from Service. By doing so, the model learns the correct associations and reduces ambiguity.

Example Training Data:

"Create an eligibility of 55 and 10" → (Age = 55, Service = 10)

"Create an eligibility of 80 points" → (Points = 80)

"Eligibility should be 70 points" → (Points = 70)

"I want to set eligibility with 40 and 20" → (Age = 40, Service = 20)

By incorporating these utterances, the model becomes more accurate in associating "points" with the Points entity, ensuring consistent and reliable entity extraction.

To improve entity recognition, create a Phrase List in Language Studio under Custom Features and name it Points Keyword. Add words like points, score, credit, threshold, and ranking, then associate it with the Points entity. This ensures the model correctly assigns values to Points whenever these keywords appear in an utterance.

After deploying the updated model, test edge cases where similar numbers appear without the word "points" to evaluate accuracy. If misclassifications still occur, refine the model by adding more training examples to reinforce correct entity recognition. Continuously testing and retraining will improve the model's performance.

I Hope this helps. Do let me know if you have any further queries.

Thank you!

2 answers

Your answer

Answer 1

In Language Studio, you can define patterns to improve the accuracy of your model's intent predictions. However, the specific mention of patterns is not detailed in the context provided. It is important to note that patterns are typically used in LUIS, which is being retired, and the focus is shifting towards conversational language understanding.

To address your issue with the model interpreting "points" incorrectly, you may want to explore the possibility of using entity roles or refining your training examples to clarify the context in which "points" is used. Unfortunately, the context does not provide specific instructions on where to define patterns in Language Studio.

For the most accurate guidance, I recommend checking the official documentation for Language Studio or reaching out to support for assistance with defining patterns in your specific setup.

References:

Answer 2

Green, Jim 75

Thanks for the quick response.

Utterances: so it would be helpful to add more utterances containing "## points" even if the phrasing is not what users would use?

Phrase List: could you point me to where I would find Custom Features in Language Studio? I don't see it anywhere.

Share via

Refining a language service setup

2 answers

Your answer