HuggingFacePromptExecutionSettings.UseCache Property
Definition
Important
Some information relates to prerelease product that may be substantially modified before it’s released. Microsoft makes no warranties, express or implied, with respect to the information provided here.
(Default: true). Boolean. There is a cache layer on the inference API to speedup requests we have already seen. Most models can use those results as is as models are deterministic (meaning the results will be the same anyway). However if you use a non deterministic model, you can set this parameter to prevent the caching mechanism from being used resulting in a real new query.
[System.Text.Json.Serialization.JsonPropertyName("use_cache")]
public bool UseCache { get; set; }[<System.Text.Json.Serialization.JsonPropertyName("use_cache")>]
member this.UseCache : bool with get, setPublic Property UseCache As BooleanProperty Value
- Attributes
Remarks
This may not be supported by all models/inference API.