IntraopNumThreads interface
Provides an ability to change the number of threads used in the threadpool for Intra Operator Execution for CPU operators through LearningModelSessionOptions. By default, WinML sets the value as the maximum number of threads, which is the same number of logical cores on the user's CPU. Setting this value higher than the number of logical cores on the CPU may result in an inefficient threadpool and a slower evaluation.
Sample code
void SetIntraOpNumThreads(LearningModel model) {
// Create LearningModelSessionOptions
auto options = LearningModelSessionOptions();
auto nativeOptions = options.as<ILearningModelSessionOptionsNative>();
// Set the number of intra op threads to half of logical cores.
uint32_t desiredThreads = std::thread::hardware_concurrency() / 2;
nativeOptions->SetIntraOpNumThreadsOverride(desiredThreads);
// Create session
LearningModelSession session = nullptr;
WINML_EXPECT_NO_THROW(session = LearningModelSession(model, LearningModelDeviceKind::Cpu, options));
}
Requirements
Requirement | |
---|---|
Minimum supported client | Windows 10, build 17763 |
Minimum supported server | Windows Server 2019 with Desktop Experience |
Header | windows.ai.machinelearning.native.h |
Note
Use the following resources for help with Windows ML:
- To ask or answer technical questions about Windows ML, please use the windows-machine-learning tag on Stack Overflow.
- To report a bug, please file an issue on our GitHub.