You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The following provider options are **deprecated** and should be migrated to `load_config` for better compatibility with future releases.
90
+
The following provider options are **deprecated since ORT 1.23** and should be migrated to `load_config` for better compatibility with future releases.
@@ -147,7 +147,7 @@ Runs the same model on multiple devices in parallel to improve device utilizatio
147
147
---
148
148
149
149
### `precision`
150
-
**DEPRECATED:** This option is deprecated and can be set via `load_config` using the `INFERENCE_PRECISION_HINT` property.
150
+
**DEPRECATED:** This option is deprecated since OpenVINO 2025.3/ORT 1.23 and can be set via `load_config` using the `INFERENCE_PRECISION_HINT` property.
151
151
- Controls numerical precision during inference, balancing **performance** and **accuracy**.
152
152
153
153
**Precision Support on Devices:**
@@ -167,7 +167,7 @@ Runs the same model on multiple devices in parallel to improve device utilizatio
167
167
---
168
168
### `num_of_threads` & `num_streams`
169
169
170
-
**DEPRECATED:** These options are deprecated and can be set via `load_config` using the `INFERENCE_NUM_THREADS` and `NUM_STREAMS` properties respectively.
170
+
**DEPRECATED:** These options are deprecated since OpenVINO 2025.3/ORT 1.23 and can be set via `load_config` using the `INFERENCE_NUM_THREADS` and `NUM_STREAMS` properties respectively.
171
171
172
172
**Multi-Threading**
173
173
@@ -185,31 +185,33 @@ Manages parallel inference streams for throughput optimization (default: `1` for
185
185
186
186
### `cache_dir`
187
187
188
-
**DEPRECATED:** This option is deprecated and can be set via `load_config` using the `CACHE_DIR` property.
188
+
**DEPRECATED:** This option is deprecated since OpenVINO 2025.3/ORT 1.23 and can be set via `load_config` using the `CACHE_DIR` property.
189
189
190
-
Enables model caching to significantly reduce subsequent load times. Supports CPU, NPU, and GPU devices with kernel caching on iGPU/dGPU.
190
+
191
+
Enables model caching to significantly reduce subsequent load times. Supports CPU, NPU, and GPU devices with kernel caching on iGPU/dGPU.
191
192
192
193
**Benefits**
193
-
- Saves compiled models and `cl_cache` files for dynamic shapes
194
+
- Saves compiled models for faster subsequent loading
194
195
- Eliminates recompilation overhead on subsequent runs
195
-
- Particularly useful for complex models and frequent application restarts
196
-
196
+
- Particularly useful for optimizing application startup latencies, especially for complex models
197
197
198
198
---
199
199
200
200
### `load_config`
201
201
202
-
**Recommended Configuration Method** for setting OpenVINO runtime properties. Provides direct access to OpenVINO properties through a JSON configuration file during runtime.
202
+
**Recommended Configuration Method** for setting OpenVINO runtime properties. Provides direct access to OpenVINO properties through a JSON String during runtime.
203
203
204
204
#### Overview
205
205
206
-
`load_config` enables fine-grained control over OpenVINO inference behavior by loading properties from a JSON file. This is the **preferred method** for configuring advanced OpenVINO features, offering:
206
+
`load_config` enables fine-grained control over OpenVINO inference behavior by loading properties from a JSON String. This is the **preferred method** for configuring advanced OpenVINO features, offering:
207
207
208
208
- Direct access to OpenVINO runtime properties
209
209
- Device-specific configuration
210
210
- Better compatibility with future OpenVINO releases
211
211
- No property name translation required
212
212
213
+
214
+
213
215
#### JSON Configuration Format
214
216
```json
215
217
{
@@ -219,6 +221,33 @@ Enables model caching to significantly reduce subsequent load times. Supports CP
219
221
}
220
222
```
221
223
224
+
`load_config` now supports nested JSON objects up to **8 levels deep** for complex device configurations.
225
+
226
+
**Maximum Nesting:** 8 levels deep.
227
+
228
+
**Example: Multi-Level Nested Configuration**
229
+
```python
230
+
import onnxruntime as ort
231
+
import json
232
+
233
+
# Complex nested configuration for AUTO device
234
+
config = {
235
+
"AUTO": {
236
+
"PERFORMANCE_HINT": "THROUGHPUT",
237
+
"DEVICE_PROPERTIES": {
238
+
"CPU": {
239
+
"PERFORMANCE_HINT": "LATENCY",
240
+
"NUM_STREAMS": "3"
241
+
},
242
+
"GPU": {
243
+
"EXECUTION_MODE_HINT": "ACCURACY",
244
+
"PERFORMANCE_HINT": "LATENCY"
245
+
}
246
+
}
247
+
}
248
+
}
249
+
```
250
+
222
251
**Supported Device Names:**
223
252
-`"CPU"` - Intel CPU
224
253
-`"GPU"` - Intel integrated/discrete GPU
@@ -327,7 +356,7 @@ Property keys used in `load_config` JSON must match the string literal defined i
327
356
328
357
### `enable_qdq_optimizer`
329
358
330
-
**DEPRECATED:** This option is deprecated and can be set via `load_config` using the `NPU_QDQ_OPTIMIZATION` property.
359
+
**DEPRECATED:** This option is deprecated since OpenVINO 2025.3/ORT 1.23 and can be set via `load_config` using the `NPU_QDQ_OPTIMIZATION` property.
331
360
332
361
NPU-specific optimization for Quantize-Dequantize (QDQ) operations in the inference graph. This optimizer enhances ORT quantized models by:
333
362
@@ -362,7 +391,7 @@ This configuration is required for optimal NPU memory allocation and management.
362
391
363
392
### `model_priority`
364
393
365
-
**DEPRECATED:** This option is deprecated and can be set via `load_config` using the `MODEL_PRIORITY` property.
394
+
**DEPRECATED:** This option is deprecated since OpenVINO 2025.3/ORT 1.23 and can be set via `load_config` using the `MODEL_PRIORITY` property.
366
395
367
396
Configures resource allocation priority for multi-model deployment scenarios.
0 commit comments