[Bug]: NPU not detected in OpenVINO container #24496

gooishin · 2024-05-14T03:58:19Z

OpenVINO Version

openvino/ubuntu22_dev:latest

Operating System

Other (Please specify in description)

Device used for inference

NPU

Framework

None

Model used

No response

Issue description

I install openvino using docker pull openvino/ubuntu22_dev:latest . When i try to run and query device, NPU is not listed.

Step-by-step reproduction

docker pull openvino/ubuntu22_dev:latest
Download hello_query_device.py sample script into my workspace /data/workspace
docker run -it -v /etc/group:/etc/group --device=/dev/dri --device=/dev/accel --group-add=$(stat -c "%g" /dev/dri/render* | head -n 1) --group-add=$(stat -c "%g" /dev/dri/card* | head -n 1) -v /usr/bin:/usr/bin -v /home/user/workspace:/data/workspace -w /data/workspace openvino/ubuntu22_dev:latest
python3 hello_query_device.py

Relevant log output

$ docker run -it -v /etc/group:/etc/group --device=/dev/dri --device=/dev/accel --group-add=$(stat -c "%g" /dev/dri/render* | head -n 1) --group-add=$(stat -c "%g" /dev/dri/card* | head -n 1) -v /usr/bin:/usr/bin -v /home/user/workspace:/data/workspace -w /data/workspace openvino/ubuntu22_dev:latest /bin/bash

openvino@c7fc79ee23c7:/data/workspace$ python3 hello_query_device.py 

[ INFO ] Available devices:

[ INFO ] CPU :

[ INFO ] 	SUPPORTED_PROPERTIES:

[ INFO ] 		AVAILABLE_DEVICES: 

[ INFO ] 		RANGE_FOR_ASYNC_INFER_REQUESTS: 1, 1, 1

[ INFO ] 		RANGE_FOR_STREAMS: 1, 22

[ INFO ] 		EXECUTION_DEVICES: CPU

[ INFO ] 		FULL_DEVICE_NAME: Intel(R) Core(TM) Ultra 7 165H

[ INFO ] 		OPTIMIZATION_CAPABILITIES: FP32, FP16, INT8, BIN, EXPORT_IMPORT

[ INFO ] 		DEVICE_TYPE: Type.INTEGRATED

[ INFO ] 		DEVICE_ARCHITECTURE: intel64

[ INFO ] 		NUM_STREAMS: 1

[ INFO ] 		AFFINITY: Affinity.HYBRID_AWARE

[ INFO ] 		INFERENCE_NUM_THREADS: 0

[ INFO ] 		PERF_COUNT: False

[ INFO ] 		INFERENCE_PRECISION_HINT: <Type: 'float32'>

[ INFO ] 		PERFORMANCE_HINT: PerformanceMode.LATENCY

[ INFO ] 		EXECUTION_MODE_HINT: ExecutionMode.PERFORMANCE

[ INFO ] 		PERFORMANCE_HINT_NUM_REQUESTS: 0

[ INFO ] 		ENABLE_CPU_PINNING: True

[ INFO ] 		SCHEDULING_CORE_TYPE: SchedulingCoreType.ANY_CORE

[ INFO ] 		MODEL_DISTRIBUTION_POLICY: set()

[ INFO ] 		ENABLE_HYPER_THREADING: True

[ INFO ] 		DEVICE_ID: 

[ INFO ] 		CPU_DENORMALS_OPTIMIZATION: False

[ INFO ] 		LOG_LEVEL: Level.NO

[ INFO ] 		CPU_SPARSE_WEIGHTS_DECOMPRESSION_RATE: 1.0

[ INFO ] 		DYNAMIC_QUANTIZATION_GROUP_SIZE: 0

[ INFO ] 		KV_CACHE_PRECISION: <Type: 'float16'>

[ INFO ] 

[ INFO ] GPU :

[ INFO ] 	SUPPORTED_PROPERTIES:

[ INFO ] 		AVAILABLE_DEVICES: 0

[ INFO ] 		RANGE_FOR_ASYNC_INFER_REQUESTS: 1, 2, 1

[ INFO ] 		RANGE_FOR_STREAMS: 1, 2

[ INFO ] 		OPTIMAL_BATCH_SIZE: 1

[ INFO ] 		MAX_BATCH_SIZE: 1

[ INFO ] 		DEVICE_ARCHITECTURE: GPU: vendor=0x8086 arch=v785.128.0

[ INFO ] 		FULL_DEVICE_NAME: Intel(R) Graphics [0x7d55] (iGPU)

[ INFO ] 		DEVICE_UUID: 8680557d080000000002000000000000

[ INFO ] 		DEVICE_LUID: 409a0000499a0000

[ INFO ] 		DEVICE_TYPE: Type.INTEGRATED

[ INFO ] 		DEVICE_GOPS: {<Type: 'float16'>: 9420.7998046875, <Type: 'float32'>: 4710.39990234375, <Type: 'int8_t'>: 18841.599609375, <Type: 'uint8_t'>: 18841.599609375}

[ INFO ] 		OPTIMIZATION_CAPABILITIES: FP32, BIN, FP16, INT8, EXPORT_IMPORT

[ INFO ] 		GPU_DEVICE_TOTAL_MEM_SIZE: 26629963776

[ INFO ] 		GPU_UARCH_VERSION: 785.128.0

[ INFO ] 		GPU_EXECUTION_UNITS_COUNT: 128

[ INFO ] 		GPU_MEMORY_STATISTICS: {}

[ INFO ] 		PERF_COUNT: False

[ INFO ] 		MODEL_PRIORITY: Priority.MEDIUM

[ INFO ] 		GPU_HOST_TASK_PRIORITY: Priority.MEDIUM

[ INFO ] 		GPU_QUEUE_PRIORITY: Priority.MEDIUM

[ INFO ] 		GPU_QUEUE_THROTTLE: Priority.MEDIUM

[ INFO ] 		GPU_ENABLE_LOOP_UNROLLING: True

[ INFO ] 		GPU_DISABLE_WINOGRAD_CONVOLUTION: False

[ INFO ] 		CACHE_DIR: 

[ INFO ] 		CACHE_MODE: CacheMode.OPTIMIZE_SPEED

[ INFO ] 		PERFORMANCE_HINT: PerformanceMode.LATENCY

[ INFO ] 		EXECUTION_MODE_HINT: ExecutionMode.PERFORMANCE

[ INFO ] 		COMPILATION_NUM_THREADS: 22

[ INFO ] 		NUM_STREAMS: 1

[ INFO ] 		PERFORMANCE_HINT_NUM_REQUESTS: 0

[ INFO ] 		INFERENCE_PRECISION_HINT: <Type: 'float16'>

[ INFO ] 		ENABLE_CPU_PINNING: False

[ INFO ] 		DEVICE_ID: 0

[ INFO ]

Issue submission checklist

I'm reporting an issue. It's not a question.
I checked the problem with the documentation, FAQ, open issues, Stack Overflow, etc., and have not found a solution.
There is reproducer code and related data files such as images, videos, models, etc.

gooishin · 2024-05-14T04:00:27Z

natively run hello_query_device.py able to get NPU. Issue only happen when run in container

$ python3 hello_query_device.py 

[ INFO ] Available devices:

[ INFO ] CPU :

[ INFO ] 	SUPPORTED_PROPERTIES:

[ INFO ] 		AVAILABLE_DEVICES: 

[ INFO ] 		RANGE_FOR_ASYNC_INFER_REQUESTS: 1, 1, 1

[ INFO ] 		RANGE_FOR_STREAMS: 1, 22

[ INFO ] 		EXECUTION_DEVICES: CPU

[ INFO ] 		FULL_DEVICE_NAME: Intel(R) Core(TM) Ultra 7 165H

[ INFO ] 		OPTIMIZATION_CAPABILITIES: FP32, FP16, INT8, BIN, EXPORT_IMPORT

[ INFO ] 		DEVICE_TYPE: Type.INTEGRATED

[ INFO ] 		DEVICE_ARCHITECTURE: intel64

[ INFO ] 		NUM_STREAMS: 1

[ INFO ] 		AFFINITY: Affinity.HYBRID_AWARE

[ INFO ] 		INFERENCE_NUM_THREADS: 0

[ INFO ] 		PERF_COUNT: False

[ INFO ] 		INFERENCE_PRECISION_HINT: <Type: 'float32'>

[ INFO ] 		PERFORMANCE_HINT: PerformanceMode.LATENCY

[ INFO ] 		EXECUTION_MODE_HINT: ExecutionMode.PERFORMANCE

[ INFO ] 		PERFORMANCE_HINT_NUM_REQUESTS: 0

[ INFO ] 		ENABLE_CPU_PINNING: True

[ INFO ] 		SCHEDULING_CORE_TYPE: SchedulingCoreType.ANY_CORE

[ INFO ] 		MODEL_DISTRIBUTION_POLICY: set()

[ INFO ] 		ENABLE_HYPER_THREADING: True

[ INFO ] 		DEVICE_ID: 

[ INFO ] 		CPU_DENORMALS_OPTIMIZATION: False

[ INFO ] 		LOG_LEVEL: Level.NO

[ INFO ] 		CPU_SPARSE_WEIGHTS_DECOMPRESSION_RATE: 1.0

[ INFO ] 		DYNAMIC_QUANTIZATION_GROUP_SIZE: 0

[ INFO ] 		KV_CACHE_PRECISION: <Type: 'float16'>

[ INFO ] 

[ INFO ] GPU :

[ INFO ] 	SUPPORTED_PROPERTIES:

[ INFO ] 		AVAILABLE_DEVICES: 0

[ INFO ] 		RANGE_FOR_ASYNC_INFER_REQUESTS: 1, 2, 1

[ INFO ] 		RANGE_FOR_STREAMS: 1, 2

[ INFO ] 		OPTIMAL_BATCH_SIZE: 1

[ INFO ] 		MAX_BATCH_SIZE: 1

[ INFO ] 		DEVICE_ARCHITECTURE: GPU: vendor=0x8086 arch=v785.192.4

[ INFO ] 		FULL_DEVICE_NAME: Intel(R) Arc(TM) Graphics (iGPU)

[ INFO ] 		DEVICE_UUID: 8680557d080000000002000000000000

[ INFO ] 		DEVICE_LUID: 409a0000499a0000

[ INFO ] 		DEVICE_TYPE: Type.INTEGRATED

[ INFO ] 		DEVICE_GOPS: {<Type: 'float16'>: 9420.7998046875, <Type: 'float32'>: 4710.39990234375, <Type: 'int8_t'>: 18841.599609375, <Type: 'uint8_t'>: 18841.599609375}

[ INFO ] 		OPTIMIZATION_CAPABILITIES: FP32, BIN, FP16, INT8, EXPORT_IMPORT

[ INFO ] 		GPU_DEVICE_TOTAL_MEM_SIZE: 30818349056

[ INFO ] 		GPU_UARCH_VERSION: 785.192.4

[ INFO ] 		GPU_EXECUTION_UNITS_COUNT: 128

[ INFO ] 		GPU_MEMORY_STATISTICS: {}

[ INFO ] 		PERF_COUNT: False

[ INFO ] 		MODEL_PRIORITY: Priority.MEDIUM

[ INFO ] 		GPU_HOST_TASK_PRIORITY: Priority.MEDIUM

[ INFO ] 		GPU_QUEUE_PRIORITY: Priority.MEDIUM

[ INFO ] 		GPU_QUEUE_THROTTLE: Priority.MEDIUM

[ INFO ] 		GPU_ENABLE_LOOP_UNROLLING: True

[ INFO ] 		GPU_DISABLE_WINOGRAD_CONVOLUTION: False

[ INFO ] 		CACHE_DIR: 

[ INFO ] 		CACHE_MODE: CacheMode.OPTIMIZE_SPEED

[ INFO ] 		PERFORMANCE_HINT: PerformanceMode.LATENCY

[ INFO ] 		EXECUTION_MODE_HINT: ExecutionMode.PERFORMANCE

[ INFO ] 		COMPILATION_NUM_THREADS: 22

[ INFO ] 		NUM_STREAMS: 1

[ INFO ] 		PERFORMANCE_HINT_NUM_REQUESTS: 0

[ INFO ] 		INFERENCE_PRECISION_HINT: <Type: 'float16'>

[ INFO ] 		ENABLE_CPU_PINNING: False

[ INFO ] 		DEVICE_ID: 0

[ INFO ] 

[ INFO ] NPU :

[ INFO ] 	SUPPORTED_PROPERTIES:

[ INFO ] 		AVAILABLE_DEVICES: 3720

[ INFO ] 		CACHE_DIR: 

[ INFO ] 		CACHING_PROPERTIES: {'DEVICE_ARCHITECTURE': 'RW', 'NPU_COMPILATION_MODE_PARAMS': 'RW', 'NPU_TILES': 'RW', 'NPU_DPU_GROUPS': 'RW', 'NPU_DMA_ENGINES': 'RW', 'NPU_COMPILATION_MODE': 'RW', 'NPU_DRIVER_VERSION': 'RW', 'NPU_COMPILER_TYPE': 'RW', 'NPU_USE_ELF_COMPILER_BACKEND': 'RW'}

[ INFO ] 		COMPILATION_NUM_THREADS: 22

[ INFO ] 		DEVICE_ARCHITECTURE: 3720

[ INFO ] 		DEVICE_ID: 

[ INFO ] 		DEVICE_UUID: 80d1d11eb73811eab3de0242ac130004

[ INFO ] 		ENABLE_CPU_PINNING: False

[ INFO ] 		EXCLUSIVE_ASYNC_REQUESTS: False

[ INFO ] 		FULL_DEVICE_NAME: Intel(R) AI Boost

[ INFO ] 		INFERENCE_PRECISION_HINT: <Type: 'float16'>

[ INFO ] 		INTERNAL_SUPPORTED_PROPERTIES: {'CACHING_PROPERTIES': 'RW'}

[ INFO ] 		LOG_LEVEL: Level.NO

[ INFO ] 		NPU_DEVICE_ALLOC_MEM_SIZE: 0

[ INFO ] 		NPU_DEVICE_TOTAL_MEM_SIZE: 33287458816

[ INFO ] 		NPU_DRIVER_VERSION: 16866217

[ INFO ] 		NUM_STREAMS: 1

[ INFO ] 		OPTIMAL_NUMBER_OF_INFER_REQUESTS: 1

[ INFO ] 		OPTIMIZATION_CAPABILITIES: FP16, INT8, EXPORT_IMPORT

[ INFO ] 		PERFORMANCE_HINT: PerformanceMode.LATENCY

[ INFO ] 		PERFORMANCE_HINT_NUM_REQUESTS: 1

[ INFO ] 		PERF_COUNT: False

[ INFO ] 		RANGE_FOR_ASYNC_INFER_REQUESTS: 1, 10, 1

[ INFO ] 		RANGE_FOR_STREAMS: 1, 4

[ INFO ]

gooishin · 2024-05-23T09:24:22Z

I manage to resolve this. In container i still need to install npu driver again even though i have natively install. It doesnt work like GPU auto load driver inside container.

gooishin added bug Something isn't working support_request labels May 14, 2024

andrei-kochin assigned mlyashko May 14, 2024

andrei-kochin added the category: docker_env label May 14, 2024

andrei-kochin assigned dtrawins May 14, 2024

ilya-lavrenov assigned PatrikStepan and artanokhov and unassigned dtrawins May 14, 2024

avitial added the category: NPU OpenVINO NPU plugin label May 14, 2024

gooishin closed this as completed May 23, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Bug]: NPU not detected in OpenVINO container #24496

[Bug]: NPU not detected in OpenVINO container #24496

gooishin commented May 14, 2024

gooishin commented May 14, 2024

gooishin commented May 23, 2024

[Bug]: NPU not detected in OpenVINO container #24496

[Bug]: NPU not detected in OpenVINO container #24496

Comments

gooishin commented May 14, 2024

OpenVINO Version

Operating System

Device used for inference

Framework

Model used

Issue description

Step-by-step reproduction

Relevant log output

Issue submission checklist

gooishin commented May 14, 2024

gooishin commented May 23, 2024