[Questionnaire] Which model do you want to support in mmdeploy? #170

VVsssssk · 2022-02-18T11:02:11Z

Which model do you want to support in mmdeploy? Comment here. You can like a comment to increase model priority.

	Model	Codebase	Backend	Number of likes	Released
1	Swin-Transformer	mmcls	tensorrt	2	v0.6.0
2	Second	mmdet3d	tensorrt	1
3	htc	mmdet	tensorrt	1
4	Segmenter	mmseg	tensorrt	1	v0.6.0
5	ConvNeXt	mmcls	tensorrt	1
6	MASTER	mmocr	tensorrt	1
7	ABINet	mmocr	tensorrt	1
8	BEVFormer	mmdet3d	tensorrt	1
9	Oriented R-CNN	mmrotate	tensorrt	1	v0.6.0
10	SAR	mmocr	tensorrt	1

Reference:

VVsssssk · 2022-02-18T11:03:07Z

E.g:
Swin-Transformer mmcls tensorrt

OrcunCanDeniz · 2022-02-18T13:53:30Z

second mmdet3d tensorrt

haofanwang · 2022-02-19T07:57:36Z

htc mmdetection tensorrt

haofanwang · 2022-02-19T07:58:03Z

segmenter mmsegmentation tensorrt

starhou · 2022-02-19T11:01:43Z

swin-transformer mmdet tensorrt

starhou · 2022-02-21T06:59:41Z

I haven't tried the conversion yet and I just used your example fater-rcnn. Maybe now, need to try.

RunningLeon · 2022-02-23T07:17:32Z

segmenter mmsegmentation tensorrt

@haofanwang Have you tried segmenter with mmdeploy?

haofanwang · 2022-02-23T09:11:22Z

segmenter mmsegmentation tensorrt

@haofanwang Have you tried segmenter with mmdeploy?

@RunningLeon , Yes, but I receive error messages as below

TypeError: upsample_bicubic2d() received an invalid combination of arguments - got (Tensor, NoneType, bool, list), but expected one of:
 * (Tensor input, tuple of ints output_size, bool align_corners, tuple of floats scale_factors)
      didn't match because some of the arguments have invalid types: (Tensor, NoneType, bool, list)
 * (Tensor input, tuple of ints output_size, bool align_corners, float scales_h, float scales_w, *, Tensor out)

And my command is

python ./tools/deploy.py \
    ./configs/mmseg/segmentation_tensorrt-fp16_dynamic-512x1024-2048x2048.py \
    ./mmsegmentation/configs/segmenter/segmenter_vit-l_mask_8x1_512x512_160k_ade20k.py \
    ./mmsegmentation/checkpoints/segmenter_vit-l_mask_8x1_512x512_160k_ade20k_20220105_162750-7ef345be.pth \
    ./mmsegmentation/demo/demo.png \
    --work-dir work_dir\
    --device cuda\
    --dump-in

RunningLeon · 2022-02-24T03:20:41Z

512x512
@haofanwang
Try with static config with 512x512. You can change in config file.

eliabruni · 2022-03-02T20:54:29Z

maskformer mmdet tensorrt

isbrycee · 2022-03-11T02:13:24Z

swin-transformer mmdet onnx

Siannodel · 2022-03-14T03:44:46Z

Swin-Transformer mmcls tensorrt

shannanyinxiang · 2022-03-17T13:23:19Z

SATRN mmocr tensorrt

awesome01010 · 2022-03-17T14:18:32Z

SATRN for ocr, to onnx and tensorrt pls!

summerlvsong · 2022-03-18T09:25:25Z

SATRN for ocr, to onnx and tensorrt pls!

Mountchicken · 2022-03-22T12:47:04Z

SATRN MMOCR tensorrt

Xiangxu-0103 · 2022-03-29T07:03:05Z

dynamic_voxelization mmdet3d onnxruntime and tensorrt

YongboLiang · 2022-03-29T07:46:37Z

Swin-Transformer Second Segmenter

hust-lidelong · 2022-03-30T09:38:25Z

Oriented R-CNN，S2A-Net for mmrotate tensorrt, please!

eliabruni · 2022-04-04T06:44:16Z

mask2former mmdet tensorrt

jianghaijun007 · 2022-04-07T03:42:27Z

monoflex tensorrt

gzxy-0102 · 2022-04-11T01:02:59Z

mmtracking

troyliu0105 · 2022-04-13T05:29:12Z

AssociativeEmbedding pose detection in mmpose

SpaceView · 2022-04-14T15:43:13Z

mmocr

RobinHan24 · 2023-08-04T01:38:40Z

DINO mmdet tensorrt

RunningLeon · 2023-08-11T04:00:35Z

maskformer mmdet tensorrt

@eliabruni hi. If you have time, pls. try this PR #2347. Any comment on the PR is greatly appreciated.

ycdhqzhiai · 2023-08-15T09:17:11Z

fcos3d mmdet3d tensorrt

DuckJ · 2023-08-17T07:41:39Z

dino、dinomask mmdet onnx.

Zaughter · 2023-09-07T23:31:12Z

dab-detr onnx

RunningLeon · 2023-09-08T01:07:04Z

dab-detr onnx

@Zaughter @DuckJ @sibet-lb @RobinHan24 @idonahum1
hi, for DETR, DINO, Dab-DETR, Conditional-DETR, could try this PR##2410

RobinHan24 · 2023-09-10T02:56:05Z

dab-detr onnx

@Zaughter @DuckJ @sibet-lb @RobinHan24 @idonahum1 hi, for DETR, DINO, Dab-DETR, Conditional-DETR, could try this PR##2410

I just tried it, but error.

2023-09-10:10:53:30 - root - ERROR - execute_async_v2(): incompatible function arguments. The following argument types are supported:
1. (self: tensorrt.tensorrt.IExecutionContext, bindings: List[int], stream_handle: int, input_consumed: capsule = None) -> bool

Invoked with: <tensorrt.tensorrt.IExecutionContext object at 0x7f01857b5bb0>, [139631165177856, None, 139634709371904, 139634709365760], 0
Traceback (most recent call last):
File "/mnt/sda/qilibin/project/mmdeploy-support_detr_multi-batch/mmdeploy/utils/utils.py", line 41, in target_wrapper
result = target(*args, **kwargs)
File "/mnt/sda/qilibin/project/mmdeploy-support_detr_multi-batch/mmdeploy/apis/visualize.py", line 72, in visualize_model
result = model.test_step(model_inputs)[0]
File "/mnt/sda/qilibin/miniconda3/envs/py3.9/lib/python3.9/site-packages/mmengine/model/base_model/base_model.py", line 145, in test_step
return self._run_forward(data, mode='predict') # type: ignore
File "/mnt/sda/qilibin/miniconda3/envs/py3.9/lib/python3.9/site-packages/mmengine/model/base_model/base_model.py", line 340, in _run_forward
results = self(**data, mode=mode)
File "/mnt/sda/qilibin/miniconda3/envs/py3.9/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/mnt/sda/qilibin/project/mmdeploy-support_detr_multi-batch/mmdeploy/codebase/mmdet/deploy/object_detection_model.py", line 299, in forward
outputs = self.predict(inputs, shape_info)
File "/mnt/sda/qilibin/project/mmdeploy-support_detr_multi-batch/mmdeploy/codebase/mmdet/deploy/object_detection_model.py", line 323, in predict
outputs = self.wrapper(inputs)
File "/mnt/sda/qilibin/miniconda3/envs/py3.9/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/mnt/sda/qilibin/project/mmdeploy-support_detr_multi-batch/mmdeploy/backend/tensorrt/wrapper.py", line 174, in forward
self.__trt_execute(bindings=bindings)
File "/mnt/sda/qilibin/project/mmdeploy-support_detr_multi-batch/mmdeploy/utils/timer.py", line 67, in fun
result = func(*args, **kwargs)
File "/mnt/sda/qilibin/project/mmdeploy-support_detr_multi-batch/mmdeploy/backend/tensorrt/wrapper.py", line 185, in __trt_execute
self.context.execute_async_v2(bindings,
TypeError: execute_async_v2(): incompatible function arguments. The following argument types are supported:
1. (self: tensorrt.tensorrt.IExecutionContext, bindings: List[int], stream_handle: int, input_consumed: capsule = None) -> bool

Invoked with: <tensorrt.tensorrt.IExecutionContext object at 0x7f01857b5bb0>, [139631165177856, None, 139634709371904, 139634709365760], 0
09/10 10:53:32 - mmengine - ERROR - /mnt/sda/qilibin/project/mmdeploy-support_detr_multi-batch/tools/deploy.py - create_process - 82 - visualize tensorrt model failed.

tensorrt version: 8.6.1.6

command:
python tools/deploy.py configs/mmdet/detection/detection_tensorrt_dynamic-320x320-1344x1344.py dino/dino-4scale_r50_8xb2-36e_coco.py dino-4scale_r50_8xb2-12e_coco_20221202_182705-55b2bba2.pth demo/resources/det.jpg --work-dir mmdeploy_models/mmdet/dino --device cuda --show --dump-info

could you please help me？

RobinHan24 · 2023-09-10T04:53:45Z

dab-detr onnx

@Zaughter @DuckJ @sibet-lb @RobinHan24 @idonahum1 hi, for DETR, DINO, Dab-DETR, Conditional-DETR, could try this PR##2410
thanks a lot, I have got the tensorrt model. But how to infer with trt model sdk?
my code:
from mmdeploy_runtime import Detector
import cv2
img = cv2.imread('./demo/resources/det.jpg')
detector = Detector(model_path='mmdeploy_models/mmdet/dino0910', device_name='cuda', device_id=0)
res = detector(img)
print (res)
indices = [i for i in range(len(bboxes))]
for index, bbox, label_id in zip(indices, bboxes, labels):
[left, top, right, bottom], score = bbox[0:4].astype(int), bbox[4]
if score < 0.3:
continue
cv2.rectangle(img, (left, top), (right, bottom), (0, 255, 0))
cv2.imwrite('output_detection.png', img)

error:
[2023-09-10 12:51:09.788] [mmdeploy] [error] [net_module.cpp:130] sample 0 missing key shape

can you help me, thanks.

MarouaneMja · 2023-09-13T15:14:37Z

CO-DETR https://github.com/Sense-X/Co-DETR -> onnx and tensorrt

x-yy0 · 2023-10-19T03:17:40Z

mvxnet mmdet3d tensorrt

mjehanzaib999 · 2023-10-31T11:33:39Z

slowonly(spatiotemporal) mmaction tensorrt

DoctorDream · 2023-11-09T01:28:32Z

Co-DETR please!

Mayyyybe · 2023-11-09T07:03:05Z

Co-DETR to onnx please！！！

MarouaneMja · 2023-11-10T12:47:27Z

@DoctorDream @Mayyyybe I already mentionned CO-DETR, Could you please upvote my suggestion , so that it could increase this model priority?

Baboom-l · 2023-11-20T02:31:17Z

Grounding DINO mmdetection tensorrt

xinlin-xiao · 2023-11-20T06:33:37Z

CO-DETR https://github.com/Sense-X/Co-DETR -> onnx and tensorrt

CO-DETR->onnx ,TorchScript,tensorrt

OSSome01 · 2023-12-07T19:23:50Z

KNet mmsegmentaion
ConvNext mmsegmentation

WaseemDR · 2023-12-17T11:54:44Z

TensorRT for Grounding Dino

Jialeen · 2024-01-04T03:13:16Z

TensorRT for Detic

iceriver97 · 2024-01-09T06:45:44Z

fcos3d mmdet3d tensorrt

这个目前实现了吗？

chunxt · 2024-01-27T03:44:12Z

tood. thanks

Nieees · 2024-01-31T06:28:35Z

htc Series models （scnet， detectRS） Is there any progress so far?

Bovey0809 · 2024-02-19T06:40:14Z

MMRazor QAT

monologuesmw · 2024-02-19T07:39:25Z

TOOD model. Thanks!

monkeycc · 2024-04-13T00:15:28Z

YOLOv8
[Detect]
[Segment]
[Classify]
[Pose]
[OBB]

ZJU-lishuang · 2024-04-18T02:12:58Z

glip | mmdet | tensorrt

chenzhutian · 2024-04-19T20:57:20Z

3D pose (eg. MotionBert) in mmPose

xlg-go · 2024-05-09T10:43:09Z

SVTR for ocr, to onnx and tensorRT pls!

MarouaneMja · 2024-05-22T13:15:51Z

CO-DETR https://github.com/Sense-X/Co-DETR -> onnx and tensorrt

Hello @VVsssssk ,

Any updates about the exportation of Co-DETR to onnx and tensorrt please?

Best regards,
Marouane

VVsssssk pinned this issue Feb 18, 2022

RunningLeon assigned VVsssssk Feb 18, 2022

RunningLeon mentioned this issue Feb 21, 2022

CARAFE op to onnx Fails and why not support it yet? #172

Closed

Mountchicken mentioned this issue Mar 29, 2022

How do I can transfer psenet model to tensorrt model in mmocr? open-mmlab/mmocr#895

Closed

RunningLeon mentioned this issue Mar 30, 2022

mmrotate support in mmdeploy #289

Closed

RunningLeon mentioned this issue Apr 7, 2022

KeyError: 'ori_shape' #311

Closed

RunningLeon mentioned this issue Sep 18, 2023

Adding CO-DETR export to onnx #2440

Closed

[Questionnaire] Which model do you want to support in mmdeploy? #170

[Questionnaire] Which model do you want to support in mmdeploy? #170

Comments

VVsssssk commented Feb 18, 2022 • edited by lvhan028

VVsssssk commented Feb 18, 2022

OrcunCanDeniz commented Feb 18, 2022

haofanwang commented Feb 19, 2022

haofanwang commented Feb 19, 2022 • edited

starhou commented Feb 19, 2022 • edited

starhou commented Feb 21, 2022 • edited

RunningLeon commented Feb 23, 2022

haofanwang commented Feb 23, 2022

RunningLeon commented Feb 24, 2022

eliabruni commented Mar 2, 2022

isbrycee commented Mar 11, 2022

Siannodel commented Mar 14, 2022

shannanyinxiang commented Mar 17, 2022

awesome01010 commented Mar 17, 2022

summerlvsong commented Mar 18, 2022

Mountchicken commented Mar 22, 2022 • edited

Xiangxu-0103 commented Mar 29, 2022

YongboLiang commented Mar 29, 2022

hust-lidelong commented Mar 30, 2022

eliabruni commented Apr 4, 2022

jianghaijun007 commented Apr 7, 2022

gzxy-0102 commented Apr 11, 2022

troyliu0105 commented Apr 13, 2022

SpaceView commented Apr 14, 2022

RobinHan24 commented Aug 4, 2023

RunningLeon commented Aug 11, 2023

ycdhqzhiai commented Aug 15, 2023

DuckJ commented Aug 17, 2023

Zaughter commented Sep 7, 2023

RunningLeon commented Sep 8, 2023 • edited

RobinHan24 commented Sep 10, 2023

RobinHan24 commented Sep 10, 2023 • edited

MarouaneMja commented Sep 13, 2023 • edited

x-yy0 commented Oct 19, 2023

mjehanzaib999 commented Oct 31, 2023

DoctorDream commented Nov 9, 2023

Mayyyybe commented Nov 9, 2023

MarouaneMja commented Nov 10, 2023

Baboom-l commented Nov 20, 2023

xinlin-xiao commented Nov 20, 2023

OSSome01 commented Dec 7, 2023

WaseemDR commented Dec 17, 2023

Jialeen commented Jan 4, 2024

iceriver97 commented Jan 9, 2024

chunxt commented Jan 27, 2024

Nieees commented Jan 31, 2024

Bovey0809 commented Feb 19, 2024

monologuesmw commented Feb 19, 2024

monkeycc commented Apr 13, 2024

ZJU-lishuang commented Apr 18, 2024

chenzhutian commented Apr 19, 2024 • edited

xlg-go commented May 9, 2024

MarouaneMja commented May 22, 2024

VVsssssk commented Feb 18, 2022 •

edited by lvhan028

haofanwang commented Feb 19, 2022 •

edited

starhou commented Feb 19, 2022 •

edited

starhou commented Feb 21, 2022 •

edited

Mountchicken commented Mar 22, 2022 •

edited

RunningLeon commented Sep 8, 2023 •

edited

RobinHan24 commented Sep 10, 2023 •

edited

MarouaneMja commented Sep 13, 2023 •

edited

chenzhutian commented Apr 19, 2024 •

edited