Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How it Modify for YOLOV8 pose for multi classes #190

Open
faithmaker opened this issue Jan 11, 2024 · 3 comments
Open

How it Modify for YOLOV8 pose for multi classes #190

faithmaker opened this issue Jan 11, 2024 · 3 comments

Comments

@faithmaker
Copy link

The model learned with multi-class(3) is performed normally with the yolov8 cli Predict command.
I confirmed through debugging that only one class objects was detected when using the example here.

*. you recommed.
You should modify pose postprocess.
Default is only one class so that the score is only for person.
But if enable multiclass you should use argmax and max to find the highest score and label.
=> Where should I fix it?

*. I check it.

YOLOv8-TensorRT\csrc\pose\normal\include\yolov8-pose.hpp
....
void YOLOv8_pose::postprocess(std::vector& objs, float score_thres, float iou_thres, int topk)
{
...
auto kps_ptr = row_ptr + 5;

// classnum = 3
auto maxSPtr = std::max_element(kps_ptr, kps_ptr + 3);
// auto maxSPtr = std::max_element(kps_ptr+2, kps_ptr + 3);
// auto maxSPtr = std::max_element(kps_ptr+3, kps_ptr + 3);

float score = *scores_ptr;
if(score > score_thres) 
	{
	label = maxSPtr - scores_ptr;

// Not Detect label
cout << func << ";label=" << label << endl;
.....

int cnt = 0;
/*
#ifdef BATCHED_NMS
cv::dnn::NMSBoxesBatched(bboxes, scores, labels, score_thresd, iou_thres, indices);
#else
cv::dnn::NMSBoxes(bboxes, scores, score_thres, iou_thres, indices);
#endif

for(auto& i : indices)
*/
// 8 object Detect, but almost same position and size.
for(int i = 0; i < bboxes.size(); i++)
{
if(cnt >= topk)
break;
Object obj;
obj.rect = bboxes[i];
obj.prob = scores[i];
obj.label = labels[i];
obj.kps = kpss[i];
objs.push_back(obj);
cnt += 1;
}
}
....

@triple-Mu
Copy link
Owner

The model learned with multi-class(3) is performed normally with the yolov8 cli Predict command. I confirmed through debugging that only one class objects was detected when using the example here.

*. you recommed. You should modify pose postprocess. Default is only one class so that the score is only for person. But if enable multiclass you should use argmax and max to find the highest score and label. => Where should I fix it?

*. I check it.

YOLOv8-TensorRT\csrc\pose\normal\include\yolov8-pose.hpp .... void YOLOv8_pose::postprocess(std::vector& objs, float score_thres, float iou_thres, int topk) { ... auto kps_ptr = row_ptr + 5;

// classnum = 3 auto maxSPtr = std::max_element(kps_ptr, kps_ptr + 3); // auto maxSPtr = std::max_element(kps_ptr+2, kps_ptr + 3); // auto maxSPtr = std::max_element(kps_ptr+3, kps_ptr + 3);

float score = *scores_ptr;
if(score > score_thres) 
	{
	label = maxSPtr - scores_ptr;

// Not Detect label cout << func << ";label=" << label << endl; .....

int cnt = 0; /* #ifdef BATCHED_NMS cv::dnn::NMSBoxesBatched(bboxes, scores, labels, score_thresd, iou_thres, indices); #else cv::dnn::NMSBoxes(bboxes, scores, score_thres, iou_thres, indices); #endif

for(auto& i : indices) */ // 8 object Detect, but almost same position and size. for(int i = 0; i < bboxes.size(); i++) { if(cnt >= topk) break; Object obj; obj.rect = bboxes[i]; obj.prob = scores[i]; obj.label = labels[i]; obj.kps = kpss[i]; objs.push_back(obj); cnt += 1; } } ....

Sure you should find the top label by scores_ptr.
And make the Object.label to this label.
And do postprocess and draw label on image.

@faithmaker
Copy link
Author

faithmaker commented Jan 13, 2024

Even if all objects were detected and output, the location could only be estimated for one class. The 17 location points are also wrong. Still, objects of other classes are not even detected. Objects are not detected properly before labeling.

*. YoloV8-TensorRt Program result(8 object Detect, but almost same position and size)
GetDetectBoxQue;Over1;Class=0;Center=1124,1218;Box=839,880;cv::Size=569,675
TrackYoloV8TrtPose;0;kps_s=198.936
TrackYoloV8TrtPose;1;kps_s=203.058
TrackYoloV8TrtPose;2;kps_s=195.693
TrackYoloV8TrtPose;3;kps_s=208.787
TrackYoloV8TrtPose;4;kps_s=189.63
TrackYoloV8TrtPose;5;kps_s=216.226
TrackYoloV8TrtPose;6;kps_s=177.447
TrackYoloV8TrtPose;7;kps_s=227.426
TrackYoloV8TrtPose;8;kps_s=157.746
TrackYoloV8TrtPose;9;kps_s=211.389
TrackYoloV8TrtPose;10;kps_s=174.692
TrackYoloV8TrtPose;11;kps_s=203.97
TrackYoloV8TrtPose;12;kps_s=173.19
TrackYoloV8TrtPose;13;kps_s=197.799
TrackYoloV8TrtPose;14;kps_s=165.32
TrackYoloV8TrtPose;15;kps_s=190.509
TrackYoloV8TrtPose;16;kps_s=156.924
GetDetectBoxQue;Over1;Class=0;Center=1124,1218;Box=840,879;cv::Size=569,679
TrackYoloV8TrtPose;0;kps_s=198.781
TrackYoloV8TrtPose;1;kps_s=203.272
TrackYoloV8TrtPose;2;kps_s=195.489
TrackYoloV8TrtPose;3;kps_s=209.107
TrackYoloV8TrtPose;4;kps_s=189.734
TrackYoloV8TrtPose;5;kps_s=215.908
TrackYoloV8TrtPose;6;kps_s=177.684
TrackYoloV8TrtPose;7;kps_s=229.06
TrackYoloV8TrtPose;8;kps_s=157.92
TrackYoloV8TrtPose;9;kps_s=211.557
TrackYoloV8TrtPose;10;kps_s=174.08
TrackYoloV8TrtPose;11;kps_s=203.287
TrackYoloV8TrtPose;12;kps_s=172.927
TrackYoloV8TrtPose;13;kps_s=197.695
TrackYoloV8TrtPose;14;kps_s=165.009
TrackYoloV8TrtPose;15;kps_s=189.77
TrackYoloV8TrtPose;16;kps_s=157.612
GetDetectBoxQue;Over1;Class=0;Center=1123,1218;Box=838,878;cv::Size=570,679
TrackYoloV8TrtPose;0;kps_s=198.684
TrackYoloV8TrtPose;1;kps_s=203.246
TrackYoloV8TrtPose;2;kps_s=195.461
TrackYoloV8TrtPose;3;kps_s=209.105
TrackYoloV8TrtPose;4;kps_s=189.693
TrackYoloV8TrtPose;5;kps_s=215.875
TrackYoloV8TrtPose;6;kps_s=177.634
TrackYoloV8TrtPose;7;kps_s=228.773
TrackYoloV8TrtPose;8;kps_s=157.929
TrackYoloV8TrtPose;9;kps_s=210.562
TrackYoloV8TrtPose;10;kps_s=173.788
TrackYoloV8TrtPose;11;kps_s=203.522
TrackYoloV8TrtPose;12;kps_s=173.2
TrackYoloV8TrtPose;13;kps_s=197.458
TrackYoloV8TrtPose;14;kps_s=165.086
TrackYoloV8TrtPose;15;kps_s=189.363
TrackYoloV8TrtPose;16;kps_s=157.905
GetDetectBoxQue;Over1;Class=0;Center=1122,1218;Box=836,879;cv::Size=571,678
TrackYoloV8TrtPose;0;kps_s=198.432
TrackYoloV8TrtPose;1;kps_s=202.424
TrackYoloV8TrtPose;2;kps_s=195.297
TrackYoloV8TrtPose;3;kps_s=208.145
TrackYoloV8TrtPose;4;kps_s=189.213
TrackYoloV8TrtPose;5;kps_s=215.082
TrackYoloV8TrtPose;6;kps_s=177.583
TrackYoloV8TrtPose;7;kps_s=226.372
TrackYoloV8TrtPose;8;kps_s=157.573
TrackYoloV8TrtPose;9;kps_s=211.008
TrackYoloV8TrtPose;10;kps_s=174.175
TrackYoloV8TrtPose;11;kps_s=203.245
TrackYoloV8TrtPose;12;kps_s=173.486
TrackYoloV8TrtPose;13;kps_s=197.998
TrackYoloV8TrtPose;14;kps_s=165.479
TrackYoloV8TrtPose;15;kps_s=190.475
TrackYoloV8TrtPose;16;kps_s=157.531
GetDetectBoxQue;Over1;Class=0;Center=1124,1218;Box=838,879;cv::Size=571,679
TrackYoloV8TrtPose;0;kps_s=198.532
TrackYoloV8TrtPose;1;kps_s=202.854
TrackYoloV8TrtPose;2;kps_s=195.302
TrackYoloV8TrtPose;3;kps_s=208.825
TrackYoloV8TrtPose;4;kps_s=189.529
TrackYoloV8TrtPose;5;kps_s=215.159
TrackYoloV8TrtPose;6;kps_s=177.943
TrackYoloV8TrtPose;7;kps_s=228.456
TrackYoloV8TrtPose;8;kps_s=157.458
TrackYoloV8TrtPose;9;kps_s=210.602
TrackYoloV8TrtPose;10;kps_s=173.761
TrackYoloV8TrtPose;11;kps_s=202.909
TrackYoloV8TrtPose;12;kps_s=173.135
TrackYoloV8TrtPose;13;kps_s=197.75
TrackYoloV8TrtPose;14;kps_s=165.267
TrackYoloV8TrtPose;15;kps_s=189.916
TrackYoloV8TrtPose;16;kps_s=157.957
GetDetectBoxQue;Over1;Class=0;Center=1123,1218;Box=837,880;cv::Size=572,677
TrackYoloV8TrtPose;0;kps_s=198.699
TrackYoloV8TrtPose;1;kps_s=203.128
TrackYoloV8TrtPose;2;kps_s=195.515
TrackYoloV8TrtPose;3;kps_s=209.187
TrackYoloV8TrtPose;4;kps_s=189.707
TrackYoloV8TrtPose;5;kps_s=215.665
TrackYoloV8TrtPose;6;kps_s=178.23
TrackYoloV8TrtPose;7;kps_s=228.964
TrackYoloV8TrtPose;8;kps_s=157.863
TrackYoloV8TrtPose;9;kps_s=210.612
TrackYoloV8TrtPose;10;kps_s=173.778
TrackYoloV8TrtPose;11;kps_s=203.795
TrackYoloV8TrtPose;12;kps_s=173.664
TrackYoloV8TrtPose;13;kps_s=198.012
TrackYoloV8TrtPose;14;kps_s=164.791
TrackYoloV8TrtPose;15;kps_s=189.592
TrackYoloV8TrtPose;16;kps_s=157.448
GetDetectBoxQue;Over1;Class=0;Center=1122,1218;Box=838,877;cv::Size=568,681
TrackYoloV8TrtPose;0;kps_s=198.826
TrackYoloV8TrtPose;1;kps_s=203.418
TrackYoloV8TrtPose;2;kps_s=195.816
TrackYoloV8TrtPose;3;kps_s=208.668
TrackYoloV8TrtPose;4;kps_s=190.348
TrackYoloV8TrtPose;5;kps_s=215.541
TrackYoloV8TrtPose;6;kps_s=177.889
TrackYoloV8TrtPose;7;kps_s=228.712
TrackYoloV8TrtPose;8;kps_s=157.739
TrackYoloV8TrtPose;9;kps_s=210.797
TrackYoloV8TrtPose;10;kps_s=174.608
TrackYoloV8TrtPose;11;kps_s=204.512
TrackYoloV8TrtPose;12;kps_s=172.62
TrackYoloV8TrtPose;13;kps_s=197.175
TrackYoloV8TrtPose;14;kps_s=165.003
TrackYoloV8TrtPose;15;kps_s=188.71
TrackYoloV8TrtPose;16;kps_s=156.816
GetDetectBoxQue;Over1;Class=0;Center=1122,1217;Box=836,877;cv::Size=572,680
TrackYoloV8TrtPose;0;kps_s=198.887
TrackYoloV8TrtPose;1;kps_s=203.393
TrackYoloV8TrtPose;2;kps_s=195.857
TrackYoloV8TrtPose;3;kps_s=208.635
TrackYoloV8TrtPose;4;kps_s=190.238
TrackYoloV8TrtPose;5;kps_s=215.461
TrackYoloV8TrtPose;6;kps_s=178.049
TrackYoloV8TrtPose;7;kps_s=228.574
TrackYoloV8TrtPose;8;kps_s=158.14
TrackYoloV8TrtPose;9;kps_s=210.797
TrackYoloV8TrtPose;10;kps_s=174.844
TrackYoloV8TrtPose;11;kps_s=204.366
TrackYoloV8TrtPose;12;kps_s=173.327
TrackYoloV8TrtPose;13;kps_s=197.403
TrackYoloV8TrtPose;14;kps_s=165.003
TrackYoloV8TrtPose;15;kps_s=189.178
TrackYoloV8TrtPose;16;kps_s=157.538
GetDetectBoxQue;Over1;Class=0;Center=1122,1217;Box=838,877;cv::Size=569,680
TrackYoloV8TrtPose;0;kps_s=198.956
TrackYoloV8TrtPose;1;kps_s=203.516
TrackYoloV8TrtPose;2;kps_s=195.814
TrackYoloV8TrtPose;3;kps_s=208.714
TrackYoloV8TrtPose;4;kps_s=190.404
TrackYoloV8TrtPose;5;kps_s=215.592
TrackYoloV8TrtPose;6;kps_s=178.2
TrackYoloV8TrtPose;7;kps_s=228.806
TrackYoloV8TrtPose;8;kps_s=157.672
TrackYoloV8TrtPose;9;kps_s=211.009
TrackYoloV8TrtPose;10;kps_s=174.543
TrackYoloV8TrtPose;11;kps_s=204.407
TrackYoloV8TrtPose;12;kps_s=172.916
TrackYoloV8TrtPose;13;kps_s=197.595
TrackYoloV8TrtPose;14;kps_s=165.03
TrackYoloV8TrtPose;15;kps_s=189.387
TrackYoloV8TrtPose;16;kps_s=157.959
TensorRtResultError

*. Ultralytics Cli result
UltralyticsResult640

@triple-Mu
Copy link
Owner

Do you have further questions?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants