posenet resnet18-body

[Femi1984]

Hi, Thank you for your contribution

I built a node in ROS a couple of years ago with your posenet. It was working then. When I run the node now it compiles to a certain point then hangs there.

femi@femi-desktop:~/catkin_ws$ rosrun pan_and_tilt_control detector_pose.py
jetson.inference -- poseNet loading build-in network 'resnet18-body'

poseNet -- loading pose estimation model from:
-- model networks/Pose-ResNet18-Body/pose_resnet18_body.onnx
-- topology networks/Pose-ResNet18-Body/human_pose.json
-- colors networks/Pose-ResNet18-Body/colors.txt
-- input_blob 'input'
-- output_cmap 'cmap'
-- output_paf 'paf'
-- threshold 0.150000
-- batch_size 1

[TRT] topology -- keypoint 0 nose
[TRT] topology -- keypoint 1 left_eye
[TRT] topology -- keypoint 2 right_eye
[TRT] topology -- keypoint 3 left_ear
[TRT] topology -- keypoint 4 right_ear
[TRT] topology -- keypoint 5 left_shoulder
[TRT] topology -- keypoint 6 right_shoulder
[TRT] topology -- keypoint 7 left_elbow
[TRT] topology -- keypoint 8 right_elbow
[TRT] topology -- keypoint 9 left_wrist
[TRT] topology -- keypoint 10 right_wrist
[TRT] topology -- keypoint 11 left_hip
[TRT] topology -- keypoint 12 right_hip
[TRT] topology -- keypoint 13 left_knee
[TRT] topology -- keypoint 14 right_knee
[TRT] topology -- keypoint 15 left_ankle
[TRT] topology -- keypoint 16 right_ankle
[TRT] topology -- keypoint 17 neck
[TRT] topology -- skeleton link 0 16 14
[TRT] topology -- skeleton link 1 14 12
[TRT] topology -- skeleton link 2 17 15
[TRT] topology -- skeleton link 3 15 13
[TRT] topology -- skeleton link 4 12 13
[TRT] topology -- skeleton link 5 6 8
[TRT] topology -- skeleton link 6 7 9
[TRT] topology -- skeleton link 7 8 10
[TRT] topology -- skeleton link 8 9 11
[TRT] topology -- skeleton link 9 2 3
[TRT] topology -- skeleton link 10 1 2
[TRT] topology -- skeleton link 11 1 3
[TRT] topology -- skeleton link 12 2 4
[TRT] topology -- skeleton link 13 3 5
[TRT] topology -- skeleton link 14 4 6
[TRT] topology -- skeleton link 15 5 7
[TRT] topology -- skeleton link 16 18 1
[TRT] topology -- skeleton link 17 18 6
[TRT] topology -- skeleton link 18 18 7
[TRT] topology -- skeleton link 19 18 12
[TRT] topology -- skeleton link 20 18 13
[TRT] poseNet -- keypoint 00 'nose' color 255 0 85 255
[TRT] poseNet -- keypoint 01 'left_eye' color 255 0 0 255
[TRT] poseNet -- keypoint 02 'right_eye' color 255 85 0 255
[TRT] poseNet -- keypoint 03 'left_ear' color 255 170 0 255
[TRT] poseNet -- keypoint 04 'right_ear' color 255 255 0 255
[TRT] poseNet -- keypoint 05 'left_shoulder' color 170 255 0 255
[TRT] poseNet -- keypoint 06 'right_shoulder' color 85 255 0 255
[TRT] poseNet -- keypoint 07 'left_elbow' color 0 255 0 255
[TRT] poseNet -- keypoint 08 'right_elbow' color 0 255 85 255
[TRT] poseNet -- keypoint 09 'left_wrist' color 0 255 170 255
[TRT] poseNet -- keypoint 10 'right_wrist' color 0 255 255 255
[TRT] poseNet -- keypoint 11 'left_hip' color 0 170 255 255
[TRT] poseNet -- keypoint 12 'right_hip' color 0 85 255 255
[TRT] poseNet -- keypoint 13 'left_knee' color 0 0 255 255
[TRT] poseNet -- keypoint 14 'right_knee' color 255 0 170 255
[TRT] poseNet -- keypoint 15 'left_ankle' color 170 0 255 255
[TRT] poseNet -- keypoint 16 'right_ankle' color 255 0 255 255
[TRT] poseNet -- keypoint 17 'neck' color 85 0 255 255
[TRT] poseNet -- loaded 18 class colors
[TRT] TensorRT version 7.1.3
[TRT] loading NVIDIA plugins...
[TRT] Registered plugin creator - ::GridAnchor_TRT version 1
[TRT] Registered plugin creator - ::NMS_TRT version 1
[TRT] Registered plugin creator - ::Reorg_TRT version 1
[TRT] Registered plugin creator - ::Region_TRT version 1
[TRT] Registered plugin creator - ::Clip_TRT version 1
[TRT] Registered plugin creator - ::LReLU_TRT version 1
[TRT] Registered plugin creator - ::PriorBox_TRT version 1
[TRT] Registered plugin creator - ::Normalize_TRT version 1
[TRT] Registered plugin creator - ::RPROI_TRT version 1
[TRT] Registered plugin creator - ::BatchedNMS_TRT version 1
[TRT] Could not register plugin creator - ::FlattenConcat_TRT version 1
[TRT] Registered plugin creator - ::CropAndResize version 1
[TRT] Registered plugin creator - ::DetectionLayer_TRT version 1
[TRT] Registered plugin creator - ::Proposal version 1
[TRT] Registered plugin creator - ::ProposalLayer_TRT version 1
[TRT] Registered plugin creator - ::PyramidROIAlign_TRT version 1
[TRT] Registered plugin creator - ::ResizeNearest_TRT version 1
[TRT] Registered plugin creator - ::Split version 1
[TRT] Registered plugin creator - ::SpecialSlice_TRT version 1
[TRT] Registered plugin creator - ::InstanceNormalization_TRT version 1
[TRT] detected model format - ONNX (extension '.onnx')
[TRT] desired precision specified for GPU: FASTEST
[TRT] requested fasted precision for device GPU without providing valid calibrator, disabling INT8
[TRT] native precisions detected for GPU: FP32, FP16, INT8
[TRT] selecting fastest native precision for GPU: FP16
[TRT] attempting to open engine cache file /usr/local/bin/networks/Pose-ResNet18-Body/pose_resnet18_body.onnx.1.1.7103.GPU.FP16.engine
[TRT] loading network plan from engine cache... /usr/local/bin/networks/Pose-ResNet18-Body/pose_resnet18_body.onnx.1.1.7103.GPU.FP16.engine
[TRT] device GPU, loaded /usr/local/bin/networks/Pose-ResNet18-Body/pose_resnet18_body.onnx
[TRT] Using an engine plan file across different models of devices is not recommended and is likely to affect performance or even cause errors.
[TRT] Deserialize required 2468298 microseconds.
[TRT]
[TRT] CUDA engine context initialized on device GPU:
[TRT] -- layers 46
[TRT] -- maxBatchSize 1
[TRT] -- workspace 0
[TRT] -- deviceMemory 33779712
[TRT] -- bindings 3
[TRT] binding 0
-- index 0
-- name 'input'
-- type FP32
-- in/out INPUT
-- # dims 4
-- dim #0 1 (SPATIAL)
-- dim #1 3 (SPATIAL)
-- dim #2 224 (SPATIAL)
-- dim #3 224 (SPATIAL)
[TRT] binding 1
-- index 1
-- name 'cmap'
-- type FP32
-- in/out OUTPUT
-- # dims 4
-- dim #0 1 (SPATIAL)
-- dim #1 18 (SPATIAL)
-- dim #2 56 (SPATIAL)
-- dim #3 56 (SPATIAL)
[TRT] binding 2
-- index 2
-- name 'paf'
-- type FP32
-- in/out OUTPUT
-- # dims 4
-- dim #0 1 (SPATIAL)
-- dim #1 42 (SPATIAL)
-- dim #2 56 (SPATIAL)
-- dim #3 56 (SPATIAL)
[TRT]
[TRT] binding to input 0 input binding index: 0
[TRT] binding to input 0 input dims (b=1 c=3 h=224 w=224) size=602112
[TRT] binding to output 0 cmap binding index: 1
[TRT] binding to output 0 cmap dims (b=1 c=18 h=56 w=56) size=225792
[TRT] binding to output 1 paf binding index: 2
[TRT] binding to output 1 paf dims (b=1 c=42 h=56 w=56) size=526848
[TRT]
[TRT] device GPU, /usr/local/bin/networks/Pose-ResNet18-Body/pose_resnet18_body.onnx initialized.

[dusty-nv]

Hi @Femi1984, are you sure it isn't loaded and just waiting for input from your ROS node? I would try adding more print statements to it and find out where it is hanging.

[dusty-nv]

You could also try mv /usr/local/bin/networks/Pose-ResNet18-Body/pose_resnet18_body.onnx.1.1.7103.GPU.FP16.engine /usr/local/bin/networks/Pose-ResNet18-Body/pose_resnet18_body.onnx.1.1.7103.GPU.FP16.engine.backup and re-running it (which will re-build the TensorRT engine, in case you have changed devices or jetpack versions)

[Femi1984]

Thanks for your response. I tried what you mentioned and it stopped in the same spot. I presume something must have changed since the code I wrote a while ago. I'm using CVBridge to convert the image to Jetson cuda format. Does that still work? I'm using D435 camera

from numpy.core.fromnumeric import shape
import rospy
import cv2
import jetson.inference
import jetson.utils
import message_filters
import numpy as np
from cv_bridge import CvBridge
from geometry_msgs.msg import TransformStamped
from sensor_msgs.msg import Image
from geometry_msgs.msg import PoseStamped
from tf2_msgs.msg import TFMessage
from std_msgs.msg import Bool, Int16
from cv_bridge import CvBridge, CvBridgeError
from pan_and_tilt_control.msg import PanAndTilt

class Node(object):
def init(self):
self.detector = jetson.inference.poseNet("resnet18-body", threshold=0.35)
self.cv_bridge = CvBridge()
self.depth_sub = message_filters.Subscriber('/camera/depth/image_rect_raw', Image)
#self.depth_sub = message_filters.Subscriber('/camera/aligned_depth_to_color/image_raw', Image)
self.rgba_sub = message_filters.Subscriber('/camera/color/image_raw', Image)

def on_new_img(self, depth_img, rgba_img):
biggest = 0
#counter_res = 0

    depth_frame = self.cv_bridge.imgmsg_to_cv2(depth_img)
    rgba_frame = self.cv_bridge.imgmsg_to_cv2(rgba_img)
    # Convert to jetson cuda format
    cuda_frame = jetson.utils.cudaFromNumpy(rgba_frame)
    poses = self.detector.Process(cuda_frame, overlay="links,keypoints")
    bgr_img = jetson.utils.cudaAllocMapped(width=rgba_img.width,height=rgba_img.height,format='bgr8')
    jetson.utils.cudaConvertColor(cuda_frame, bgr_img)
    jetson.utils.cudaDeviceSynchronize()
    cv_img = jetson.utils.cudaToNumpy(bgr_img)

if self.show_camera:
cv2.imshow("Result", cv_img)
cv2.waitKey(3)

if name == "main":
rospy.init_node('test')
n = Node()
rospy.spin()

[dusty-nv]

Are you still able to run the original posenet/posenet.py examples ok?

[Femi1984]

Are you still able to run the original posenet/posenet.py examples ok?

Yes

[Femi1984]

Could you point me in the direction where an example of posenet is used with ros?