Problem with tilted camera mount

[sandeepdutta]

Hello,
The camera on my robot is mounted pointed upwards . VSLAM seems to be having trouble
with this setup. It looses "Visual Tracking" frequently and creates a map (as visualized in RVIZ)
which looks like it is at an angle to the ground plane. The visual track has a vertical component as well.
I have set "force_planar" to true. Is there something I can tweak to make this work.
It works fine in the same environment when I straighten the camera to point in the direction of motion.

Thank you
Sandeep

[swapnesh-wani-nvidia]

Could you share the tf tree for the sensors and the base_link? Also, it would help what kind of environment are you working in. If the motion of the robot is not perfectly planar, force_planar flag won't help here.

[sandeepdutta]

Hello Swapnesh,

I have attached the full URDF file for the robot. Of interest are two joints
a) This is the top camera pointed in the same direction of X
"<!-- camera body, with origin at bottom screw mount -->
<joint name="rs_top_joint" type="fixed">
<origin rpy="0 0 0" xyz="0 0 0"/>
<parent link="Camera_Top_1"/>
<child link="rs_top_bottom_screw_frame"/>
</joint>"

b) The bottom camera pointed slightly upwards
"<!-- camera body, with origin at bottom screw mount -->
<joint name="rs_bottom_joint" type="fixed">
<origin rpy="0 -0.436332 0" xyz="0 0 0"/>
<parent link="Camera_Bottom_1"/>
<child link="rs_bottom_bottom_screw_frame"/>
</joint>"

The environment is simple planar area with some obstacles. When I use the top camera VSLAM works fine it
is able to map the area. When I use the bottom camera however it has difficulty in loop closure, it seems to think it
is moving in a plane which is tilted at an angle (the same angle as the camera) to the ground plane. The VIO reports
a movement in the Z direction . I am trying to recreate the experiment to get you some screen shots, it will take some time.
Let me know if you need any other information in the meantime .

Thank you
Sandeep
full_urdf.txt

[swapnesh-wani-nvidia]

Thanks for the update. Screnshots will be helpful in oerder to diagnose. Could you also share the laucnh files you are using?

[sandeepdutta]

Hello Swapnesh,

Sorry it took longer than I thought it would. I have run some experiments, and captured some screen shots, note all experiments in the same environment and same starting pose.
a) Camera mounted pointing in direction of travel force_planar_mode = false , all works ok.

b) Camera mounted with 10 deg upward tilt (this is done to get more visual artifacts) force_planar_mode = false. in the screen shot observe the odometry moving upwards in the same angle as the camera. Once we get some loop closure,
the trajectory is corrected the odometry moves straight for a little while , if no loop closure for a while the odometry will start drifting upwards.

c) Camera mounted with 10 deg upward tilt force_palar_mode = true. In this case the odometry consistantly moves in the same plane, the plane is tilted by the same amount as the camera tilt.

With force_planar_mode=true, it seems to be consistent albeit working on a tilted plane .
I have tried using the "odometry_flattening" node from the nvblox packages, it does not seem to help.
Would be great to get a solution or work around to this problem.
Thank you
Sandeep

[sandeepdutta]

Hello Swapnesh,

I have some further updates to this issue. I ran two more experiments. I removed the camera tilt from the URDF, i.e.
the camera mount does not include the tilt, the transform thinks that the camera is mounted in the direction of travel.
a) With force_planar_mode=True , the VO generated is correct, it shows the odometry pointing upwards and the
trajectory is correct (no Z travel). It looks like the VO is able to infer rhat the camera is mounted at an angle .

b) With force_planar_mode=False, however the odometry travels upwards at the same angle as the camera mount,
so it seems the VO thinks that the camera is moving upwards, same result as tilt in URDF & force_planar_mode =True

Best Regards
Sandeep

[sandeepdutta]

Additionally , "Loadandlocalize" never worked with databases created with tilted camera.
Sandeep

[hermanndererdmann]

I'm having the same issues. did you find a solution?