CoordinatesGetter.py is for object detection. Through this file, you can perform object detection and get the x,y,z coordinates of the object for further manipulation
move_test.py is the file you can use to collect instruction from users. Once the user input his/her instruction, the system will automatically capture the current image and send it to the VLM with the instruction to perform affordance learning and reasoning. Once it has the results from the VLM, it will pass these results with user instruction to generate sequences of actions.
noactionlib.py is the action I designed for pre-setup
back_to_home.py is the action I designed to return things to one fixed place.
msg is the new data structure I designed for Pose