I am currently a PhD student in the RAIDS Research Group at the Department of Industrial and Systems Engineering of The Hong Kong Polytechnic University (PolyU), mainly engaged in the research of robot learning, human-robot collaboration and robot teleoperation.
I earned my bachelor’s degree at School of Mechanical Engineering, Hebei University of Technology (河北工业大学机械工程学院). I then pursued a master’s degree at the National Key Laboratory of Robot Technology and Systems, Harbin Institute of Technology (哈尔滨工业大学机器人技术与系统国家重点实验室). Now, I’m pursuing a doctoral degree at the RAIDS Research Group in The Hong Kong Polytechnic University, under the guidance of Prof. Pai Zheng.
My research interest includes robot learning, human-robot collaboration. I have submitted 5 papers in SCI journals, applied 1 invention patent, authorized 3 utility model patents and 1 software copyright.
📖 Educations
- 2024.09 - now, The Hong Kong Polytechnic University, Hong Kong.
- 2022.09 - 2024.06, Harbin Institute of Technology, Harbin, (Score: 90.65, rank: 13/177).
- 2018.09 - 2022.06, Hebei University of Technology, Tianjin, (GPA: 3.95/4, rank: 1/133).
🔍 Projects
Sampling Robots in Complex Environments
Because of the irregularities on the object’s surface and variations in lighting conditions, point cloud images obtained from a single perspective often contain significant gaps and errors at the object’s edges, which can lead to inaccurate grasping pose estimations. To tackle these challenges, this article proposes a practical robot grasping method based on 6D pose estimation and point cloud fusion. First, 6D pose estimation is conducted, utilizing the results from model point cloud and pose estimation to complete the input point cloud through ICP (Iterative Closest Point). Subsequently, the resulting complete object point cloud is used to estimate the 6D grasping pose of the object with the help of the grasping direction estimation network.
Finished Works:
-
Establish a model-free 6D pose estimation network based on perspective matching. Construct and train the pose estimation network to achieve precise pose estimation for seen objects and a rough estimation method for untrained objects. Create a pose refinement network for untrained objects when sparse point clouds are accessible.
-
Propose a point cloud fusion and filtering method based on pose estimation. This method addresses the issue of point cloud gaps at the edge of objects when the robot collects samples from a single perspective. The point cloud fusion method is designed to utilize the output of pose estimation to fill in the gaps in the input point cloud and remove noise.
-
Perform grasp pose estimation using the completed point cloud. Utilizing the grasping angle prediction network and a fast search strategy, the completed point cloud enhances the stability of grasping pose generation, thereby improving the success rate of grasping and its robustness.
-
Test the system in simulation and the real world using a 6DoF robot, a Realsense camera, and a laptop (ROS/Ubuntu 20.04) as the host computer.
Autonomous Suture Robot System for Endoscopic Surgery
The research objective is to enhance the efficiency and accuracy of autonomous suturing in endoscopic surgery. This involves designing a 3DOF autonomous suturing instrument, developing an efficient laparoscopic surgery robot based on 3D laparoscopy, and establishing a robot system that integrates visual, expert, and navigation systems.
Finished Works:
-
Perform the recognition and segmentation of surgical instruments and lesion tissues based on Yolov8, and determine the spatial position of the end of the surgical instruments.
-
Build a vision and robotic arm fusion system, perform hand-eye calibration of cameras and robotic arms, and execute basic robot actions for visual servo.
Rock Core Box Handling Robot
Rock Core samples obtained from drilling before oil extraction are crucial data for assessing mining value, and they are stored in dedicated core boxes. During research and analysis, it’s essential to arrange these rock core boxes neatly on-site in a designated order, one box at a time. However, these boxes are typically heavy, resulting in a high labor intensity that can impact transportation efficiency. The present invention aims to address the labor intensity associated with the current method of transporting core boxes, ultimately improving transportation efficiency.
Finished Works:
-
Build a sensing system that utilizes infrared sensors to detect surrounding obstacles, as well as fractional laser sensors to detect the current stacking height and alignment.
-
Implement rock core box instance segmentation based on Mask RCNN, combined with a depth camera to determine its corner space coordinates for visual servo during the robot handling process.
-
Utilize a monocular camera and Aruco markers to achieve 2D pose adjustment of the robot in place by identifying the offset distance and angle of the Aruco marker.
Medical Pan-Tilt Control System Based on Binocular Vision
In traditional medical processes, traditional Chinese medicine practitioners often need to frequently change positions and adjust their angles to view the surgical area. Additionally, during rehabilitation treatment, the camera’s range can be inadequate to cover the patient’s moving area. Current recording methods commonly used suffer from limitations and lack automation. This article introduces a multi-degree-of-freedom pan-tilt system designed to track the positions of doctors and patients.
Finished Works:
-
Design the mechanical structure and simulate the platform’s motion, using internal toothed rotary bearings to minimize the structural size. Protective structures have been designed for all sensor components.
-
Design hardware control algorithms for pan-tilt using STM32 and A4988 drivers, and establish communication between STM32 and the host computer’s ROS system for pan-tilt motion control within ROS.
-
Propose a target tracking method based on HOG and SIFT feature matching, capable of short-term single target tracking while maintaining robustness to changes in the target object’s appearance.
-
Construct an experimental Pan-Tilt system and conduct experiments with all proposed algorithms, using 3D printed structural components, a Realsense camera, and stepper motors.
Spherical Environmental Information Collection Robot
Monitoring essential information in various hazardous environments, such as petrochemical plants, chemical plants, and disaster sites, is vital for disaster prevention and reduction. In response to this need, our team has designed a spherical environment monitoring robot system. It features a spherical metamorphic structure with multiple telescopic feet that enable the robot to rotate, roll, and achieve all-round motion with high stability. These robots utilize self-organized network communication technology to offer multiple monitoring modes, and the collected data is packaged and transmitted to the upper computer system, enabling real-time dynamic monitoring of complex environmental information over a wide area.
Finished Works:
- Design the three-dimensional structure of the spherical robot, including the placement of controllers, sensors inside the sphere, and the spatial arrangement of leg motors.
- Implement control of spherical robot leg motors using STM32, calculate motion strategies, and achieve robot motion.
- Integrate , accelerometers, temperature and humidity sensors, gas sensors, etc., into the STM32 control system for Bluetooth-controlled robot motion and sensor data reception.
📝 Publications
📃 Papers
Robotic Grasping Method with 6D Pose Estimation and Point Cloud Fusion
Haofei Ma, Gongcheng Wang, Hua Bai, Zhiyu Xia, Weidong Wang, Zhijiang Du
- A grasping pose estimation framework based on point cloud fusion and filtering is proposed, which solves the problem of sparse point clouds at object edges and facilitates more robust grasping.
- A novel pose estimation method based on viewpoint selection is introduced, which first uses an RGBD cam era to reconstruct the point cloud model of the object, and then applies the principle of viewpoint selection to obtain the 6D pose of the object.
- We demonstrate that through pose estimation and point cloud fusion, this grasping framework can accurately grasp object from a single-view RGBD image, maintaining a high success rate even in cluttered scenes.
-
IJAMT
Haofei Ma, Gongcheng Wang, Hua Bai, Zhiyu Xia, Weidong Wang, and Zhijiang Du. “Robotic Grasping Method with 6D Pose Estimation and Point Cloud Fusion.” The International Journal of Advanced Manufacturing Technology (2024): 1-11. doi: 10.1007/s00170-024-14372-3 -
RAS
Gongcheng Wang, Haofei Ma, Han Wang, Pengchao Ding, Hua Bai, Wenda Xu, Weidong Wang, and Zhijiang Du. “Reactive mobile manipulation based on dynamic dual-trajectory tracking.” Robotics and Autonomous Systems 172 (2024): 104589. doi: 10.1016/j.robot.2023.104589. -
IEEE Sensors Journal
Zhiyu Xia, Han Wang, Yulong Men, Haofei Ma, Zexin Cao, Weidong Wang, Zhijiang Du. “Kalman Filter-based EM-optical Sensor Fusion for Bone Needle Position Tracking.” IEEE Sensors Journal (2024). doi: 10.1109/JSEN.2024.3364701 -
RAS
Hua Bai, Wenrui Gao, Haofei Ma, Pengchao Ding, Gongcheng Wang, Wenda Xu, Weidong Wang, Zhijiang Du. “A study of robotic search strategy for multi-radiation sources in unknown environments.” Robotics and Autonomous Systems 169 (2023): 104529. doi: 10.1109/JSEN.2024.3364701.
📚 Patents
-
Invention Patent
A Rock Core Box Handling Robot, Weidong Wang, Hengbin Liang, Haofei Ma, Gongcheng Wang (CN202310547284.5, Pending) -
Utility Model Patent
A Spherical Metamorphic Robot and An Environmental Information Monitoring System, Yuhan Rao, Manhong Li, Haofei Ma, Yuchong Gao, Nuo Zhang, Xinyu Liu (CN202120212154.2) -
Utility Model Patent
A Rope Driven Cleaning Robot, Bao Li, Manhong Li, Shuofan Li, Haofei Ma, Jidong Guo, Yuchong Gao, Yingxin Dong (CN202120545507.0) -
Utility Model Patent
Small Ocean Vehicles Using Wave Energy to Generate Electricity, Yihan Gao, Haofei Ma, Shaoan Chen, Haoran Sun, Chenxi Song (CN202020078465.X) -
Software Copyright
Identity Recognition and Infrared Temperature Measurement Control System, Haofei Ma (2021SR1391064)
🏆 Honors and Awards
🏅 Honors
- 2024.06, HeGao Scholarship in Harbin Institute of Technology
- 2023.12, Leading Intelligence · Wang Yanqing Scholarship
- 2023.10, Excellent Students of Harbin Institute of Technology
- 2023.09, Top Grade Scholarship in Harbin Institute of Technology
- 2022.06, Provincial Outstanding Graduates (Top 1%)
- 2021.05, Provincial Merit Student (Top 1%)
- 2021.06, Finely Crafted Technology Scholarship
- 2021.01, Outstanding Student in Hebei University of Technology
- 2020.12, National Scholarship (Top 0.3%)
- 2020.12, Excellent Student Cadre in Hebei University of Technology
- 2020.12, The First Prize Scholarship in Hebei University of Technology
🎏 Competitions
- 2021.09, “Internet +” Innovation and Entrepreneurship Competition Provincial Silver Award .
- 2021.08, E-commerce “Innovation, Creativity, and Entrepreneurship” Challenge Provincial Third Prize.
- 2021.07, Zhou Peiyuan Mechanics Competition Provincial Second Prize and National Excellence Award.
- 2020.12, Hebei Province College Robot Competition Seconda Prize.
- 2020.10, Electronic Design Competition Provincial Second Prize.
- 2020.10, iCAN International Innovation and Entrepreneurship Competition Provincial Second Prize.
- 2020.09, Mechanical Innovation Design Competition Provincial Second Prize.
- 2019.12, Mathematics Competition Provincial First Prize.
- 2019.09, iCAN International Innovation and Entrepreneurship Competition Provincial Third Prize.
- 2019.06, Mathematics Competition Provincial Second Prize.
💼 Societies
- 2023.02 - 2023.07, School Office Assistant in Harbin Institute of Technology.
- 2022.02 - 2022.06, Class Leader in Hebei University of Technology.
- 2019.09 - 2022.06, Class Study Monitor in Hebei University of Technology.
- 2020.09 - 2021.08, Director of Haier Key Maker-Lab in Hebei University of Technology.