Share / Export Citation / Email / Print / Text size:

International Journal on Smart Sensing and Intelligent Systems

Professor Subhas Chandra Mukhopadhyay

Exeley Inc. (New York)

Subject: Computational Science & Engineering, Engineering, Electrical & Electronic


eISSN: 1178-5608



VOLUME 9 , ISSUE 2 (June 2016) > List of articles


Danial Nakhaeinia * / Pierre Payeur * / Alberto Chávez-Aragón * / Ana-Maria Cretu * / Robert Laganière * / Rizwan Macknojia *

Keywords : Security robotics, sensor based navigation, RGB-D imaging, curvature estimation, path planning, surface following.

Citation Information : International Journal on Smart Sensing and Intelligent Systems. Volume 9, Issue 2, Pages 419-447, DOI:

License : (CC BY-NC-ND 4.0)

Received Date : 01-February-2016 / Accepted: 12-April-2016 / Published Online: 01-June-2016



This paper presents the design and integration of a vision-guided robotic system for automated and rapid vehicle inspection. The main goal of this work is to scan and explore regions of interest over an automotive vehicle while a manipulator’s end effector operates in close proximity of the vehicle and safely accommodates its curves and inherent surface obstacles, such as outside mirrors or door handles, in order to perform a series of close inspection tasks. The project is motivated by applications in automated vehicle inspection, cleaning, and security screening. In order to efficiently navigate the robotic manipulator along the vehicle’s surface within regions of interest that are selectively identified, an efficient and accurate integration of information from multiple RGB-D sensors and robotic components is proposed. The main components of the proposed approach include: automated vehicle category recognition from visual information; RGB-D sensors calibration; extraction of specific areas to inspect over the vehicle body, and path planning from an efficiently reconstructed 3D surface mesh to move the robotic arm along and in close proximity of the vehicle. The proposed multi-stage system developed merges all components to achieve rapid 3D profiling over a complex surface in order to fully automate the process of surface following for vehicles of various types and shapes. To validate the feasibility and effectiveness of the proposed method experiments are carried out with a 7-DOF manipulator navigating over automotive body panels.

Content not available PDF Share



[1] Y.H. Anis, M.R. Holl and D.R. Meldrum, “Automated Selection and Placement of Single Cells Using Vision-Based Feedback Control”, IEEE Transactions on Automation Science and Engineering, vol. 7, no. 3, July 2010, pp. 598-606.
[2] R. Araujo, U. Nunes and A. T. de Almeida, "Robot 3D Force-Based Surface-Tracking," 20th International Conference on Industrial Electronics, Control and Instrumentation, vol. 2, 1994, Bologna, pp. 788-793.
[3] Y. Yin, H. Hu and Y. Xia, “Active Tracking of Unknown Surface Using Force Sensing and Control Technique for Robot”, Sensors and Actuators, vol.112, 2004, pp. 313–319.
[4] X. Papageorgiou, H. G. Tanner and K. J. Kyriakopoulos, "Motion Tasks for Robot Manipulators on Embedded 2-D Manifolds Under Input Constraints," Control Conference (ECC), 2007 European, Kos, 2007, pp. 3783-3789.
[5] X. Papageorgiou, S. G. Loizou and K. J. Kyriakopoulos, "Motion Tasks and Force Control for Robot Manipulators on Embedded 2-D Manifolds," IEEE International Conference on Robotics and Automation, Roma, 2007, pp. 4202-4207.
[6] R.S. Dahiya, G. Metta, G. Cannata and M. Valle, “Guest Editorial: Special Issue on Robotic Sense of Touch,” IEEE Transactions on Robotics, vol. 27, no. 3, 2011, pp. 385–388.
[7] R. Ibrayev and Y.B. Jia, “Recognition of Curved Surfaces from “One-dimensional” Tactile Data,” IEEE Transactions on Automation Science and Engineering, vol. 9, no. 3, 2012, pp. 613-621.
[8] J.L. Pedreño-Molina, A. Guerrero-González, J. Calabozo-Moran, J. López-Coronado and P. Gorce, “A Neural Tactile Architecture Applied to Real-Time Stiffness Estimation for a Large Scale of Robotic Grasping Systems,” Journal of Intelligent and Robotic Systems, vol. 49, 2007, pp. 311-323.
[9] Y. Ito, Y. Kim, C. Nagai and G. Obinata,"Vision-Based Tactile Sensing and Shape Estimation Using a Fluid-Type Touchpad," IEEE Transactions on Automation Science and Engineering, vol. 9, no. 4, Oct. 2012, pp. 734-744.
[10] J. Wang and Y. Li, “Surface-Tracking of a 5-DOF Manipulator Equipped with Tactile Sensors,” 11th International Conference on Control Automation Robotics & Vision, 2010, Singapore, pp. 2448- 2453.
[11] J. Wang and Y. Li, “Tracking Control of a Redundant Manipulator with the Assistance of Tactile Sensing,” Intelligent Automation and Soft Computing, vol. 17, no. 7, 2011, pp. 833-845.
[12] D. Nakhaeinia, P. Payeur and R. Laganière, "Adaptive Robotic Contour Following from Low Accuracy RGB-D Surface Profiling and Visual Servoing," Conference on Computer and Robot Vision (CRV), Montreal, 2014, pp. 48-55.
[13] A. Pichler and M. Jagersand, "Uncalibrated Hybrid Force-Vision Manipulation," IEEE/RSJ International Conference on Intelligent Robots and Systems, (IROS 2000), vol. 3, Takamatsu, 2000, pp. 1866-1871.
[14] J. Baeten, H. Bruyninckx and J. D. Schutter, “Integrated Vision/Force Robotic Servoing in the Task Frame Formalism,” The International Journal of Robotics Research, vol. 22, 2003, pp. 941–954.
[15] E. Li, Z. Li and J. He, “Robotic Adaptive Impedance Control Based on Visual Guidance,” Int. Journal Smart Sensing and Intelligent Systems, vol. 8, no 4, December 2015, pp. 2159-2174.
[16] C.C. Cheah, S.P. Hou, Y. Zhao and J.J.E. Slotine, “Adaptive Vision and Force Tracking Control for Robots with Constraint Uncertainty”. IEEE/ASME Transactions on Mechatronics, vol. 15, no. 3, 2010, pp. 389–399.
[17] D. Nakhaeinia, P. Laferrière, P. Payeur, R. Laganière, “Safe Close-Proximity and Physical Human-Robot Interaction Using Industrial Robots,” Conference on Computer and Robot Vision (CRV), Halifax, 2015, pp. 237-244.
[18] R. Fareh, P. Payeur, D. Nakhaeinia, R. Macknojia, A. Chávez-Aragón, A.M. Cretu, P. Laferrière, R. Laganière and R. Toledo, “An Integrated Vision-Guided Robotic System for Rapid Vehicle Inspection,” IEEE International Systems Conference, Ottawa, 2014, pp. 446 - 451.
[19] C. Pillajo and J.E. Sierra, “Human Machine Interface HMI Using Kinect Sensor toC a SCARA Robot,” IEEE Colombian Conference on Communications and Computing, Medellin, 2013, pp. 1-5.
[20] A. Teichman, J.T. Lussier and S. Thrun, “Learning to Segment and Track in RGBD,” IEEE Transactions on Automation Science and Engineering, vol. 10, no. 4, 2013, pp. 841-852.
[21] F. Nagata, Y. Kusumoto, K. Watanabe, K. Kiguchi, K. Tsuda, K. Yasuda, K. Yokoyama, M. Umetsu, N. Mori and M. Omoto, "High Precision Polishing Robot Using a Learning-Based Surface Following Controller," IEEE International Symposium on Computational Intelligence in Robotics and Automation, vol.1, 2003, pp. 91-96.
[22] J. J. Park, J.H. Kim and J.B. Song, “Path Planning for a Robot Manipulator Based on Probabilistic Roadmap and Reinforcement Learning,” International Journal of Control, Automation, and Systems, vol. 5, no. 6, 2007, pp. 674-680.
[23] B. Hamner, S. Singh, S. Scherer, “Learning Obstacle Avoidance Parameters from Operator Behavior,” Journal of Field Robotics, Special Issue on Machine Learning Based Robotics in Unstructured Environments, vol. 23, no. 11-12, 2006, pp. 1037–1058.
[24] D. Nakhaeinia, R. Fareh, P. Payeur and R. Laganière, “Trajectory Planning for Surface Following with a Manipulator Under RGB-D Visual Guidance. IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR), Linkoping, Sweden, 2013, pp. 1-6.
[25] R. Macknojia, A. Chávez-Aragón, P. Payeur and R. Laganière, “Calibration of a Network of Kinect Sensors for Robotic Inspection Over a Large Workspace,” IEEE Workshop on Robot Vision, Clearwater, FL, 2013, pp. 184-190.
[26] A.-M. Cretu, and P. Payeur, "Biologically-Inspired Visual Attention Features for a Vehicle Classification Task", Int. Journal Smart Sensing and Intelligent Systems, vol. 4, no. 3, Sep. 2011, pp. 402-423.
[27] L. Shui, “Adaptive Image Segmentation Based on Saliency Detection,” Int. Journal Smart Sensing and Intelligent Systems, vol. 8, no. 1, March 2015, pp. 408-428.
[28] L. Itti, C. Koch and E. Niebur, “A Model of Saliency-Based Visual Attention for Rapid Scene Analysis,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.20, no.11, 1998, pp. 1254–1259.
[29] A. Chávez-Aragón, R. Macknojia, P. Payeur and R. Laganière, “Rapid 3D Modeling and Parts Recognition on Automotive Vehicles Using a Network of RGB-D Sensors for Robot Guidance” Journal of Sensors, Hindawi (ed), 2013, 16 pages.
[30] P. Lindstrom, “Out-of-Core Simplification of Large Polygonal Models,” 27th Annual Conference on Computer Graphics and Interactive Techniques, New York, 2000, pp. 259-262.