Volume 16, Issue 4 (Journal of Control, V.16, N.4 Winter 2023)
JoC 2023, 16(4): 57-73 |
Back to browse issues page
Download citation:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
Ranjkesh N, Shojaei K. Look-ahead control of a car-like mobile robot via reinforcement learning. JoC 2023; 16 (4) :57-73
URL: http://joc.kntu.ac.ir/article-1-952-en.html
URL: http://joc.kntu.ac.ir/article-1-952-en.html
1- Najafabad Branch, Islamic Azad University
Abstract: (1986 Views)
In this paper, the performance improvement problem of a reference trajectory tracking for a car-like mobile robot with nonholonomic constraints in the presence of external disturbances, nonlinearities and uncertain parameters is investigated. For this purpose, at first the dynamic and kinematic equations of the car-like mobile robot are expressed and then the look-ahead control method in two dimensional is used for the tracking of the car-like mobile robot. The purposed controller will be designed by using the dynamic surface control method with the reinforcement learning based on the actor-critic neural network and an adaptive robust controller is proposed to compensate the effects of external disturbances. Moreover, the prescribed performance control method will be utilized to improve the transient state and steady state. An actor neural network is used to estimate unknown nonlinearities and uncertainties and a critic neural network is employed to evaluate system performance. In the sequel, the direct Lyapunov method will be used to prove the closed-loop control system stability and uniform ultimate boundedness of tracking errors. Finally, the effectiveness and efficiency of the proposed control scheme are confirmed by using MATLAB software.
Keywords: Actor neural network, Adaptive robust controller, Car-like mobile robot, Critic neural network, Look-ahead control, Reinforcement learning
Type of Article: Review paper |
Subject:
Special
Received: 2022/09/9 | Accepted: 2023/01/18 | ePublished ahead of print: 2023/02/13 | Published: 2023/02/20
Received: 2022/09/9 | Accepted: 2023/01/18 | ePublished ahead of print: 2023/02/13 | Published: 2023/02/20
References
1. [1] Dhall D, Kaur R, Juneja M. Machine learning: a review of the algorithms and its applications. Proceedings of ICRIC 2019. 2020:47-63. [DOI:10.1007/978-3-030-29407-6_5]
2. [2] Nasteski V. An overview of the supervised machine learning methods. Horizons. b. 2017 Dec;4:51-62. [DOI:10.20544/HORIZONS.B.04.1.17.P05]
3. [3] Sen PC, Hajra M, Ghosh M. Supervised classification algorithms in machine learning: A survey and review. InEmerging technology in modelling and graphics 2020 (pp. 99-111). Springer, Singapore. [DOI:10.1007/978-981-13-7403-6_11]
4. [4] Rajoub B. Supervised and unsupervised learning. InBiomedical Signal Processing and Artificial Intelligence in Healthcare 2020 Jan 1 (pp. 51-89). Academic Press. [DOI:10.1016/B978-0-12-818946-7.00003-2]
5. [5] Mahesh B. Machine learning algorithms-a review. International Journal of Science and Research (IJSR).[Internet]. 2020 Oct;9:381-6.
6. [6] Alpaydin E. Introduction to machine learning. MIT press; 2020 Mar 24. [DOI:10.7551/mitpress/13811.001.0001]
7. [7] Qiang W, Zhongli Z. Reinforcement learning model, algorithms and its application. In2011 International Conference on Mechatronic Science, Electric Engineering and Computer (MEC) 2011 Aug 19 (pp. 1143-1146). IEEE. [DOI:10.1109/MEC.2011.6025669]
8. [8] Kurinov I, Orzechowski G, Hämäläinen P, Mikkola A. Automated Excavator Based on Reinforcement Learning and Multibody System Dynamics. IEEE Access. 2020 Nov 24;8:213998-4006. [DOI:10.1109/ACCESS.2020.3040246]
9. [9] Sutton RS, Barto AG. Reinforcement learning: An introduction. MIT press; 2018 Nov 13.
10. [10] Hidalgo-Paniagua A, Vega-Rodríguez MA, Ferruz J. Applying the MOVNS (multi-objective variable neighborhood search) algorithm to solve the path planning problem in mobile robotics. Expert Systems with Applications. 2016 Oct 1;58:20-35 [DOI:10.1016/j.eswa.2016.03.035]
11. [11] Pourboghrat F, Karlsson MP. Adaptive control of dynamic mobile robots with nonholonomic constraints. Computers & Electrical Engineering. 2002 Jul 1;28(4):241-53. [DOI:10.1016/S0045-7906(00)00053-7]
12. [12] Tzafestas SG. Introduction to mobile robot control. Elsevier; 2013 Oct 3. [DOI:10.1016/B978-0-12-417049-0.00005-5]
13. [13] Simpson J, Jacobsen CL, Jadud MC. Mobile robot control. Communicating Process Architectures. 2006 Sep 6:225.
14. [14] Lee K, Chung W. Calibration of kinematic parameters of a car-like mobile robot to improve odometry accuracy. In2008 IEEE International Conference on Robotics and Automation 2008 May 19 (pp. 2546-2551). IEEE. [DOI:10.1109/ROBOT.2008.4543596]
15. ]15[کاظمی پور ملیحه، شجاعی خوشنام. کنترل آرایش بندی سطح دینامیکی زمان محدود فازی تطبیقی مبتنی بر رؤیتگر اغتشاش غیرخطی ربات های متحرک چرخ دار تراکتور- تریلر. مجله کنترل. ۱۴۰۰؛ ۱۵ (۱) :۱۱۲-۹۳
16. [16] Lee JH, Lin C, Lim H, Lee JM. Sliding mode control for trajectory tracking of mobile robot in the RFID sensor space. International Journal of Control, Automation and Systems. 2009 Jun;7(3):429-35. [DOI:10.1007/s12555-009-0312-7]
17. [17] Künhe F, Gomes J, Fetter W. Mobile robot trajectory tracking using model predictive control. In IEEE latin-American Robotics Symposium 2005 Sep (Vol. 51).
18. [18] Koubaa Y, Boukattaya M, Damak T. Adaptive sliding mode control for trajectory tracking of nonholonomic mobile robot with uncertain kinematics and dynamics. Applied Artificial Intelligence. 2018 Nov 26;32(9-10):924-38. [DOI:10.1080/08839514.2018.1519100]
19. [19] Mac Thi T, Copot C, De Keyser R, Tran TD, Vu T. MIMO fuzzy control for autonomous mobile robot. Journal of Automation and Control Engineering. 2016;4(1):65-70. [DOI:10.12720/joace.4.1.65-70]
20. [20] Mirzaeinejad H. Optimization-based nonlinear control laws with increased robustness for trajectory tracking of non-holonomic wheeled mobile robots. Transportation Research Part C: Emerging Technologies. 2019 Apr 1;101:1-7. [DOI:10.1016/j.trc.2019.02.003]
21. [21] Sabiha AD, Kamel MA, Said E, Hussein WM. ROS-based trajectory tracking control for autonomous tracked vehicle using optimized backstepping and sliding mode control. Robotics and Autonomous Systems. 2022 Jun 1;152:104058. [DOI:10.1016/j.robot.2022.104058]
22. [22] Shafei HR, Bahrami M. Trajectory tracking control of a wheeled mobile robot in the presence of matched uncertainties via a composite control approach. Asian Journal of Control. 2021 Nov;23(6):2805-23. [DOI:10.1002/asjc.2418]
23. [23] Chen Z, Liu Y, He W, Qiao H, Ji H. Adaptive Neural Network-Based Trajectory Tracking Control for a Nonholonomic Wheeled Mobile Robot with Velocity Constraints. IEEE Transactions on Industrial Electronics. 2020 Apr 28. [DOI:10.1109/TIE.2020.2989711]
24. [24] Fan QY, Yang GH, Ye D. Quantization-based adaptive actor-critic tracking control with tracking error constraints. IEEE transactions on neural networks and learning systems. 2017 Feb 1; 29 (4): 970-80. [DOI:10.1109/TNNLS.2017.2651104]
25. [25] Pan G, Xiang Y, Wang X, Yu Z, Zhou X. Research on path planning algorithm of mobile robot based on reinforcement learning. Soft Computing. 2022 Jul 21:1-0.
26. [26] Kiumarsi B, Vamvoudakis KG, Modares H, Lewis FL. Optimal and autonomous control using reinforcement learning: A survey. IEEE transactions on neural networks and learning systems. 2017 Dec 7;29(6):2042-62. [DOI:10.1109/TNNLS.2017.2773458]
27. [27] Yang Y, Gao W, Modares H, Xu CZ. Robust actor-critic learning for continuous-time nonlinear systems with unmodeled dynamics. IEEE Transactions on Fuzzy Systems. 2021 Apr 26. [DOI:10.1109/TFUZZ.2021.3075501]
28. [28] Zhang H, Wang H, Niu B, Zhang L, Ahmad AM. Sliding-mode surface-based adaptive actor-critic optimal control for switched nonlinear systems with average dwell time. Information Sciences. 2021 Nov 1;580:756-74. [DOI:10.1016/j.ins.2021.08.062]
29. [29] Yang X, Li B, Wen G. Adaptive Neural Network Optimized Control Using Reinforcement Learning of Critic-Actor Architecture for a Class of Non-Affine Nonlinear Systems. IEEE Access. 2021 Oct 15;9:141758-65. [DOI:10.1109/ACCESS.2021.3120835]
30. [30] Wen G, Xu L, Li B. Optimized backstepping tracking control using reinforcement learning for a class of stochastic nonlinear strict-feedback systems. IEEE Transactions on Neural Networks and Learning Systems. 2021 Aug 26.
31. [31] Yan L, Liu Z, Chen CP, Zhang Y, Wu Z. Reinforcement learning based adaptive optimal control for constrained nonlinear system via a novel state-dependent transformation. ISA Transactions. 2022 Jul 12. [DOI:10.21203/rs.3.rs-1066840/v1]
32. [32] Wei C, Luo J, Yin Z, Yuan J. Leader‐following consensus of second‐order multi‐agent systems with arbitrarily appointed‐time prescribed performance. IET Control Theory & Applications. 2018 Nov;12(16):2276-86. [DOI:10.1049/iet-cta.2018.5158]
33. [33] Yin Z, Luo J, Wei C. Robust prescribed performance control for Euler-Lagrange systems with practically finite-time stability. European Journal of Control. 2020 Mar 1;52:1-0. [DOI:10.1016/j.ejcon.2019.06.010]
34. [34] Li J, Du J, Lewis FL. Distributed three‐dimension time‐varying formation control with prescribed performance for multiple underactuated autonomous underwater vehicles. International Journal of Robust and Nonlinear Control. 2021 Sep 10;31(13):6272-87. [DOI:10.1002/rnc.5611]
35. [35] Kiumarsi B, Lewis FL. Actor-critic-based optimal tracking for partially unknown nonlinear discrete-time systems. IEEE Transactions on Neural Networks and Learning Systems. 2014 Oct 8;26(1):140-51. [DOI:10.1109/TNNLS.2014.2358227]
36. [36] Shojaei K. Neural adaptive PID formation control of car-like mobile robots without velocity measurements. Advanced Robotics. 2017 Sep 17;31(18):947-64. [DOI:10.1080/01691864.2017.1368413]
37. [37] Sarkar N, Yun X, Kumar V. Control of mechanical systems with rolling constraints: Application to dynamic control of mobile robots. The International Journal of Robotics Research. 1994 Feb;13(1):55-69. [DOI:10.1177/027836499401300104]
38. ]38[کیماسی خلجی علی، یزدانی ابوالفضل. کنترل تعقیب مسیر یک ربات چرخدار در حرکتهای رو به عقب و رو به جلو. مجله کنترل. ۱۳۹۹؛ ۱۴ (۲) :۴۵-۳۵
39. [39] Elhaki O, Shojaei K. "Neural network-based target tracking control of underactuated autonomous underwater vehicles with a prescribed performance." Ocean Engineering. Vol. 167, pp. 239-56, 2018. [DOI:10.1016/j.oceaneng.2018.08.007]
40. [40] Yun X, Yamamoto Y. Stability analysis of the internal dynamics of a wheeled mobile robot. Journal of Robotic Systems. 1997 Oct;14(10):697-709.
https://doi.org/10.1002/(SICI)1097-4563(199710)14:10<697::AID-ROB1>3.0.CO;2-P [DOI:10.1002/(SICI)1097-4563(199710)14:103.0.CO;2-P]
41. [41] Slotine JJ, Li W. Applied nonlinear control. Englewood Cliffs, NJ: Prentice hall; 1991 Jan.
42. [42] Swaroop D, Hedrick JK, Yip PP, Gerdes JC. Dynamic surface control for a class of nonlinear systems. IEEE Transactions on Automatic Control. 2000 Oct;45(10):1893-9. [DOI:10.1109/TAC.2000.880994]
43. [43] Bechlioulis CP, Rovithakis GA. Robust adaptive control of feedback linearizable MIMO nonlinear systems with prescribed performance. IEEE Transactions on Automatic Control. 2008 Oct 7;53(9):2090-9. [DOI:10.1109/TAC.2008.929402]
44. [44] Elhaki O, Shojaei K. A novel model-free robust saturated reinforcement learning-based controller for quadrotors guaranteeing prescribed transient and steady state performance. Aerospace Science and Technology. 2021 Dec 1;119:107128. [DOI:10.1016/j.ast.2021.107128]
45. [45] Xu L, Yao B. Output feedback adaptive robust precision motion control of linear motors. Automatica. 2001 Jul 1;37(7):1029-39. [DOI:10.1016/S0005-1098(01)00052-8]
Send email to the article author
| Rights and permissions | |
 |
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License. |
