1. [1] L. Ljung, "System identification, Theory for the user." System science series, Prentice Hall, Upper Saddle River, NJ, USA, Second edition, 1999.
2. [2] Zadeh, L. "On the identification problem." IRE Transactions on Circuit Theory 3.4 (1956): 277-281. [
DOI:10.1109/TCT.1956.1086328]
3. [3] Nelles, Oliver. "Nonlinear system identification, from classical approach to neural networks, fuzzy systems, and Gaussian process". Springer, Berlin, Heidelberg, 2020. [
DOI:10.1007/978-3-030-47439-3]
4. [4] T. Soderstrom, P. Stocia, "System identification", Prentice-Hall, Inc, 1988.
5. [5] Schoukens, Johan, and Lennart Ljung. "Nonlinear system identification: A user-oriented road map." IEEE Control Systems Magazine 39.6 (2019): 28-99. [
DOI:10.1109/MCS.2019.2938121]
6. [6] Billings, Stephen A. "Identification of nonlinear systems-a survey." IEE Proceedings D (Control Theory and Applications). Vol. 127. No. 6. IET Digital Library, 1980. [
DOI:10.1049/ip-d.1980.0047]
7. [7] Zheng, Qingsheng, and Evanghelos Zafiriou. "Nonlinear system identification for control using Volterra-Laguerre expansion." Proceedings of 1995 American Control Conference-ACC'95. Vol. 3. IEEE, 1995.
8. [8] Korenberg, Michael J., and Ian W. Hunter. "The identification of nonlinear biological systems: Wiener kernel approaches." Annals of Biomedical Engineering 18 (1990): 629-654. [
DOI:10.1007/BF02368452]
9. [9] Schoukens, Maarten, and Koen Tiels. "Identification of block-oriented nonlinear systems starting from linear approximations: A survey." Automatica 85 (2017): 272-292. [
DOI:10.1016/j.automatica.2017.06.044]
10. [10] Chiuso, Alessandro, and Gianluigi Pillonetto. "System identification: A machine learning perspective." Annual Review of Control, Robotics, and Autonomous Systems 2 (2019): 281-304. [
DOI:10.1146/annurev-control-053018-023744]
11. [11] Pillonetto, Gianluigi, et al. "Kernel methods in system identification, machine learning and function estimation: A survey." Automatica 50.3 (2014): 657-682. [
DOI:10.1016/j.automatica.2014.01.001]
12. [12] Bishop, Christopher M., and Nasser M. Nasrabadi. Pattern recognition and machine learning. Vol. 4. No. 4. New York: springer, 2006.
13. [13] Heaton, Jeff. Ian goodfellow, yoshua bengio, and aaron courville: "Deep learning." (2018): 305-307. [
DOI:10.1007/s10710-017-9314-z]
14. [14] Bengio, Yoshua, Aaron Courville, and Pascal Vincent. "Representation learning: A review and new perspectives." IEEE transactions on pattern analysis and machine intelligence 35.8 (2013): 1798-1828. [
DOI:10.1109/TPAMI.2013.50]
15. [15] Noroozi, Mehdi, and Paolo Favaro. "Unsupervised learning of visual representations by solving jigsaw puzzles." European conference on computer vision. Cham: Springer International Publishing, 2016. [
DOI:10.1007/978-3-319-46466-4_5]
16. [16] Kingma, Diederik P., and Max Welling. "Auto-encoding variational bayes." arXiv preprint arXiv: 1312.6114 (2013).
17. [17] Fraccaro, Marco. "Deep latent variable models for sequential data." English. PhD thesis DTU University (2018).
18. [18] Barber, David. Bayesian reasoning and machine learning. Cambridge University Press, 2012. [
DOI:10.1017/CBO9780511804779]
19. [19] Koller, Daphne, and Nir Friedman. Probabilistic graphical models: principles and techniques. MIT press, 2009.
20. [20] Hinton, Geoffrey E. "A practical guide to training restricted Boltzmann machines." Neural Networks: Tricks of the Trade: Second Edition. Berlin, Heidelberg: Springer Berlin Heidelberg, 2012. 599-619. [
DOI:10.1007/978-3-642-35289-8_32]
21. [21] Fischer, Asja, and Christian Igel. "An introduction to restricted Boltzmann machines." Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications: 17th Iberoamerican Congress, CIARP 2012, Buenos Aires, Argentina, September 3-6, 2012. Proceedings 17. Springer Berlin Heidelberg, 2012.
22. [22] Raia Hadsell, Sumit Chopra, and Yann LeCun. Dimensionality reduction by learning an invariant mapping. In CVPR, 2006.
23. [23] Gianluigi Pillonetto, Aleksandr Aravkin, Daniel Gedon, Lennart Ljung, Antônio H. Ribeiro, Thomas B. Schön, Deep network for system identification, arXiv, 2023.
24. [24] Ljung, Lennart, et al. "Deep learning and system identification." IFAC-PapersOnLine 53.2 (2020): 1175-1181. [
DOI:10.1016/j.ifacol.2020.12.1329]
25. [25] M. Forgione and D. Piga. Dynonet: A neural network architecture for learning dynamical systems. Int. J. Adapt. Control Signal Process., 35(4):612-626, 2021. [
DOI:10.1002/acs.3216]
26. [26] J. Hendriks, F.K. Gustafsson, A.H. Ribeiro, A. Wills, and T.B. Schon. Deep energy-based NARX models. In Proceedings of the 19th IFAC Symposium on System Identification (SYSID), 2021. [
DOI:10.1016/j.ifacol.2021.08.410]
27. [27] LeCun, Yann, et al. "Energy-based models in document recognition and computer vision." Ninth International Conference on Document Analysis and Recognition (ICDAR 2007). Vol. 1. IEEE, 2007. [
DOI:10.1109/ICDAR.2007.4378728]
28. [28] Andersson, Carl, et al. "Deep convolutional networks in system identification." 2019 IEEE 58th conference on decision and control (CDC). IEEE, 2019. [
DOI:10.1109/CDC40024.2019.9030219]
29. [29] Lea, Colin, et al. "Temporal convolutional networks: A unified approach to action segmentation." Computer Vision-ECCV 2016 Workshops: Amsterdam, The Netherlands, October 8-10 and 15-16, 2016, Proceedings, Part III 14. Springer International Publishing, 2016.
30. [30] R. Calandra, J. Peters, C. Rasmussen, and M.P. Deisenroth. Manifold gaussian processes for regression. In 2016 International Joint Conference on Neural Networks (IJCNN), pages 3338-3345,2016. [
DOI:10.1109/IJCNN.2016.7727626]
31. [31] Y. Cho and L. Saul. Kernel methods for deep learning. In Advances in Neural Information Processing Systems, volume 22, 2009.
32. [32] Nagel, Tobias, and Marco F. Huber. "Autoencoder-inspired Identification of LTI systems." 2021 European Control Conference (ECC). IEEE, 2021. [
DOI:10.23919/ECC54610.2021.9655185]
33. [33] D. Gedon, N. Wahlstr¨om, T.B. Sch¨on, and L. Ljung. Deep state space models for nonlinear system identification. In Proceedings of the 19th IFAC Symposium on System Identification (SYSID), 2021. [
DOI:10.1016/j.ifacol.2021.08.406]
34. [34] M. Karl, M. Soelch, J. Bayer, and P. van der Smagt. Deep variational Bayes filters: Unsupervised learning of state space models from raw data, 2017.
35. [35] M. Watter, J. Springenberg, J. Tobias, J. Boedecker, and M. Riedmiller. Embed to control: A locally linear latent dynamics model for control from raw images. In Proceedings of the 28th International Conference on Neural Information Processing Systems, Volume 2, pages 2746-2754, Cambridge, MA, USA, 2015. MIT Press.
36. [36] Rangapuram, Syama Sundar, et al. "Deep state space models for time series forecasting." Advances in neural information processing systems 31 (2018).
37. [37] Courts, Jarrad, et al. "Variational state and parameter estimation." IFAC-PapersOnLine 54.7 (2021): 732-737. [
DOI:10.1016/j.ifacol.2021.08.448]
38. [38] Courts, Jarrad, et al. "Variational System Identification for Nonlinear State-Space Models." arXiv preprint arXiv: 2012.05072 (2020).
39. [39] Menghani, Gaurav. "Efficient deep learning: A survey on making deep learning models smaller, faster, and better." ACM Computing Surveys 55.12 (2023): 1-37. [
DOI:10.1145/3578938]
40. [40] Xu, Canwen, and Julian McAuley. "A survey on model compression and acceleration for pretrained language models." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 37. No. 9. 2023. [
DOI:10.1609/aaai.v37i9.26255]
41. [41] Choudhary, Tejalal, et al. "A comprehensive survey on model compression and acceleration." Artificial Intelligence Review 53 (2020): 5113-5155. [
DOI:10.1007/s10462-020-09816-7]
42. [42] Cheng, Yu, et al. "A survey of model compression and acceleration for deep neural networks." arXiv preprint arXiv:1710.09282 (2017).
43. [43] Cheng, Yu, et al. "Model compression and acceleration for deep neural networks: The principles, progress, and challenges." IEEE Signal Processing Magazine 35.1 (2018): 126-136. [
DOI:10.1109/MSP.2017.2765695]
44. [44] Liang, Tailin, et al. "Pruning and quantization for deep neural network acceleration: A survey." Neurocomputing 461 (2021): 370-403. [
DOI:10.1016/j.neucom.2021.07.045]
45. [45] Rokh, Babak, Ali Azarpeyvand, and Alireza Khanteymoori. "A comprehensive survey on model quantization for deep neural networks." arXiv preprint arXiv: 2205.07877 (2022).
46. [46] Gholami, Amir, et al. "A survey of quantization methods for efficient neural network inference." Low-Power Computer Vision. Chapman and Hall/CRC, 2022. 291-326. [
DOI:10.1201/9781003162810-13]
47. [47] Liang, Tailin, et al. "Pruning and quantization for deep neural network acceleration: A survey." Neurocomputing 461 (2021): 370-403. [
DOI:10.1016/j.neucom.2021.07.045]
48. [48] Frankle, Jonathan, and Michael Carbin. "The lottery ticket hypothesis: Finding sparse, trainable neural networks." arXiv preprint arXiv: 1803.03635 (2018).
49. [49] Hinton, Geoffrey, Oriol Vinyals, and Jeff Dean. "Distilling the knowledge in a neural network." arXiv preprint arXiv: 1503.02531 (2015).
50. [50] Gou, Jianping, et al. "Knowledge distillation: A survey." International Journal of Computer Vision 129 (2021): 1789-1819. [
DOI:10.1007/s11263-021-01453-z]
51. [51] Hastie, Trevor, et al. The elements of statistical learning: data mining, inference, and prediction. Vol. 2. New York: springer, 2009. [
DOI:10.1007/978-0-387-84858-7]
52. [52] Doersch, Carl. "Tutorial on variational autoencoders." arXiv preprint arXiv: 1606.05908 (2016).
53. [53] Burda, Yuri, Roger Grosse, and Ruslan Salakhutdinov. "Importance weighted autoencoders." arXiv preprint arXiv: 1509.00519 (2015) [40] Barber, David, A. Taylan Cemgil, and Silvia Chiappa, eds. Bayesian time series models. Cambridge University Press, 2011.
54. [54] Makhzani, Alireza, et al. "Adversarial autoencoders." arXiv preprint arXiv: 1511.05644 (2015). [39] Burda, Yuri, Roger Grosse, and Ruslan Salakhutdinov. "Importance weighted autoencoders." arXiv preprint arXiv: 1509.00519 (2015)
55. [55] Girin, Laurent, et al. "Dynamical variational autoencoders: A comprehensive review." arXiv preprint arXiv:2008.12595 (2020). [
DOI:10.1561/9781680839135]
56. [56] Beal, Matthew James. Variational algorithms for approximate Bayesian inference. University of London, University College London (United Kingdom), 2003.
57. [57] Hochreiter, Sepp, and Jürgen Schmidhuber. "Long short-term memory." Neural computation 9.8 (1997): 1735-1780. [
DOI:10.1162/neco.1997.9.8.1735]
58. [58] Chung, Junyoung, et al. "Empirical evaluation of gated recurrent neural networks on sequence modeling." arXiv preprint arXiv:1412.3555 (2014).
59. [59] Su, Yuanhang, and C-C. Jay Kuo. "Recurrent neural networks and their memory behavior: a survey." APSIPA Transactions on Signal and Information Processing 11.1 (2022). [
DOI:10.1561/116.00000123]
60. [60] Graves, Alex. "Generating sequences with recurrent neural networks." arXiv preprint arXiv:1308.0850 (2013).
61. [61] Krishnan, Rahul G., Uri Shalit, and David Sontag. "Deep kalman filters." arXiv preprint arXiv: 1511.05121 (2015).
62. [62] Bahdanau, Dzmitry, Kyunghyun Cho, and Yoshua Bengio. "Neural machine translation by jointly learning to align and translate." arXiv preprint arXiv: 1409.0473 (2014).
63. [63] Vaswani, Ashish, et al. "Attention is all you need." Advances in neural information processing systems 30 (2017).
64. [64] Han, Kai, et al. "A survey on vision transformer." IEEE transactions on pattern analysis and machine intelligence 45.1 (2022): 87-110. [
DOI:10.1109/TPAMI.2022.3152247]
65. [65] Khan, Salman, et al. "Transformers in vision: A survey." ACM computing surveys (CSUR) 54.10s (2022): 1-41. [
DOI:10.1145/3505244]
66. [66] Liu, Yang, et al. "A survey of visual transformers." IEEE Transactions on Neural Networks and Learning Systems (2023). [
DOI:10.1109/TNNLS.2022.3227717]
67. [67] Lin, Tianyang, et al. "A survey of transformers." AI Open (2022). [
DOI:10.1016/j.aiopen.2022.10.001]
68. [68] Rosendahl, Jan, et al. "Analysis of positional encodings for neural machine translation." Proceedings of the 16th International Conference on Spoken Language Translation. 2019.
69. [69] Beintema, Gerben I., Maarten Schoukens, and Roland Tóth. "Deep subspace encoders for nonlinear system identification." Automatica 156 (2023): 111210. [
DOI:10.1016/j.automatica.2023.111210]
70. [70] Masti, Daniele, and Alberto Bemporad. "Learning nonlinear state-space models using autoencoders." Automatica 129 (2021): 109666. [
DOI:10.1016/j.automatica.2021.109666]
71. [71] Lopez, Ryan, and Paul J. Atzberger. "Variational autoencoders for learning nonlinear dynamics of physical systems." arXiv preprint arXiv:2012.03448 (2020).
72. [72] Chung, Junyoung, et al. "A recurrent latent variable model for sequential data." Advances in neural information processing systems 28 (2015).
73. [73] Bayer, Justin, and Christian Osendorfer. "Learning stochastic recurrent networks." arXiv preprint arXiv: 1411.7610 (2014).
74. [74] Fraccaro, Marco, et al. "Sequential neural models with stochastic layers." Advances in neural information processing systems 29 (2016).
75. [75] Fraccaro, Marco, et al. "A disentangled recognition and nonlinear dynamics model for unsupervised learning." Advances in neural information processing systems 30 (2017).