Remaining useful life prediction using physics-informed neural network with self-attention mechanism and deep separable convolutional network
doi: 10.51393/j.jamst.2024018
Remaining useful life prediction using physics-informed neural network with self-attention mechanism and deep separable convolutional network
-
摘要: The remaining useful life prediction of rolling bearing holds significant importance in enhancing the operational reliability and reducing maintenance costs of the entire rotating machinery system. Deep learning techniques have shown promise in remaining useful life (RUL) prediction by leveraging their powerful representation learning capabilities. However, existing deep learning-based approaches still suffer from limitations such as reliance on hand-crafted features and lack of interpretability. Therefore, we propose an improved physicsinformed neural networks (PINNs) based on deep separable convolutional network (DSCN) and attention mechanism for the RUL estimation of rolling bearings. Specifically, a deep separable convolutional network is introduced for feature extraction, which directly utilizes multi-sensor data as inputs and employs separable convolutional building blocks to automatically learn high-level representations. The features are then mapped to RUL using a self-attention mechanism-based physics-informed neural network. The hybrid prediction framework called DSCN-AttnPINN has demonstrated superior performance on the XJTU-SY dataset. The results of the experiments reveal that the DSCN-AttnPINN can accurately predict RUL and outperforms certain current datadriven prognostics methods.Abstract: The remaining useful life prediction of rolling bearing holds significant importance in enhancing the operational reliability and reducing maintenance costs of the entire rotating machinery system. Deep learning techniques have shown promise in remaining useful life (RUL) prediction by leveraging their powerful representation learning capabilities. However, existing deep learning-based approaches still suffer from limitations such as reliance on hand-crafted features and lack of interpretability. Therefore, we propose an improved physicsinformed neural networks (PINNs) based on deep separable convolutional network (DSCN) and attention mechanism for the RUL estimation of rolling bearings. Specifically, a deep separable convolutional network is introduced for feature extraction, which directly utilizes multi-sensor data as inputs and employs separable convolutional building blocks to automatically learn high-level representations. The features are then mapped to RUL using a self-attention mechanism-based physics-informed neural network. The hybrid prediction framework called DSCN-AttnPINN has demonstrated superior performance on the XJTU-SY dataset. The results of the experiments reveal that the DSCN-AttnPINN can accurately predict RUL and outperforms certain current datadriven prognostics methods.
-
[1] . Rezaeianjouybari B, Shang Y. Deep learning for prognostics and health management: State of the art, challenges, and opportunities. Measurement 2020;163:107929. [2] . Wei J, Dong G, Chen Z. Remaining useful life prediction and state of health diagnosis for lithium-ion batteries using particle filter and support vector regression. IEEE Transactions on Industrial Electronics 2017;65(7):5634–5643. [3] . Cui L, Wang X, Wang H, et al. Research on remaining useful life prediction of rolling element bearings based on time-varying Kalman filter. IEEE Transactions on Instrumentation and Measurement 2019;69(6):2858–2867. [4] . Li X, Ding Q, Sun JQ. Remaining useful life estimation in prognostics using deep convolution neural networks. Reliability Engineering & System Safety 2018;172:1–11. [5] . Liao Y, Zhang L, Liu C. Uncertainty prediction of remaining useful life using long short-term memory network based on bootstrap method. In: IEEE International Conference on Prognostics and Health Management (ICPHM); 2018. [6] . Ren L, Sun Y, Cui J, et al. Bearing remaining useful life prediction based on deep autoencoder and deep neural networks. Journal of Manufacturing Systems 2018;48:71–77. [7] . Cao Y, Ding Y, Jia M, et al. A novel temporal convolutional network with residual selfattention mechanism for remaining useful life prediction of rolling bearings. Reliability Engineering & System Safety 2021;215:107813. [8] . Ding H, Yang L, Cheng Z, et al. A remaining useful life prediction method for bearing based on deep neural networks. Measurement 2021;172: 108878. [9] . Hinton GE, Osindero S, Teh YW. A fast learning algorithm for deep belief nets. Neural Computation 2006;18(7):1527–1554. [10] . Bengio Y, Simard P, Frasconi P. Learning longterm dependencies with gradient descent is difficult. IEEE Transactions on Neural Networks 1994;5(2):157–166. [11] . Loutas TH, Roulias D, Georgoulas G. Remaining useful life estimation in rolling bearings utilizing data-driven probabilistic Esupport vectors regression. IEEE Transactions on Reliability 2013; 62(4):821–832. [12] . Alfarizi MG, Tajiani B, Vatn J, et al. Optimized random forest model for remaining useful life prediction of experimental bearings. IEEE Transactions on Industrial Informatics 2023; 19(6): 7771–7779. [13] . Ali JB, Chebel-Morello B, Saidi L, et al. Accurate bearing remaining useful life prediction based on Weibull distribution and artificial neural network. Mechanical Systems and Signal Processing 2015;56–57:150–172. [14] . Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need. In: 31st Conference on Neural Information Processing Systems; 2017. [15] . Liu L, Song X, Zhou Z. Aircraft engine remaining useful life estimation via a double attention-based data-driven architecture. Reliability Engineering & System Safety 2022;221:108330. [16] . Xu D, Xiao X, Liu J, et al. Spatio-temporal degradation modeling and remaining useful life prediction under multiple operating conditions based on attention mechanism and deep learning. Reliability Engineering & System Safety 2023;229: 108886. [17] . Li H, Cao P, Wang X, et al. Multi-task spatiotemporal augmented net for industry equipment remaining useful life prediction. Advanced Engineering Informatics 2023;55:101898. [18] . You KS, Qiu GQ and Gu YK. A 3D attentionenhanced hybrid neural network for turbofan engine remaining life prediction using CNN and BiLSTM models. IEEE Sensors Journal 2023; online, doi: 10.1109/JSEN.2023.3296670. [19] . Soleimani M, Campean F, Neagu D. Integration of Hidden Markov Modelling and Bayesian Network for fault detection and prediction of complex engineered systems. Reliability Engineering & System Safety 2021;215:107808. [20] . Karniadakis GE, Kevrekidis IG, Lu L, et al. Physics-informed machine learning. Nature Reviews Physics 2021;3(6):422–440. [21] . Karimian SF, Moradi R, Cofre-Martel S, et al. Neural network and particle filtering: a hybrid framework for crack propagation prediction. arXiv preprint 2020; arXiv:2004.13556. [22] . Xu Y, Kohtz S, Boakye J, et al. Physics-informed machine learning for reliability and systems safety applications: State of the art and challenges. Reliability Engineering & System Safety 2023; 230:108900. [23] . Raissi M, Perdikaris P, Karniadakis GE. Physicsinformed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics 2019;378: 686–707. [24] . Cai S, Mao Z, Wang Z, et al. Physics-informed neural networks (PINNs) for fluid mechanics: A review. Acta Mechanica Sinica 2021;37(12): 1727–1738. [25] . Cofre-Martel S, Lopez Droguett E, Modarres M. Remaining useful life estimation through deep learning partial differential equation models: A framework for degradation dynamics interpretation using latent variables. Shock and Vibration 2021;2021:9937846. [26] . Raissi M. Deep hidden physics models: Deep learning of nonlinear partial differential equations. The Journal of Machine Learning Research 2018;19(1):932–955. [27] . Liao X, Chen S, Wen P, et al. Remaining useful life with self-attention assisted physics-informed neural network. Advanced Engineering Informatics 2023;58:102195. [28] . Wang B, Lei Y, Li N, et al. Deep separable convolutional network for remaining useful life prediction of machinery. Mechanical Systems and Signal Processing 2019;134:106330. [29] . Wang B, Lei Y, Li N, et al. A hybrid prognostics approach for estimating remaining useful life of rolling element bearings. IEEE Transactions on Reliability 2018;69(1):401–412. [30] . Ioffe S, Szegedy C. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In: 32nd International Conference on Machine Learning; 2015. p. 448– 456. [31] . Nair V, Hinton GE. Rectified linear units improve restricted Boltzmann machines. In: 27th international conference on machine learning;2010. p. 807–814. [32] . Saxena A, Goebel K, Simon D, et al. Damage propagation modeling for aircraft engine run-tofailure simulation. In: 2008 International Conference on Prognostics and Health Management; 2008. [33] . Zheng Y, Liu Q, Chen E, et al. Time series classification using multi-channels deep convolutional neural networks. In: International Conference on Web-Age Information Management; 2014. p. 298–310. [34] . He K, Zhang X, Ren S, et al. Deep residual learning for image recognition. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR); 2016. p. 770–778. [35] . Li X, Zhang W, Ding Q. Deep learning-based remaining useful life estimation of bearings using multi-scale feature extraction. Reliability Engineering & System Safety 2019;182:208–218. [36] . Yao D, Li B, Liu H, et al. Remaining useful life prediction of roller bearings based on improved 1D-CNN and simple recurrent unit. Measurement 2021;175:109166. [37] . Si XS, Wang W, Chen MY, et al. A degradation path-dependent approach for remaining useful life estimation with an exact and closed-form solution. European Journal of Operational Research 2013; 226(1):53–66. [38] . Liu X, Liu S, Xiang J, et al. A transfer learning strategy based on numerical simulation driving 1D Cycle-GAN for bearing fault diagnosis. Information Sciences 2023;642:119175. [39] . Xiang P, Yan L, Xiao H, et al. Development of a novel radial-flux machine with enhanced torque profile employing quasi-cylindrical pm pattern. IEEE Transactions on Energy Conversion 2023;38(4):2772–2783. -
下载:
点击查看大图
图(1)
计量
- 文章访问数: 163
- HTML全文浏览量: 79
- PDF下载量: 12
- 被引次数: 0
京公网安备 11010502036328号 京ICP备17033152号