Optimal modeling of nonlinear systems

Method of variable injections

Authors

DOI:

https://doi.org/10.22199/issn.0717-6279-5984

Keywords:

Nonlinear systems, Error minimization, Optimization

Abstract

Our work addresses a development and justification of the new approach to the modeling of nonlinear systems. Let $\f$ be an unknown input-output map of the system with a random input and output $\y$ and $\x$, respectively. It is assumed that $\y$ and $\x$ are available and covariance matrices formed from $\y$ and $\x$ are known. We determine a model of $\f$ so that an associated error is minimized. To this end, the model $\ttt_p$ is constructed as a sum of $p+1$ particular parts, in the form $\ttt_p (\y) = \sum_{j=0}^{p}G_j H_j Q_j(\vv_j)$ where $G_j$ and $ H_j$, for $j=0,\ldots, p$, are matrices to be determined, and $\vv_j$, for $j=1,\ldots,p$, is a special random vector called the injection. We denote $\vv_0=\y$. Injections $\vv_1,\ldots, \vv_p$ are aimed to diminish the error associated with the proposed model $\ttt_p$. Further, $Q_j$ is a special transform aimed to facilitate the numerical realization of model $\ttt_p$. It is determined in the way allowing us to optimally determine $G_j$ and $ H_j$ as a solution of $p+1$ separate error minimization problems which are simpler than the original minimization problem. The empirical determination of injections $\vv_1,\ldots, \vv_p$ is considered. The proposed method has several degrees of freedom to diminish the associated error. They are `degree' $p$ of $\ttt_p$, choice of matrices $G_0, H_0, \ldots,$ $G_p, H_p$, dimensions of matrices $G_0, H_0, \ldots,$ $G_p, H_p$ and injections $\vv_1,\ldots, \vv_p$, respectively. Four numerical examples are provided. At the end, the open problem is formulated.

Author Biographies

Anatoli Torokhti, University of South Australia.

STEM discipline.

Juan Pablo Soto-Quiros, Instituto Tecnológico de Costa Rica.

Escuela de Matemática

References

Adali, T., Schreier, P. J., and Scharf, L. L. Complex-valued signal processing: The proper way to deal with impropriety. IEEE Transactions on Signal Processing 59, 11 (Nov 2011), 5101–5125.

Adamczak, R., Litvak, A. E., Pajor, A., and Tomczak-Jaegermann, N. Quantitative estimates of the convergence of the empirical covariance matrix in log-concave ensembles. Journal of the American Mathematical Society 2010, 23 (2009), 535–561.

Alter, O., and Golub, G. H. Singular value decomposition of genome-scale mRNA lengths distribution reveals asymmetry in RNA gel electrophoresis band broadening. Processing of the National Academy of Sciences of USA 103, 32 (2006), 11828 –11833.

Ambrosi, D., Quarteroni, A., and (Eds)., G. R. Modelling of Physiological Flows. Vol. V, Springer -Verlag Italia, Milano, 2011.

Aminzare, Z., and Sontag, E. D. Synchronization of diffusively-connected nonlinear systems: Results based on contractions with respect to general norms. IEEE Tranactions on Network Science and Engineering 1, 2 (2014), 91 – 106.

Azimia, R., Ghayekhlooa, M., Ghofranib, M., and Sajedic, H. A novel clustering algorithm based on data transformation approaches. Expert Systems with Applications 76, 15 (2017), 59 – 70.

Bai, L., Dou, S., Xiao, Z., and Choi, J. Doubly iterative multiple-input-multiple-output-bit-interleaved coded modulation receiver with joint channel estimation and randomised sampling detection. IET Signal Processing 10, 4 (2016), 335–341.

Ben-Israel, A., and Greville, T. N. E. Generalized Inverses: Theory and Applications (2nd ed.). Springer, New York, 2003.

Boutsidis, C., and Woodruff, D. P. Optimal CUR matrix decompositions. SIAM Journal on Computing 46, 2 (2017), 543–589.

Brillinger, D. R. Time Series: Data Analysis and Theory. SIAM, San Francisco, 2001.

Bühlmann, P., and van de Geer, S. Statistics for High-Dimensional Data. Springer, 2011.

Burge, M. J., and Burger, W. Digital Image Processing - An Algorithmic Introduction Using Java. Springer-Verlag London, 2006.

Chen, T., and Chen, H. Universal approximation to nonlinear operators by neural networks with arbitrary activation functions and its application to dynamical systems. IEEE Transactions on Neural Networks 6, 4 (1995), 911 – 917.

Chenn, S., and Billings, S. A. Representations of non-linear systems: the NARMAX model. International Journal of Control 49, 3 (1989), 1013 – 1032.

Chenn, S., and Billings, S. A. Neural networks for nonlinear dynamic system modelling and identification. International Journal of Control 56, 2 (1992), 319 – 346.

Chenn, S., Billings, S. A., and Grant, P. Non-linear system identification using neural networks. International Journal of Control 51, 6 (1990), 1191 – 1214.

de Moura, E. P., de Abreu Melo Junior, F., Damasceno, F., Figueiredo, L., de Andrade, C., and de Almeida, M. Classification of imbalance levels in a scaled wind turbine through detrended fluctuation analysis of vibration signals. Renewable Energy 96, Part A (2016), 993 – 1002.

Dingle, K., Camargo, C. Q., and Louis, A. A. Input–output maps are strongly biased towards simple outputs. Nature Communications 9 (2018) Article number: 761 (2018).

Fomin, V. N., and Ruzhansky, M. V. Abstract optimal linear filtering. SIAM Journal on Control and Optimization 38, 5 (2000), 1334 – 1352.

Formaggia, L., Quarteroni, A., and (Eds.), A. V. Cardiovascular Mathematics - Modeling and simulation of the circulatory system. Vol. 1, Springer - Verlag Italia, Milano, 2009.

Friedland, S., and Torokhti, A. Generalized rank-constrained matrix approximations. SIAM Journal on Matrix Analysis and Applications 29, 2 (2007), 656–659.

Gallman, P. G., and Narendra, K. Representations of nonlinear systems via the Stone-Weierstrass theorem. Automatica 12, 6 (1976), 619 – 622.

Gianfelici, F., Turchetti, C., and Crippa, P. A non-probabilistic recognizer of stochastic signals based on KLT. Signal Processing 89, 4 (2009), 422 – 437.

Gini, F., and Lombardini, F. Multibaseline cross-track SAR interferometry: a signal processing perspective. IEEE Aerospace and Electronic Systems Magazine 20, 8 (Aug 2005), 71–93.

Howlett, P. G., Torokhti, A. P., and Pearce, C. E. M. A philosophy for the modelling of realistic non-linear systems. Processing of Amer. Math. Soc. 131, 2 (2003), 353–363.

Hua, Y., and Liu, W. Generalized Karhunen-Loeve transform. IEEE Signal Processing Letters 5, 6 (June 1998), 141–142.

Hua, Y., Nikpour, M., and Stoica, P. Optimal reduced-rank estimation and filtering. IEEE Transactions on Signal Processing 49, 3 (Mar 2001), 457–469.

Jolliffe, I. Principal Component Analysis, Second Edition. Springer, New York, 2002.

Joong-Ho Won, Johan Lim, S.-J. K., and Rajaratnam, B. Condition-number-regularized covariance estimation. Journal of the Royal Statistical Society: Series B) 75, 3 (2013), 427–450.

Kailath, T., Saed, A., and Hassibi, B. Linear estimation. Prentice-Hall, 2000.

Kekatos, V., and Giannakis, G. B. Sparse Volterra and polynomial regression models: Recoverability and estimation. IEEE Transactions on Signal Processing 59, 12 (2011), 5907 – 5920.

Kitagawa, G., and Gersch, W. Estimation of Time Varying Variance. Springer New York, 1996, ch. 10, pp. 137–145.

Kountchev, R., and Kountcheva, R. Hierarchical adaptive KL-based transform: Algorithms and applications. Computer Vision in Control Systems-1, Favorskaya, Margarita N and Jain, Lakhmi C. (Eds.) 73 (2015), 91–136.

Ledoit, O., and Wolf, M. A well-conditioned estimator for large-dimensional covariance matrices. Journal of Multivariate Analysis 88, 2 (2004), 365 – 411.

Ledoit, O., and Wolf, M. Nonlinear shrinkage estimation of large-dimensional covariance matrices. The Annals of Statistics 40, 2 (2012), 1024–1060.

Liao, Y., and Chen, K. Estimation of stationary phase noise by the autocorrelation of the ici weighting function in OFDM systems. IEEE Transactions on Wireless Comm. 5, 12 (December 2006), 3370–3374.

Liesen, J., and Strakos, Z. Krylov Subspace Methods: Principles and Analysis. OUP Oxford, 2013.

Liu, W., Zhang, H., Yu, K., and Tan, X. Optimal linear filtering for networked systems with communication constraints, fading measurements, and multiplicative noises. International Journal of Adaptive Control and Signal Processing (https://onlinelibrary.wiley.com/doi/abs/10.1002/acs.2745 2016).

Liu, X., Li, W., and Wang, H. Rank constrained matrix best approximation problem with respect to (skew) Hermitian matrices. Journal of Computational and Applied Mathematics 319 (2017), 77 – 86.

Lombardini, F., and Gini, F. Model order selection in multi-baseline interferometric radar systems. EURASIP Journal on Advances in Signal Processing 2005, 20 (2005), 3206–3219.

Navarro-Moreno, J., Estudillo-Martinez, M. D., Fernandez-Alcala, R. M., and Ruiz-Molina, J. C. Estimation of improper complex-valued random signals in colored noise by using the Hilbert space theory. IEEE Transactions on Information Theory 55 (2009), 2859–2867.

Papadatos, N., and Xifara, T. A simple method for obtaining the maximal correlation coefficient and related characterizations. Journal of Multivariate Analysis 118 (2013), 102–114.

Phophalia, A., Rajwade, A., and K. Mitra, S. Rough set based image denoising for brain MR images. Signal Processing 103 (2014), 24 – 35.

Piotrowski, T., Cavalcante, R., and Yamada, I. Stochastic MV-PURE estimator - robust reducedrank estimator for stochastic linear model. IEEE Transactions on Signal Processing 57, 4 (2009), 1293 –1303.

Piotrowski, T., and Yamada, I. Performance of the stochastic MV-PURE estimator in highly noisy settings. Journal of The Franklin Insitute 351 (2014), 3339 – 3350.

Piroddi, L., and Spinelli, W. An identification algorithm for polynomial NARX models based on simulation error minimization. International Journal of Control 76, 17 (2003), 1767 – 1781.

Ponnusamy, S. Foundations of Mathematical Analysis. Birkh¨auser Boston, 2011.

Poor, H. V. An Introduction to Signal Processing and Estimation. Springer-Verlag, 2nd ed. New York, 2001.

Saghri, J. A., Schroeder, S., and Tescher, A. G. Adaptive two-stage Karhunen-Loeve-transform scheme for spectral decorrelation in hyperspectral bandwidth compression. Optical Engineering 49, 5 (2010), 057001–1 – 057001–7.

Sandberg, I. Time-delay polynomial networks and quality of approximation. IEEE Trans. on Circuits and Systems. Part 1, Fundamental theory and applications 47 (2000), 40 – 49.

Sandberg, I. W. Notes on uniform approximation of time–varying systems on finite time intervals. IEEE Trans. on Circuits and Syst. Part 1. Fund. Theory and Applic. 45 (1998), 863 – 864.

Sandberg, I. W. R+ fading memory and extensions of input - output maps. IEEE Trans. on Circuits and Syst. Part 1. Fund. Theory and Applic. 49, 11 (2002), 1586 – 1591.

Scharf, L. The SVD and reduced rank signal processing. Signal Processing 25, 2 (1991), 113 – 133.

Schneider, M. K., and Willsky, A. S. A Krylov subspace method for covariance approximation and simulation of random processes and fields. Multidimensional Systems and Signal Processing 14, 4 (2003), 295–318.

Schoukens, M., and Tiels, K. Identification of block-oriented nonlinear systems starting from linear approximations: A survey. Automatica 85 (November 2017), 272 – 292.

Schreier, P. J., Scharf, L. L., and Mullis, C. T. Detection and estimation of improper complex random signals. IEEE Transactions on Information Theory 51, 1 (Jan 2005), 306–312.

She, Y., Li, S., and Wu, D. Robust orthogonal complement principal component analysis. Journal of the American Statistical Association 111, 514 (2016), 763–771.

Stein, C. Estimation of the mean of a multivariate normal distribution. In Processing Prague Symp. Asymptotic Statist. (1973), pp. 345 – 381.

Temlyakov, V. N. Nonlinear methods of approximation. Foundations of Computational Mathematics 3, 1 (2003), 33–107.

Torokhti, A., and Friedland, S. Towards theory of generic Principal Component Analysis. Journal of Multivariate Analysis 100, 4 (2009), 661 – 669.

Torokhti, A., and Howlett, P. Optimal fixed rank transform of the second degree. IEEE Trans. CAS. Part II, Analog and Digital Signal Processing 48, 3 (2001), 309 – 315.

Torokhti, A., and Howlett, P. Computational Methods for Modelling of Nonlinear Systems. Elsevier, Amsterdam, 2007.

Torokhti, A., Howlett, P., and Laga, H. Estimation of stochastic signals under partially missing information. Signal Processing 11 (2015), 199–209.

Torokhti, A., and Soto-Quiros, P. Generalized Brillinger-Like Transforms. IEEE Signal Processing Letters 23, 6 (2016), 843 – 847.

Vershynin, R. How close is the sample covariance matrix to the actual covariance matrix? J. Th. Prob. 25, 3 (2012), 655–686.

Wang, H. Rank constrained matrix best approximation problem. Applied Mathematics Letters 50 (2015), 98 – 104.

Wang, Q., and Jing, Y. New rank detection methods for reduced-rank mimo systems. EURASIP Journal on Wireless Comm. and Networking 2015, 1 (2015), 1–16.

Yang, R., and Berger, J. O. Estimation of a covariance matrix using the reference prior. The Annals of Statistics 22, 3 (1994), 1195–1211.

Zhu, X.-L., Zhang, X.-D., Ding, Z.-Z., and Jia, Y. Adaptive nonlinear PCA algorithms for blind source separation without prewhitening. IEEE Transactions on Circuits and Systems - I 53, 3 (2006), 745– 753.

Published

2024-03-11

How to Cite

[1]
A. Torokhti and J. P. Soto-Quiros, “Optimal modeling of nonlinear systems: Method of variable injections”, Proyecciones (Antofagasta, On line), vol. 43, no. 1, pp. 189-224, Mar. 2024.

Issue

Section

Artículos

Most read articles by the same author(s)