Probability theory often follows a familiar process of modelling, observation and inference and plays an important role in solving problems in engineering and computer science. To use probability theory, you usually start by creating a mathematical model that describes the behaviour of a system.
This model is typically based on equations that have a physical meaning, like the dynamics of a robotic vehicle. A subset of the variables is observed in the model and used to learn about other variables that are related but not directly observable.
A good example of this is the Kalman filter, which is used to estimate the state of a system over time based on noisy measurements. When working with continuous variables like those used in the Kalman filter, it’s often convenient to make the Gaussian assumption for all prior and conditional distributions.
In his doctoral research, Dr Johannes Schoeman, under the supervision of Prof Johan du Preez and Dr Corne van Daalen, at the Department of Electrical and Electronic Engineering, Stellenbosch University, proposed a parametrised factor that enables accurate and automatic inference on Gaussian networks in such degenerate settings at little additional computational cost.
Motivation for the research
When working with Gaussian models, there is a constraint that the covariance matrix must be positive definite. If the covariance matrix is only positive semi-definite, the likelihood of the distribution and precision matrix won’t exist, causing issues with basic operations like multiplication and conditioning.
Degenerate cases, which correspond to perfect correlation between random variables, result in densities with support only on lower-dimensional manifolds. The cause of linear dependencies between random variables can be model parameters or machine precision limitations.
One solution to handle degeneracies is redefining the problem to manually remove redundant variables. However, this can be tedious, especially in time-dependent models where degeneracies arise unpredictably. An alternative approach that can automatically handle degenerate cases is preferable.
Research aims and objectives
With this research, Dr Schoeman aims to:
- propose a representation for Gaussian factors that can express possible degeneracies without suffering from over-parametrisation.
- derive the appropriate statistical operations such that the capabilities of non-degenerate factors are preserved.
- implement the developed solution and showcase its advantages at the hand of a representative example.
Final thoughts
If successful, this will ultimately enable numerically stable inference (due to well-conditioned matrices) in degenerate or near-degenerate settings without resorting to unnecessary approximations.To read the complete research, including a discussion of possible avenues for future research, visit: https://scholar.sun.ac.za/handle/10019.1/123876