Discussing the article: "Fundamentals of Bayesian Inference in Discrete and Continuous Settings: From Theory to Practical Model Implementation"

[MetaQuotes]

Check out the new article: Fundamentals of Bayesian Inference in Discrete and Continuous Settings: From Theory to Practical Model Implementation.

The article discusses the fundamentals of Bayesian statistics in both discrete and continuous settings. We will progress from classical Bayes’ theorem and simple examples involving coin flips to conjugate distributions and dynamic Bayesian updating, which enables real-time analysis of market quotes. Using the beta-binomial model as an example, a simple change-point detection indicator has been implemented to help identify shifts in market regimes.

The subject of probability theory is the modeling of data D based on known parameters θ using the probability density function p(D|θ). Mathematical statistics, in turn, solves the inverse problem: estimating unknown parameters θ based on the available data D.

There are two main approaches to solving this problem — the frequentist approach and the Bayesian approach. And while we are all familiar to some extent with the frequentist approach, the Bayesian perspective has received relatively little attention. At the same time, today's leading experts in machine learning embrace the Bayesian view of statistics.

In this article, we will try to understand the basics of this method. To keep the discussion manageable, we will deliberately avoid conditional models of the form P(y∣X, θ), where the value of the target variable y depends on a set of input features X (as, for example, in Bayesian logistic regression). To understand Bayesian philosophy, it is best to start with simpler models—that is, with the classic problem of parameter estimation.

We will follow a "from theory to practice" approach: abstract set-theoretical arguments will be immediately illustrated with concrete examples. This will clearly demonstrate that working with Bayesian models often requires nothing more complex than the basic operations of adding and multiplying probabilities.

Author: Evgeniy Chernish