Starting with Bayes' theorem that "we all agree on", I will argue that the step towards Bayesian inference seems rather small. I will give some simple examples of advantages of Bayesian over classical inference: 1. automatic inclusion of known constraints and 2. straightforward inference for functions of parameters.
Another point I will make is that posterior distributions (of unknown parameters) are often equivalent to sampling distributions (of estimators) required for classical inference. However, when the latter are difficult/impossible to obtain, and Normal approximations are applied, the former tend to be clearly preferable for inference.