Frequentist parameter inference using profile likelihoods has received increased attention in the cosmology literature recently since it can give important complementary information to Bayesian credible intervals. In this talk, I will give an overview about Bayesian and frequentist parameter constraints and focus on the limitations of both approaches, highlighting the necessary assumptions behind them. This becomes particularly relevant in the context of cosmological tensions, which motivate new physics beyond LCDM, often introducing many additional parameters, which are not well constrained by the data and can lead to misleading results. I will illustrate this with several examples. In the context of the “Hubble tension”, I will discuss why frequentist and Bayesian approaches can give different answers to the question as to whether Early Dark Energy can resolve the “Hubble tension”. I will also touch on the apparent preference for negative neutrino masses in cosmology and the “$\sigma_8$ tension” in the context of the Effective Field Theory of large scale structure using both statistical approaches.