Textbook in PDF format Bayesian statistics and statistical practice have evolved over the years, driven by advancements in theory, methods, and computational tools. This book explores the intricate workflows of applied Bayesian statistics, aiming to uncover the tacit knowledge often overlooked in published papers and textbooks. By systematizing the process of Bayesian model development, the book seeks to improve applied analyses and inspire future innovations in theory, methods, and software. It emphasizes the importance of iterative model building, model checking, computational troubleshooting, and simulated-data experimentation, offering a comprehensive perspective on statistical analysis. How to use this book From Bayesian inference to Bayesian workflow Bayesian theory and Bayesian practice Why Bayes? Varieties of Bayesian theory There is no safe haven Convincing evidence Exercises Statistical modeling and workflow From Bayesian inference to Bayesian data analysis to Bayesian workflow “Workflow” and its relation to statistical theory and practice Four modeling scenarios Psychological struggles Exercises Computational tools Statistical programming environments Software for Bayesian inference (probabilistic programming) Additional workflow tools Notation in equations and code A simple example of probabilistic programming Exercises Introduction to workflow: Modeling performance on a multiple choice exam Fitting a simple model to data Expanding the model Starting over with a fully generative model Performing a simulation experiment to understand the model Breaking the model General lessons from the multiple-choice testing example Exercises Statistical workflow Building statistical models Choosing an initial model Relating a statistical model to subject-matter assumptions Expressing a Bayesian model using probability distributions A data model is not just a “likelihood” Generative and partially generative models Prior distribution Specifying the data model and specifying the prior Modeled and unmodeled data Prior predictive checking Tail behavior and how the prior and likelihood combine Exercises Using simulations to capture uncertainty and experiment with models Simulation to express modeling and inference Point estimates and uncertainties Designing and performing simulated-data experiments Simulating an underlying process, data collection, and inference Exercises Prediction, generalization, and causal inference Poststratification: Inferences about old averages and new scenarios Causal inference as generalization From inference to decision Different perspectives on statistical modeling and prediction Exercises Visualizing and checking fitted models Displaying high-dimensional inference Posterior predictive checking Cross validation checking Assessing the influence of individual data points and subsets of the data Influence of likelihood and prior information Exercises Comparing and improving models Big data need big models A topology of models Visualizing models in relation to each other Model selection using predictive checking and predictive performance Model selection and overfitting Stacking and predictive model averaging Model expansion: Predictive consistency and coherence Exercises Statistical inference and scientific inference Science as iteration between modeling, data collection, and data analysis A virtuous tangle Software assisted workflow The replication crisis and its connection to multiple levels of variation Simulated-data experimentation as virtual replication Exercises Computational workflow Fitting statistical models The typical set and the distribution of the log posterior density Initial values Adaptation and warmup How many parallel chains and how many iterations Effective sample size and Monte Carlo standard error How many digits to report based on posterior uncertainty Replicability of stochastic computation Exercises Diagnosing and fixing problems with fitting Fit fast, fail fast Discovering computational challenges by experimenting Failure modes and steps forward Using modeling ideas to address computing problems What to do about convergence problems Exercises Approximate algorithms and approximate models Approximations based on joint and conditional posterior modes Variational inference and Pathfinder Simulation-based and amortized inference Divide-and-conquer algorithms Fitting simpler models for computational purposes Application of workflow ideas to approximate computation Exercises Simulation-based calibration checking The process of simulation-based calibration checking Visual and numerical calibration diagnostics Incorporating calibration checking into workflow Exercises Statistical modeling as software development Writing code for single or multiple uses Version control smooths collaborations with others and with your past self Testing as you go Making it essentially reproducible Making it readable and maintainable Exercises Case studies Coding a series of models: Simulated data of movie ratings Model for two movies Extending the model to 퐽 movies Item-response model with parameters for raters and for movies General lessons from the movie ratings example Exercises Prior specification for regression models: Reanalysis of a sleep study Priors for linear models Priors for linear multilevel models Priors for generalized linear multilevel models Model checking General lessons from the prior-specification example Exercises Predictive model checking and comparison: Clinical trial Initial models Model extension and sensitivity analysis Inference for the treatment effect Sensitivity to prior and data model Comparing predictive performance General lessons from the clinical trial example Exercises Building up to a hierarchical model: Coronavirus testing Background Modeling a test with uncertain sensitivity and specificity Hierarchical model for varying testing conditions Prior sensitivity analysis Extensions of the model General lessons from the coronavirus testing example Exercises Using a fitted model for decision analysis: Classification competition A decision problem Data exploration Fitting a mixture model and computing predicted probabilities Solving the decision problem General lessons from the time series competition example Exercises Posterior predictive checking: Stochastic learning in dogs An early example of simulation-based model checking Fitting a series of models Further model checking Summarizing the inferences from a fitted hierarchical model Simulated-data experimentation General lessons from the dogs example Exercises Incremental development and testing: Black Cat adoptions The sample and the inferential goal Synthetic cat adoptions Modeling censored data Discrete outcomes instead of durations General lessons from the cat adoptions example Exercises Debugging a model: World Cup football Constructing a model for score differentials Model checking and exploration Finding the bug and checking the corrected model Assessing the role of the prior team rankings Expanding to a discrete-data model Evaluating models based on predictive accuracy General lessons from the World Cup example Exercises Leave-one-out cross validation model checking and comparison: Roaches Poisson model Negative binomial model Poisson model with varying intercepts Poisson model with varying intercepts and integrated LOO Zero-inflated negative binomial model General lessons from the cross validation example Exercises Model building and expansion: Golf putting First model: Logistic regression Modeling from first principles Testing the fitted model on new data A new model accounting for how hard the ball is hit Expanding the model by including a fudge factor General lessons from the golf example Exercises Model building with latent variables: Markov models for animal movement Time series of white shark movements Developing a hidden Markov model State decoding Model checking, expansion, and conclusions General lessons from the animal movement example Exercises Model building: Time-series decomposition for birthdays From 30 days to 366 Additive decompositions Gaussian process models Incremental model building General lessons from the birthdays example Exercises Models for regression coefficients and variable selection: Student grades Variable selection in general Models for regression coefficients Marginal posteriors Model checking Projection predictive variable selection General lessons from the variable selection example Exercises Sampling problems with latent variables: No vehicles in the park Fitting a model Understanding the fitted model Near-collinearity in the posterior distribution Fitting a Bayesian model How to choose parameterization and other tricks General lessons from the vehicles in the park example Exercises Challenge of multimodality: Differential equation for planetary motion Mechanistic model of motion Analyzing a simplified model Improving the inference Analyzing the full model General lessons from the planetary motion example Exercises Simulation-based calibration checking in model development workflow Target model Developing the mixture submodel Developing the logistic regression submodel Putting it together General lessons from the calibration example Exercises Appendices Statistical and computational workflow for Bayesians and non-Bayesians Use of prior information Combining information from multiple sources Regularization Using simulations to capture uncertainty Prediction, generalization, and causal inference Visualizing model checking and model fit Fitting a sequence of models rather than focusing on just one Initial values and tuning parameters Simulated-data experimentation Understanding methods by applying them to multiple problems Awareness of goals How to get the most out of Bayesian Data Analysis BDA3 part I: Fundamentals of Bayesian inference BDA3 part II: Fundamentals of Bayesian data analysis BDA3 part III: Advanced computation BDA3 parts IV and V: Regression models and nonlinear and nonparametric models BDA3 Appendixes References Author Index Subject Index