{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```{math}\n",
    "\n",
    "\\newcommand\\floor[1]{\\lfloor#1\\rfloor}\n",
    "\n",
    "\\newcommand{\\real}{\\mathbb{R}}\n",
    "\n",
    "% Numbers\n",
    "\\newcommand{\\vzero}{\\boldsymbol{0}}\n",
    "\\newcommand{\\vone}{\\boldsymbol{1}}\n",
    "\n",
    "% Greek https://www.latex-tutorial.com/symbols/greek-alphabet/\n",
    "\\newcommand{\\valpha}{\\boldsymbol{\\alpha}}\n",
    "\\newcommand{\\vbeta}{\\boldsymbol{\\beta}}\n",
    "\\newcommand{\\vchi}{\\boldsymbol{\\chi}}\n",
    "\\newcommand{\\vdelta}{\\boldsymbol{\\delta}}\n",
    "\\newcommand{\\vDelta}{\\boldsymbol{\\Delta}}\n",
    "\\newcommand{\\vepsilon}{\\boldsymbol{\\epsilon}}\n",
    "\\newcommand{\\vzeta}{\\boldsymbol{\\zeta}}\n",
    "\\newcommand{\\vXi}{\\boldsymbol{\\Xi}}\n",
    "\\newcommand{\\vell}{\\boldsymbol{\\ell}}\n",
    "\\newcommand{\\veta}{\\boldsymbol{\\eta}}\n",
    "%\\newcommand{\\vEta}{\\boldsymbol{\\Eta}}\n",
    "\\newcommand{\\vgamma}{\\boldsymbol{\\gamma}}\n",
    "\\newcommand{\\vGamma}{\\boldsymbol{\\Gamma}}\n",
    "\\newcommand{\\vmu}{\\boldsymbol{\\mu}}\n",
    "\\newcommand{\\vmut}{\\boldsymbol{\\tilde{\\mu}}}\n",
    "\\newcommand{\\vnu}{\\boldsymbol{\\nu}}\n",
    "\\newcommand{\\vkappa}{\\boldsymbol{\\kappa}}\n",
    "\\newcommand{\\vlambda}{\\boldsymbol{\\lambda}}\n",
    "\\newcommand{\\vLambda}{\\boldsymbol{\\Lambda}}\n",
    "\\newcommand{\\vLambdaBar}{\\overline{\\vLambda}}\n",
    "%\\newcommand{\\vnu}{\\boldsymbol{\\nu}}\n",
    "\\newcommand{\\vomega}{\\boldsymbol{\\omega}}\n",
    "\\newcommand{\\vOmega}{\\boldsymbol{\\Omega}}\n",
    "\\newcommand{\\vphi}{\\boldsymbol{\\phi}}\n",
    "\\newcommand{\\vvarphi}{\\boldsymbol{\\varphi}}\n",
    "\\newcommand{\\vPhi}{\\boldsymbol{\\Phi}}\n",
    "\\newcommand{\\vpi}{\\boldsymbol{\\pi}}\n",
    "\\newcommand{\\vPi}{\\boldsymbol{\\Pi}}\n",
    "\\newcommand{\\vpsi}{\\boldsymbol{\\psi}}\n",
    "\\newcommand{\\vPsi}{\\boldsymbol{\\Psi}}\n",
    "\\newcommand{\\vrho}{\\boldsymbol{\\rho}}\n",
    "\\newcommand{\\vtheta}{\\boldsymbol{\\theta}}\n",
    "\\newcommand{\\vthetat}{\\boldsymbol{\\tilde{\\theta}}}\n",
    "\\newcommand{\\vTheta}{\\boldsymbol{\\Theta}}\n",
    "\\newcommand{\\vsigma}{\\boldsymbol{\\sigma}}\n",
    "\\newcommand{\\vSigma}{\\boldsymbol{\\Sigma}}\n",
    "\\newcommand{\\vSigmat}{\\boldsymbol{\\tilde{\\Sigma}}}\n",
    "\\newcommand{\\vsigmoid}{\\vsigma}\n",
    "\\newcommand{\\vtau}{\\boldsymbol{\\tau}}\n",
    "\\newcommand{\\vxi}{\\boldsymbol{\\xi}}\n",
    "\n",
    "\n",
    "% Lower Roman (Vectors)\n",
    "\\newcommand{\\va}{\\mathbf{a}}\n",
    "\\newcommand{\\vb}{\\mathbf{b}}\n",
    "\\newcommand{\\vBt}{\\mathbf{\\tilde{B}}}\n",
    "\\newcommand{\\vc}{\\mathbf{c}}\n",
    "\\newcommand{\\vct}{\\mathbf{\\tilde{c}}}\n",
    "\\newcommand{\\vd}{\\mathbf{d}}\n",
    "\\newcommand{\\ve}{\\mathbf{e}}\n",
    "\\newcommand{\\vf}{\\mathbf{f}}\n",
    "\\newcommand{\\vg}{\\mathbf{g}}\n",
    "\\newcommand{\\vh}{\\mathbf{h}}\n",
    "%\\newcommand{\\myvh}{\\mathbf{h}}\n",
    "\\newcommand{\\vi}{\\mathbf{i}}\n",
    "\\newcommand{\\vj}{\\mathbf{j}}\n",
    "\\newcommand{\\vk}{\\mathbf{k}}\n",
    "\\newcommand{\\vl}{\\mathbf{l}}\n",
    "\\newcommand{\\vm}{\\mathbf{m}}\n",
    "\\newcommand{\\vn}{\\mathbf{n}}\n",
    "\\newcommand{\\vo}{\\mathbf{o}}\n",
    "\\newcommand{\\vp}{\\mathbf{p}}\n",
    "\\newcommand{\\vq}{\\mathbf{q}}\n",
    "\\newcommand{\\vr}{\\mathbf{r}}\n",
    "\\newcommand{\\vs}{\\mathbf{s}}\n",
    "\\newcommand{\\vt}{\\mathbf{t}}\n",
    "\\newcommand{\\vu}{\\mathbf{u}}\n",
    "\\newcommand{\\vv}{\\mathbf{v}}\n",
    "\\newcommand{\\vw}{\\mathbf{w}}\n",
    "\\newcommand{\\vws}{\\vw_s}\n",
    "\\newcommand{\\vwt}{\\mathbf{\\tilde{w}}}\n",
    "\\newcommand{\\vWt}{\\mathbf{\\tilde{W}}}\n",
    "\\newcommand{\\vwh}{\\hat{\\vw}}\n",
    "\\newcommand{\\vx}{\\mathbf{x}}\n",
    "%\\newcommand{\\vx}{\\mathbf{x}}\n",
    "\\newcommand{\\vxt}{\\mathbf{\\tilde{x}}}\n",
    "\\newcommand{\\vy}{\\mathbf{y}}\n",
    "\\newcommand{\\vyt}{\\mathbf{\\tilde{y}}}\n",
    "\\newcommand{\\vz}{\\mathbf{z}}\n",
    "%\\newcommand{\\vzt}{\\mathbf{\\tilde{z}}}\n",
    "\n",
    "\n",
    "% Upper Roman (Matrices)\n",
    "\\newcommand{\\vA}{\\mathbf{A}}\n",
    "\\newcommand{\\vB}{\\mathbf{B}}\n",
    "\\newcommand{\\vC}{\\mathbf{C}}\n",
    "\\newcommand{\\vD}{\\mathbf{D}}\n",
    "\\newcommand{\\vE}{\\mathbf{E}}\n",
    "\\newcommand{\\vF}{\\mathbf{F}}\n",
    "\\newcommand{\\vG}{\\mathbf{G}}\n",
    "\\newcommand{\\vH}{\\mathbf{H}}\n",
    "\\newcommand{\\vI}{\\mathbf{I}}\n",
    "\\newcommand{\\vJ}{\\mathbf{J}}\n",
    "\\newcommand{\\vK}{\\mathbf{K}}\n",
    "\\newcommand{\\vL}{\\mathbf{L}}\n",
    "\\newcommand{\\vM}{\\mathbf{M}}\n",
    "\\newcommand{\\vMt}{\\mathbf{\\tilde{M}}}\n",
    "\\newcommand{\\vN}{\\mathbf{N}}\n",
    "\\newcommand{\\vO}{\\mathbf{O}}\n",
    "\\newcommand{\\vP}{\\mathbf{P}}\n",
    "\\newcommand{\\vQ}{\\mathbf{Q}}\n",
    "\\newcommand{\\vR}{\\mathbf{R}}\n",
    "\\newcommand{\\vS}{\\mathbf{S}}\n",
    "\\newcommand{\\vT}{\\mathbf{T}}\n",
    "\\newcommand{\\vU}{\\mathbf{U}}\n",
    "\\newcommand{\\vV}{\\mathbf{V}}\n",
    "\\newcommand{\\vW}{\\mathbf{W}}\n",
    "\\newcommand{\\vX}{\\mathbf{X}}\n",
    "%\\newcommand{\\vXs}{\\vX_{\\vs}}\n",
    "\\newcommand{\\vXs}{\\vX_{s}}\n",
    "\\newcommand{\\vXt}{\\mathbf{\\tilde{X}}}\n",
    "\\newcommand{\\vY}{\\mathbf{Y}}\n",
    "\\newcommand{\\vZ}{\\mathbf{Z}}\n",
    "\\newcommand{\\vZt}{\\mathbf{\\tilde{Z}}}\n",
    "\\newcommand{\\vzt}{\\mathbf{\\tilde{z}}}\n",
    "\n",
    "\n",
    "%%%%\n",
    "\\newcommand{\\hidden}{\\vz}\n",
    "\\newcommand{\\hid}{\\hidden}\n",
    "\\newcommand{\\observed}{\\vy}\n",
    "\\newcommand{\\obs}{\\observed}\n",
    "\\newcommand{\\inputs}{\\vu}\n",
    "\\newcommand{\\input}{\\inputs}\n",
    "\n",
    "\\newcommand{\\hmmTrans}{\\vA}\n",
    "\\newcommand{\\hmmObs}{\\vB}\n",
    "\\newcommand{\\hmmInit}{\\vpi}\n",
    "\\newcommand{\\hmmhid}{\\hidden}\n",
    "\\newcommand{\\hmmobs}{\\obs}\n",
    "\n",
    "\\newcommand{\\ldsDyn}{\\vA}\n",
    "\\newcommand{\\ldsObs}{\\vC}\n",
    "\\newcommand{\\ldsDynIn}{\\vB}\n",
    "\\newcommand{\\ldsObsIn}{\\vD}\n",
    "\\newcommand{\\ldsDynNoise}{\\vQ}\n",
    "\\newcommand{\\ldsObsNoise}{\\vR}\n",
    "\n",
    "\\newcommand{\\ssmDynFn}{f}\n",
    "\\newcommand{\\ssmObsFn}{h}\n",
    "\n",
    "\n",
    "%%%\n",
    "\\newcommand{\\gauss}{\\mathcal{N}}\n",
    "\n",
    "\\newcommand{\\diag}{\\mathrm{diag}}\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(sec:ssm-intro)=\n",
    "# What are State Space Models?\n",
    "\n",
    "\n",
    "A state space model or SSM\n",
    "is a partially observed Markov model,\n",
    "in which the hidden state,  $\\hidden_t$,\n",
    "evolves over time according to a Markov process,\n",
    "possibly conditional on external inputs or controls $\\input_t$,\n",
    "and each hidden state generates some\n",
    "observations $\\obs_t$ at each time step.\n",
    "(In this book, we mostly focus on discrete time systems,\n",
    "although  we consider the continuous-time case in  XXX.)\n",
    "We get to see the observations, but not the hidden state.\n",
    "Our main goal is to infer the hidden state given the observations.\n",
    "However, we can also use the model to predict future observations,\n",
    "by first predicting future hidden states, and then predicting\n",
    "what observations they might generate.\n",
    "By using  a hidden state $\\hidden_t$\n",
    "to represent the past observations, $\\obs_{1:t-1}$,\n",
    "the  model can have ``infinite'' memory,\n",
    "unlike a standard Markov model.\n",
    "\n",
    "```{figure} /figures/SSM-AR-inputs.png\n",
    ":height: 300px\n",
    ":name: fig:ssm-ar\n",
    "\n",
    "Illustration of an SSM as a graphical model.\n",
    "```\n",
    "\n",
    "\n",
    "Formally we can define an SSM \n",
    "as the following joint distribution:\n",
    "```{math}\n",
    ":label: eq:SSM-ar\n",
    "p(\\hmmobs_{1:T},\\hmmhid_{1:T}|\\inputs_{1:T})\n",
    " = \\left[ p(\\hmmhid_1|\\inputs_1) \\prod_{t=2}^{T}\n",
    " p(\\hmmhid_t|\\hmmhid_{t-1},\\inputs_t) \\right]\n",
    " \\left[ \\prod_{t=1}^T p(\\hmmobs_t|\\hmmhid_t, \\inputs_t, \\hmmobs_{t-1}) \\right]\n",
    "```\n",
    "where $p(\\hmmhid_t|\\hmmhid_{t-1},\\inputs_t)$ is the\n",
    "transition model,\n",
    "$p(\\hmmobs_t|\\hmmhid_t, \\inputs_t, \\hmmobs_{t-1})$ is the\n",
    "observation model,\n",
    "and $\\inputs_{t}$ is an optional input or action.\n",
    "See {numref}`fig:ssm-ar` \n",
    "for an illustration of the corresponding graphical model.\n",
    "\n",
    "\n",
    "We often consider a simpler setting in which the\n",
    " observations are conditionally independent of each other\n",
    "(rather than having Markovian dependencies) given the hidden state.\n",
    "In this case the joint simplifies to \n",
    "```{math}\n",
    ":label: eq:SSM-input\n",
    "p(\\hmmobs_{1:T},\\hmmhid_{1:T}|\\inputs_{1:T})\n",
    " = \\left[ p(\\hmmhid_1|\\inputs_1) \\prod_{t=2}^{T}\n",
    " p(\\hmmhid_t|\\hmmhid_{t-1},\\inputs_t) \\right]\n",
    " \\left[ \\prod_{t=1}^T p(\\hmmobs_t|\\hmmhid_t, \\inputs_t) \\right]\n",
    "```\n",
    "Sometimes there are no external inputs, so the model further\n",
    "simplifies to the following unconditional generative model: \n",
    "```{math}\n",
    ":label: eq:SSM-no-input\n",
    "p(\\hmmobs_{1:T},\\hmmhid_{1:T})\n",
    " = \\left[ p(\\hmmhid_1) \\prod_{t=2}^{T}\n",
    " p(\\hmmhid_t|\\hmmhid_{t-1}) \\right]\n",
    " \\left[ \\prod_{t=1}^T p(\\hmmobs_t|\\hmmhid_t) \\right]\n",
    "```\n",
    "See {numref}`ssm-simplified` \n",
    "for an illustration of the corresponding graphical model.\n",
    "\n",
    "\n",
    "```{figure} /figures/SSM-simplified.png\n",
    ":scale: 100%\n",
    ":name: ssm-simplified\n",
    "\n",
    "Illustration of a simplified SSM.\n",
    "```\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}