The Indivisible Stochastic Picture of Quantum Mechanics
Jacob Barandes has a formulation of quantum mechanics which Iâve been thinking about. He says we should forget about wavefunctions and Hilbert spaces, and that a neglected corner of ordinary probability theory is fully equivalent to the full quantum picture. Watch his lecture on youtube if you want to listen to him explain it (heâs a good speaker, I recommend it), or read his paper. Iâll go over the basic ideas, and work through my thoughts on the theory.
I think itâs fair to say that at a first pass, quantum mechanics appears to most students as a rather artificial theory: While the mathematics involved is on its own not difficult, it is abstract, the physical interpretation of it is murky, and it is a hard pill to swallow that gadgets like complex Hilbert spaces and their tensor products which seem like very human constructions, are really necessary to describe nature. In fact, even the very first students of quantum mechanics struggled with this problem, with von Neumann apparently proclaiming âI donât believe in Hilbert spaces anymoreâ. We might ask then whether we can start with some natural set of assumptions that donât include Hilbert spaces or any quantum weirdness and show that they lead to a theory that is nevertheless equivalent to quantum mechanics.
Barandes attempts this, but he is not the first to do so. In fact, this is precisely the mission statement of the so-called Quantum Reconstruction program. This program has produced a handful of related ways to construct quantum mechanics from natural principles. See the introduction of this paper for an overview. Very roughly speaking, in these frameworks quantum mechanics distinguishes itself from a classical deterministic theory in that even maximal knowledge of a system does not allow perfect prediction of all possible measurements; And further distinguishes itself from an ordinary stochastic theory only by the requirement that information is preserved/all processes are continuously reversible in isolated systems. For example: In the paper I just linked, which adopts an information-theoretic viewpoint, both of these two statements are contained in what the authors call the âpurification principleâ, without which their axioms would be compatible with a classical theory.
Stochastic Systems
Barandesâ theory is different in that it doesnât, in its basic formulation, make recourse to any generalization or modification of probability theory. Quantum systems are described as âboringâ stochastic systems in his framework, albeit of a somewhat unusual sort. Barandes emphasizes the boringness as a feature.
Hereâs how: Given a finite set $S$ (the state space), a âgeneralized stochastic systemâ consists simply of a time-dependent matrix $\Gamma(t)$ of size $|S|\times|S|$ whose entry $\Gamma_{ij}(t)$ tells us the probability of finding the system in state $j$ at time $t$ when initialized in state $i$ at time $0$. The entries, being probabilities, are between 0 and 1 and $\Gamma$ is normalized in the way youâd expect: the columns sum to 1 (we say that $\Gamma(t)$ a stochastic matrix) and $\Gamma(0)=\mathbb 1$.
Given a generalized stochastic system (GSS), if we additionally specify an initial probability distribution $\mathbf{p}(0)$ on $S$, to be read as âthe system is in state $j$ with probability $p_{j}(0)$â, then we get an evolving probability vector $\mathbf{p}(t):=\Gamma(t)\mathbf{p}(0)$.
Crucially, $\Gamma(t)$ is not required to factorize as $\Gamma_{tâ}(t-tâ)\Gamma(tâ)$ for smaller times $tâ<t$ and some stochastic matrix $\Gamma_{tâ}(t-tâ)$. This property would be called Markovianity or divisibility or memorylessness: It would mean that we could think of the time-evolution of the state at any time as only depending on the present state and not its history (but see the warning in the next paragraph). We do not assume this, but we leave open the possibility that such a factorisation might happen to be true for special times $tâ$ and all $t\geq tâ$, in which case we call $tâ$ a division event. Usually $\Gamma(t)$ will have full rank, in which case division events can be identified by the property that $\Gamma(t)\Gamma^{-1}(tâ)$ is a stochastic matrix for all $t\geq tâ.$
Let me give one warning that is surely quite obvious to anyone knowledgeable about probability theory (which set doesnât include myself, so this point confused me for a bit): The evolving state $\mathbf{p}(t)$ falls severely short of uniquely defining whatâs usually meant by a stochastic process in probability theory! Whatâs missing is all knowledge about contingent probabilities. Basically, a genuine stochastic process would tell us how likely any given trajectory through state-space is but here we only know the pointwise probability distributions at all times. A stochastic process has its own definition of Markovianity, directly requiring that evolution only depends on present state, not past trajectory. And if $\Gamma(t)$ does happen to satisfy the divisiblity criterion above at all times, then there is indeed a unique Markovian stochastic process for any initial $\mathbf{p}(0)$ that matches all $\mathbf{p}(t)$ in its probability distributions, but there are also many non-Markovian stochastic processes that do the same! To make this concrete, consider the transition matrix $$
\Gamma(t)=\begin{pmatrix} e^{-\lambda t} & 0\\ 1-e^{-\lambda t} & 1 \end{pmatrix}.
$$This matrix clearly satisifies the divisibility condition and we can think of it as belonging to a Markovian decay process between two states, $B$ and $A$, with a fixed half-life. Then the trajectories of this process either start in state $B$ and stay there; or they start in state $A$, decay to $B$ at some point, then stay there. But we can also imagine a different stochastic process with the same transition matrix, whose trajectories that start in $B$ also stay in $B$, but if starting in $A$ are allowed to randomly switch any number of times, with transition probabilites tuned in just such a way that the overall probability vector always precisely conforms to $\Gamma(t)\mathbf{p}(0)$. In the most extreme case, we can imagine the state of the system, if starting out in $A$, to be given at any time $t$ by a completely independent random variable with distribution $(e^{-\lambda t},1-e^{-\lambda t})$! This is clearly non-Markovian, as the system remembers its initial state in order to know whether itâs allowed to jump from $B$ to $A$.
The reason Iâm saying all this is because Barandes says we should think of a GSS as somehow like a stochastic process, in that when we use it to describe actual physical systems we should think of the system as always being in one definite state, i.e. describing a trajectory through state-space. And he seems to be saying that at division events we really should throw away past information, although this is not inevitable merely from the mathematics of the $\Gamma$-matrices, as weâve just seen.
What happens away from division events, i.e. what sort of trajectories are we imagining? I have no clue. Barandes either doesnât say, or I didnât catch it. Maybe we just stay agnostic about it. Weâll come back to this later once weâve gained an understanding of how this all relates to quantum mechanics. In some sense, what trajectory the system âreallyâ takes and even whether we should actually think of a real state at all, are questions that straddle the line between the formulation of the model and its interpretation. For now weâll focus on what the $\Gamma$-matrices themselves can tell us.
Describing Quantum Systems
So we want to describe quantum systems in this language. By âquantum systemâ I mean not just a unitarily evolving state of an isolated system, but more generally any evolving mixed state of a possibly open system. We assume given a Hilbert space with an orthonormal basis $S$ and a notion of time evolution for density matrices $\rho(t)\to \rho(tâ)$ underlying some standard technical restrictions (itâs a completely positive trace-preserving linear operator; See the paper if you want all the details). We will make a GSS whose states are the same set $S$ which serves as our basis. We are obliged to obey one restriction: We must start out at time $t=0$ with a diagonal density matrix (no superposition), which we can assume to have real entries. The entries then are precisely the probabilities $p_{j}(0)$. Hereâs how to get the transition matrix $\Gamma(t)$: The entry $\Gamma_{ij}(t)$ is simply the probability of measuring state $j$ at time $t$ after starting out in state $i$ at time $0$ (if we were to measure at that time). So take the diagonal density matrix $P_{j}$ which has only one nonzero entry at place $j$, evolve it to time $t$ to get some new density matrix $\rho$, then take the trace $\mathrm{tr}( P_{i}\rho)$ to get $\Gamma_{ij}(t)$.
An isolated quantum system evolves unitarily. We define a unistochastic system to be a GSS whose transition matrices are unistochastic, which for a matrix means it can be obtained from a unitary complex matrix by taking absolute-value squared entrywise. (This is how Barandes defines a unistochastic system. In application though, he supposes a differentiably evolving unitary matrix $U(t)$ which gives $\Gamma(t)$ by taking entrywise norms. This is a priori stronger than just asking for $\Gamma(t)$ to be unistochastic at each $t$ separately, and should maybe be taken as the definition. I donât know whether the two are equivalent, i.e. whether every differentiable family of unistochastic matrices can be lifted). Isolated quantum systems are described by unistochastic systems, and any unistochastic system has a Hilbert space description. It is then possible to interpret all the distinctive features of quantum mechanics: interference, entanglement, non-commuting measurements, uncertainty and so on as features of the probability theory of unistochastic systems.
Now this might seem like a bit of a let-down at first glance: Isnât it the case that all the quantum weirdness that we wanted to explain is now just hidden behind the equally arcane condition of unistochasticity? Certainly this doesnât seem like a condition one would be able to come up with without already knowing about the Hilbert space formulation of QM. To convince us of the naturality of unistochasticity, Barandes proves the Stochastic-Quantum Correspondence: Every GSS can be embedded into a unistochastic system. That is, any GSS with state space $S$ can be seen as a subsystem of some larger unitastochastic system with state space $E\times S$, and the dynamics of $S$ follow from those of $E\times S$ by marginalizing over the additional states (which is the same thing as âtracing over the environmentâ in physics parlance). The proposed picture is then: Physical systems are a priori described by some GSS (which should not surprise us, since itâs a very general kind of description), and can therefore equivalently be described unistochastically as a mathematical convenience.
Iâm not sure I completely buy this argument. Letâs return to the example from earlier of a Markovian system describing decay with a fixed half-life, i.e. the transition matrix $$
\Gamma(t)=\begin{pmatrix} e^{-\lambda t} & 0\\ 1-e^{-\lambda t} & 1 \end{pmatrix}.
$$ This is a perfectly fine GSS, and we could imagine a world containing such a decay process at a fundamental level. How does the unistochastic/quantum description for this look like? One thing you might notice immediately is that it cannot have a constant Hamiltonian, because otherwise the system would have to return to its initial state after some time due to Poincare recurrence. So we need to add some extra states and have a Hamiltonian varying in such a way as to conspire to hide them from us at all times. One possibility is a matrix of the form: $$U(t)=\begin{pmatrix}
e^{-\lambda t/2} & \sqrt {1-e^{-\lambda t}} & 0 & 0\\
-\sqrt {1-e^{-\lambda t}} & e^{-\lambda t/2} & 0 & 0 \\
0 & 0 & 1 & 0 \\
0 & 0 & 0 & 1
\end{pmatrix},$$which is a unitary (even orthogonal) matrix. The original states $A$ & $B$ have split up into $A_{1}$, $A_{2}$, $B_{1}$, $B_{2}$. Of these, $A_{2}$ and $B_{2}$ are inert and donât change their state, while $A_{1}$ and $B_{1}$ rotate into each other. If we embed the initial conditions via $A\to A_{1}$ and $B\to B_{2},$ we recover the original system by marginalizing over the indices. But this is a terrible way to describe the system! While this is a family of unitary matrices, itâs not generated by a Hamiltonian, even a time-varying one, because the derivative at $t=0$ explodes (I think thatâs a negative answer to my previous question about lifting families). And we only get the decay behavior we started with if we take these particular initial conditions with $A_{2}$ and $B_{1}$ vanishing, and lock the time-evolution of the Hamiltonian (away from $t=0$, where itâs not defined) to this one predetermined path. A very reasonable GSS with a constant Markovian generator has turned into a rather unreasonable beast in the unitary description. Needless to say, this is not the sort of thing we see in nature! We generally see constant Hamilitonians and unitary evolution on a state space that is fully realizable, which I feel still requires explanation.
Another idea to justify unistochastic systems might be to take a page from the quantum reconstructions program which exposes quantum mechanics as the result of reconciling the two seemingly contradictory properties of stochasticity and information preservation. Unfortunately, the definition of a GSS seems to be too general to permit a characterisation of unistochastic systems as the âinformation-preservingâ GSSes in any reasonable sense. For instance, no matter how far from unistochastic $\Gamma(t)$ is, it can always retrace its path to return to the identity at $2t$, which surely would be sufficient for any reasonable definition of information preservation. A GSS is just a family of stochastic matrices, there are no constraints.
Once weâve accepted the idea of unistochastic systems, we can analyze measurement in just the same way one does in ordinary QM. That is, by considering a composite unistochastic system of âenvironmentâ and âsubject systemâ, and tracing out the environment to get a non-unistochastic description of the subject system. For an idealized measurement in basis $S$, we let the interaction between the two subsystems be such that it perfectly correlates their states at some intermediate time, and let the systems evolve independently thereafter. Barandes shows that this results in a division event in the subject system, which is how wavefunction collapse manifests itself here.
Barandes calls operators diagonal in $S$ âbe-ablesâ, and considers them genuine properties of the systemâs state, while non-diagonal operators are âemerge-ablesâ and signify emergent patterns in the systemâs dynamics. One can do a similar calculation of their measurement processes. These do not immediately entail a division event, but if one further separates from the environment system a âmeasuring systemâ, then they induce a division event in that measuring system.
Interpretation
The biggest selling point of this formulation is supposed to be the ease of interpretation. The system is always in a definite state, which changes stochastically and itâs only the indivisibility of the process which gives rise to quantum phenomena. Letâs process what this implies.
First of all, we should note that the entire mathematics of the theory is in the $\Gamma$ matrices. Given some input state, the output of a GSS is an evolving probability vector, nothing more. The âactual stateâ is a pointer on top of that, pointing at one specific element of the state space at all times but otherwise affecting nothing at all. We also have no information on how this pointer changes over time, except that we know the a priori probability with which to expect it in a given state.
Even on a purely philosophical level, I have a hard time convincing myself that this is a meaningful concept. I donât know how to assign a truth value to the statement that this pointer exists. A ball in a Pachinko machine can be modeled by a stochastic process (non-Markovian if the machine contains any gadgets that interact with, and are affected by, the ballâs movement), and here we can meaningfully say that it has a definite state if we either sample a trajectory (which is a well-defined thing with genuine stochastic processes) or actually throw a ball into the real-world machine. Neither of these things work in the case of modeling the real world by a GSS. The first option doesnât apply and in the second the state comes from the fact that the machine is real separately from the abstraction of the stochastic process. To apply it in our case, we would need to posit a mechanism by which quantum mechanics itself arises from some other thing which is independently real. Even if such a thing existed (whatever that means), we in principle couldnât detect it and have no reason to assume it prefers this particular formulation.
Supposing the âthis is the real stateâ pointer does exist in some sense, how might it change over time? Take a system consisting just of a single qubit with a nondiagonal Hamiltonian such that the state evolves (in conventional formulation) like $|0\rangle \to \frac{1}{\sqrt{2}}(|0\rangle +|1\rangle)\to|1\rangle$. At the intermediate time (call it $t_{i}$), we are in the midst of an indivisible stochastic process and the probability vector is ${0.5}$ in both entries. Nevertheless, in the stochastic interpretation we suppose that the system actually is in one of the two states. Say itâs in state $0$. Does this increase the likelihood that at time $t_{i}+\epsilon$ we will also be in state $0$? We donât know at all. Perhaps the simplest thing is to assume that it doesnât and that the pointer constantly switches between the two at a high frequency such that the overall time spent in each state is given by the output probability vector. This would effectively reproduce the many-worlds interpretation, though perhaps with more of a justification for the Born rule (though that can be argued about).
We might look to division events for an opportunity to lock in a genuine choice in the probability tree (as is suggested by the name). This doesnât hold up except as an approximation, though: As far as we can tell, any division event in the real world ultimately comes from being a subsystem of a larger unitary system with a constant Hamiltonian and meaningful space of states. In this larger system, no division event is visible and we should assume that the state can jump between different branches. And of course, the idealized conditions leading to a division event arenât present in the real world in the first place: All apparent division events are really only approximate and will break down over long enough timescales. This makes it inevitable that for instance any computer simulation of the world must simulate all branches, no matter how decohered, or be inaccurate after a sufficiently long time (potentially REALLY long like Poincare recurrence time long). Of course this is a statement thatâs not specific to this formulation of QM, but I think it kind of highlights why I find the idea of the reality pointer sitting on top of it all so strange.
Another point of friction is the unfounded breaking of symmetries intrinsic to this framework. We declare that $S$ is somehow ontologically more significant than other orthonormal bases of the Hilbert space, despite there being in many cases a perfect symmetry between many or all possible choices. Describing the spin state of particles, are we really to assume there is an unambigiously preferred axis along which to measure it, defined at every point in space? And of course thereâs time translation symmetry: What we usually mean by this corresponds to the statement that the unitary generator is constant in time, but this is a far less obvious statement in the stochastic view. Why a priori should âphysics is the same at every point in timeâ correspond to a constant unistochastic generator, if we take the more general concept of a GSS to be fundamental? I outlined above why I donât find the stochastic-quantum theorem alone a convincing answer here.
Anyway, my conclusion is that in the end these questions concerning interpretation and metaphysics are still just as muddy as ever. Itâs of course very possible that Iâve just not grokked the idea and others will have a better time with it!
On the other hand, the precise part of the theory, the presentation of QM in a very unfamiliar form, is enlightening and made me carefully think through some aspects of both ordinary probability theory and quantum mechanics that I was previously unfamiliar with. The mathematical details of the correspondence Iâve mostly omitted here, but they are interesting to go through. And if you like this sort of thing, also check out the quantum reconstructions papers that Iâve mentioned a few times now, especially the one I linked in the beginning and Quantum Theory From Five Reasonable Axioms, which seems to have kicked off the program.
I'm listening to Barandes on TOE for the 2nd time cause it seems like he might really be on to something.
Now I'm reading your rebuttal and am even more excited that he could be completely wrong!
I only have like 15% comprehension here, but it's fascinating nonetheless.
đľď¸ââď¸
arxiv.org/abs/1011.6451
but I'm unsure what to make of information as being in the basic ontology. It just begs the question as what that information is about. I like Barandes' minimalistic formulation in terms of a stochastic process.
Does it lead merely to a new/different interpretation, or does it have potential to become something more?
đ
Of course, one then wonders about the status of the underlying stochastic process.
Your point about how the Markovian GSS can give rise to non-Markovian trajectories is really interesting. If we just say that those non-Markovian trajectories aren't allowed is this a problem for the formalism?
Well the formalism stays agnostic about the real trajectories, so upon encountering a division event one is free to imagine a 'real' loss of history-dependence. My point when I mention this in the post is just to point out that in doing so one is assuming something extra.