Geometry Rescaling

One of the major issues with the stability of the timemachine is whether or not your Hessians are PSD. A non-PSD Hessian induces a transition matrix with a spectral radius p(r) > 1, causing a divergent trajectory.

Recall Brownian dynamics:

x_t = x_{t-1} - k G(x, \theta) + noise

Assuming a zero mixed partial, we can differentiate with respect to \theta

x'_t = x'_{t-1} - k H(x_{t-1}) x'_{t-1}

x'_t = (I - k H) x'_{t-1}

G and H are the Gradients and Hessians, respectively. Since H is rarely PSD in Cartesian coordinates, how can we fix this?

By playing dirty: consider an alternative dynamical equation where we shrink the geometry by a small amount:

x_t = (1 - k c) x_{t-1} - k G(x, \theta) + noise

Which has derivative:

x'_t = (1 - k c) x'_{t-1} - k H(x_{t-1}) x'_{t-1}

x'_t = (I - k c I - k H) x'_{t-1}

x'_t = (I - k (c I + H) ) x'_{t-1}

Which is equivalent to adding a diagonal to the Hessian. So long as c is bigger than the sum of each row/column, our Hessian is now PSD.

The downside to all of us? No one really knows what statistical ensemble this belongs to anymore.

Derivatives and Internal Coordinates

Brownian Dynamics has the following equations of motion, where we assume that the geometries x is dependent on the forcefield parameters \theta

x_t = x_{t-1} - k(g(x_{t-1}(\theta), \theta) + noise)

We can differentiate the above w.r.t. \theta

\dfrac{\partial x_t}{\partial \theta} = \dfrac{\partial x_{t-1}}{\partial \theta} - k( \dfrac{\partial g}{\partial x}(x_{t-1})\dfrac{\partial x_{t-1}}{\partial \theta} + \dfrac{\partial g}{\partial \theta})

For the purposes of stability analysis and without loss of generality we can assume:

\dfrac{\partial g}{\partial \theta} = 0

So we can proceed to simplify into:

\dfrac{\partial x_t}{\partial \theta} = (I-k \dfrac{\partial g}{\partial x}) \dfrac{\partial x_{t-1}}{\partial \theta}

It can be shown that this converges if and only if the Hessian \dfrac{\partial g}{\partial x} is PSD and k is sufficiently small.

The issue is that the Hessian is not PSD in standard equations of motion due to “contamination” of the translational and rotational degrees of freedom even for the simplest of systems.

What we’d really like is something that looks like:

\dfrac{\partial x_t}{\partial \theta} = (I-k P^T \dfrac{\partial g}{\partial x} P) \dfrac{\partial x_{t-1}}{\partial \theta}

We can back compute the following anti-derivative :

x_t = x_{t-1} - k P^T (g(P x, \theta) + noise)

But this is invalid since we’d be modifying the positions in two different reference frames, so how would we modify the equations of motion to give us Hessians are that always PSD?

Pure internal coordinates

We can trivially implement Brownian dynamics in either Cartesian Coordinates or Internal Coordinates, since the kinetic energy is irrelevant. Suppose we have a two particle harmonic oscillator:

U(r(x), b) = (r(x)-b)^2

Where r is the distance between the two particles.

We have two equivalent Brownian equations of motion we can use:

x_t = x_{t-1} - k(g(x_{t-1}(\theta), \theta) + noise_{cart})

r_t = r_{t-1} - k(g(r_{t-1}(\theta), \theta) + noise_{int})

noise_{int} = r(noise_{cart})

They have the following derivatives w.r.t. \theta if we again assume the mixed partial is zero.

x'_t = x'_{t-1} - k(H_x(x_{t-1})x'_{t-1})

r'_t = r'_{t-1} - k(H_r(r_{t-1})r'_{t-1})

Where H_x is the Hessian with respect to the coordinate system $x$

The same same equations of motion are generated, except the former is divergent if we sample geometries where r < b. What?

Asymmetric Hessians

I encountered a really counter-intuitive result this past week regarding the Hessians of a simple 2D Harmonic Oscillator:

U(r(x_0, y_0, x_1, y_1), b) = (r-b)^2/2 =(\sqrt{(x_0-x_1)^2+(y_0-y_1)^2}-b)^2/2

It is clear that the energies for r+\delta and r-\delta are the same, and the forces are identical in magnitude except opposite in direction.

However, while the internal Hessian \dfrac{\partial^2 U}{\partial r^2} = 1 and therefore positive semi-definite (PSD) every where w.r.t. r, this is not the case for the 4×4 Cartesian Hessian \dfrac{\partial^2 U}{\partial x_i \partial x_j}. In fact, the Cartesian Hessian is PSD only if r \geq b, that is, it has only positive eigenvalues or zero. We can actually analytically show that the eigenvalues of the Cartesian Hessian are 0, 0, 2 and \lambda where:

\lambda= 2(1-\dfrac{b}{r^2})

So if r < b then the eigenvalue is negative (no longer PSD), and if r>b then the eigenvalue is positive (PSD).

To see this for yourself in sympy, the code is:

import sympy as sp
from sympy.functions import Abs, sqrt
from IPython.display import display, Math
x0, x1, y0, y1, b = sp.symbols('x0 x1 y0 y1 b', real=True)
U = ((sqrt((x0-x1)**2+(y0-y1)**2)-b)**2)/2
args = (x0, y0, x1, y1)
H = sp.Matrix(list(sp.derive_by_array(sp.derive_by_array(U, args), args)))
H = H.reshape(4,4)
H = sp.simplify(H)

Introduce Yourself (Example Post)

This is an example post, originally published as part of Blogging University. Enroll in one of our ten programs, and start your blog right.

You’re going to publish a post today. Don’t worry about how your blog looks. Don’t worry if you haven’t given it a name yet, or you’re feeling overwhelmed. Just click the “New Post” button, and tell us why you’re here.

Why do this?

  • Because it gives new readers context. What are you about? Why should they read your blog?
  • Because it will help you focus you own ideas about your blog and what you’d like to do with it.

The post can be short or long, a personal intro to your life or a bloggy mission statement, a manifesto for the future or a simple outline of your the types of things you hope to publish.

To help you get started, here are a few questions:

  • Why are you blogging publicly, rather than keeping a personal journal?
  • What topics do you think you’ll write about?
  • Who would you love to connect with via your blog?
  • If you blog successfully throughout the next year, what would you hope to have accomplished?

You’re not locked into any of this; one of the wonderful things about blogs is how they constantly evolve as we learn, grow, and interact with one another — but it’s good to know where and why you started, and articulating your goals may just give you a few other post ideas.

Can’t think how to get started? Just write the first thing that pops into your head. Anne Lamott, author of a book on writing we love, says that you need to give yourself permission to write a “crappy first draft”. Anne makes a great point — just start writing, and worry about editing it later.

When you’re ready to publish, give your post three to five tags that describe your blog’s focus — writing, photography, fiction, parenting, food, cars, movies, sports, whatever. These tags will help others who care about your topics find you in the Reader. Make sure one of the tags is “zerotohero,” so other new bloggers can find you, too.