Geometry Rescaling

One of the major issues with the stability of the timemachine is whether or not your Hessians are PSD. A non-PSD Hessian induces a transition matrix with a spectral radius p(r) > 1, causing a divergent trajectory.

Recall Brownian dynamics:

x_t = x_{t-1} - k G(x, \theta) + noise

Assuming a zero mixed partial, we can differentiate with respect to \theta

x'_t = x'_{t-1} - k H(x_{t-1}) x'_{t-1}

x'_t = (I - k H) x'_{t-1}

G and H are the Gradients and Hessians, respectively. Since H is rarely PSD in Cartesian coordinates, how can we fix this?

By playing dirty: consider an alternative dynamical equation where we shrink the geometry by a small amount:

x_t = (1 - k c) x_{t-1} - k G(x, \theta) + noise

Which has derivative:

x'_t = (1 - k c) x'_{t-1} - k H(x_{t-1}) x'_{t-1}

x'_t = (I - k c I - k H) x'_{t-1}

x'_t = (I - k (c I + H) ) x'_{t-1}

Which is equivalent to adding a diagonal to the Hessian. So long as c is bigger than the sum of each row/column, our Hessian is now PSD.

The downside to all of us? No one really knows what statistical ensemble this belongs to anymore.

Leave a comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: