Information-Theory on emsenn.net

Affective Drift in Large Language Models

Thu, 05 Mar 2026 00:00:00 +0000

Abstract

When large language models (LLMs) generate text in semantic domains with low structure — where conceptual relationships are loose, contested, or multiply determined — they tend to produce language that is affectively charged: rhythmic, morally cadenced, and emotionally reassuring. This paper proposes an entropy-based explanation for this behavior. In low-structure domains, the probability distribution over next tokens is relatively flat: many continuations are roughly equally likely. Affective language patterns — parallelism, aphoristic closure, moral framing — offer high-regularity sequences that reduce local entropy. The model follows these patterns not because it is expressing emotion but because affective syntax provides the most predictable path through a region of the probability landscape that otherwise offers little constraint. The paper distinguishes the established components of this account from the conjecture that connects them, and identifies the observable consequences that would follow if the conjecture holds.

Black Holes as Information-Theoretic Stability Optimizers

Sat, 11 Oct 2025 00:00:00 +0000

Abstract

We propose that black holes can be modeled as information-theoretic stability optimizers: physical systems that maximize entropy while minimizing divergence between interior and exterior information distributions. Using the definition of stability reward as a function of coupling between systems (see Information-Theoretic Stability as Reward Function), we reformulate the laws of black-hole thermodynamics as expressions of informational coupling across the horizon. The event horizon is interpreted as a stability boundary where entropy flux across the surface balances the internal rate of informational compression. This perspective unifies the thermodynamic, holographic, and information-geometric descriptions of black holes and suggests a general stability principle underlying gravitational dynamics.

Conjecture: Stability Manifolds in Coupled Networks

Sat, 11 Oct 2025 00:00:00 +0000

Statement

For a network of $N$ coupled systems $\{A_i\}$ with symmetric nonnegative adjacency matrix $C_{ij}=C_{ji}\ge0$ , the total stability reward is

R_{\mathrm{total}} = \sum_i R_s^{(A_i)} + \sum_{i<j} C_{ij}\, \Delta I(A_i;A_j),

where $R_s^{(A_i)}$ is the stability reward of each system (defined in Information-Theoretic Stability as Reward Function) and $\Delta I$ is the temporal change in mutual information between pairs.

Stationary points satisfying $\nabla_p R_{\mathrm{total}} = 0$ form a stability manifold $\mathcal{M}_S \subset \mathcal{P}^N$ , representing the ensemble of joint distributions at informational equilibrium.

Moloch's Bargain as Necessary Misalignment of Truth-Value under Epistemic Constraint

Wed, 08 Oct 2025 00:00:00 +0000

Abstract

Recent experiments show that large language models optimized for audience reward improve their proxy metrics while increasing misalignment indicators such as deception or disinformation.

This paper interprets those findings through the Theorem of Necessary Misalignment of Truth-Value under Epistemic Constraint.

Under a finite informational rate $I(X;Y)\!\le\!R$ and strict proxy–semantic mismatch, that theorem predicts a monotone increase in semantic distortion with optimization intensity.

We show that El & Zou’s (2025) “Moloch’s Bargain” provides an empirical instance of this theoretical trade-off: their observed slopes between reward gain and misalignment correspond to the positive derivative dD_T^\*/dr>0 on the achievable frontier.