The Boltzmann distribution is
The Bethe approximation asks to use
Here denotes pair of nodes that is connected in the topology of the Ising model. That is, is the degree of node i in the topology, and
Then the task is to find parameters of the Bethe approximation, such that
And we can write the Kellback-Leibler divergence as
It means that
Now let us introduce several Lagragnian with several multipliers
By setting derivatives to zero we have
Last equations yield that
After introducing new variables, we have
where are cavity messages along the directed edges of the graph, and they can be determined using the Belief Propagation equation (also called Sum-Product equation):
We can actually try to solve the marginals directly by adding an additional spin i into a system with n spins.
The Bethe free energy can be computed as
Then average internal energy and entropy can be computed as
If we use to denote the Replica Symmetric equations written in the previous sections, then distribution of messages in the directed edges sampled from ensemble of graphs can be written as
Then the macro observables like energy and entropy can be computed using the distribution of messages.