Reading time: 8 min

15 /4/2019

Methodology to Calculate the Entropic Index q for 1D Signals

The determination of the q index is often employed empirically and values that are proposed in literature are commonly the result of trial and error based on sufficiently good results. In this article, I will demonstrate an algorithmic procedural way to determine the q index reported in [1]. Although, the original procedure has been devised for image segmentation, a slight modification is conducted for our application in 1D signal processing. The procedure relies on the redundancy concept and maximum entropy principle, which basically constitutes to calculating the redundancy R_T as a function of entropic index q.

Before delving into the methodology, I will provide a brief background on Tsallis entropy, as some of the theoretical equations S_T and S_{Tmax} are required in the procedure.

Background

The Tsallis entropy is given by equation (2):

Where  represents the total number of accessible microstates of system [1], each one with probability p_i, where 1\leqi\leq\Omega with probabilities p_i, satisfy equation \sum_{i=1}^{\Omega}p_i=1. The number q is an entropic index that characterizes the degree of non-extensivity of the system. A system is considered non-extensive if it satisfies the equation S_{BG}(A+B) \neq S_{BG}(A)+S_{BG}(B).

Note:

  1. When \lim_{q\to\infty}, S_T, the Tsallis entropy reduces back to the Boltzmann-Gibbs entropy or so called Shannon entropy.
  2. The maximum Tsallis entropy is obtained when each microstate is equilprobable, given by:

  • S_T will be maximum if q>0, and S_T will be minimum if q<0. If q\geq1, S_T approaches \frac{1}{q-1}, and if q\leq1, S_T diverges to \infty.

Methodology

  1. Given a 1D signal, construct a histogram then perform normalisation by dividing by the area under the curve. Luckily, Matlab has a function to do this. Remove any probability points p_i that equal zero to remove redundancy.
  2. Calculate S_T and S_{Tmax} using equation 2 and 3, where \Omega is the number of probability points in the histogram and q starting at for example -2,
  3. Using equation 1, calculate the redundancy R_T,
  4. Repeat steps 2 and 3, incrementing q value by 0.1 up to for example 10.
  5. Plot R_T versus q, and pick the q value that maximises R_T. Depending how your plot looks, you may have to adjust your q value range and increment size.

Code

function q_best = optimalQ(x,nbins,k,minq, maxq, increq)

%x = 1D signal, nbins = number of bins, k is usually 1
%minq = minimum q value to start, maxq = maximum q value to end
%increq = increment q value

%normalise histogram
h = histogram(x,nbins,’normalisation’,’probability’);
%remove p = 0 points
h1 = h.Data(h.Data~=0);
c=1;
output = [];
for q=minq:increq:maxq
	 [ST, ST_max] = tsallisE(h1,q,k);
	RT = 1-ST/ST_max;
	output(c,1) = RT;
	output(c,2) = q;
	c = c +1;
end

%find q that gives max RT
[pks,loc] = findpeaks(output(:,1),output(:,2));
If isempty(loc)
	error(‘q range does not capture maxima’);
else
	q_best = loc;
end
function [ST,ST_max] = tsallisE(h1,q,k)
	ST = (k/(q-1))*(1-sum(h1.^q));
	ST_max = k*(1-length(h1)^(1-q))/(q-1)

Reference:

[1] Ramirez-Reyes, A., Herandez-Montoya, A.R., Herrera-Corral, G., Jimenez, I.D., 2016, Determining the entropic index q of Tsallis entropy in Images through redundancy, Entropy, 18 (8), 299

Hello.

Submit a form to get to touch