# Information Theory (cs.IT)

• We solve tensor balancing, rescaling an Nth order nonnegative tensor by multiplying (N - 1)th order N tensors so that every fiber sums to one. This generalizes a fundamental process of matrix balancing used to compare matrices in a wide range of applications from biology to economics. We present an efficient balancing algorithm with quadratic convergence using Newton's method and show in numerical experiments that the proposed algorithm is several orders of magnitude faster than existing ones. To theoretically prove the correctness of the algorithm, we model tensors as probability distributions in a statistical manifold and realize tensor balancing as projection onto a submanifold. The key to our algorithm is that the gradient of the manifold, used as a Jacobian matrix in Newton's method, can be analytically obtained using the Möbius inversion formula, the essential of combinatorial mathematics. Our model is not limited to tensor balancing but has a wide applicability as it includes various statistical and machine learning models such as weighted DAGs and Boltzmann machines.
• Alternating minimization, or Fienup methods, have a long history in phase retrieval. We provide new insights related to the empirical and theoretical analysis of these algorithms when used with Fourier measurements and combined with convex priors. In particular, we show that Fienup methods can be viewed as performing alternating minimization on a regularized nonconvex least-squares problem with respect to amplitude measurements. We then prove that under mild additional structural assumptions on the prior (semi-algebraicity), the sequence of signal estimates has a smooth convergent behaviour towards a critical point of the nonconvex regularized least-squares objective. Finally, we propose an extension to Fienup techniques, based on a projected gradient descent interpretation and acceleration using inertial terms. We demonstrate experimentally that this modification combined with an $\ell_1$ prior constitutes a competitive approach for sparse phase retrieval.
• In multi-(core/mode) optical fiber communication, the transmission channel can be modeled as a complex sub-matrix of the Haar-distributed unitary matrix (complex Jacobi unitary ensemble). In this letter, we present new analytical expressions of the upper and lower bounds for the ergodic capacity of multiple-input multiple-output Jacobi-fading channels. Recent results on the determinant of the Jacobi unitary ensemble are employed to derive a tight lower bound on the ergodic capacity. We use Jensen's inequality to provide an analytical closed-form upper bound to the ergodic capacity at any signal-to-noise ratio (SNR). Closed-form expressions of the ergodic capacity, at low and high SNR regimes, are also derived. Simulation results are presented to validate the accuracy of the derived expressions.
• We describe the second (generalized) Feng-Rao distance for elements in an Arf numerical semigroup that are greater than or equal to the conductor of the semigroup. This provides a lower bound for the second Hamming weight for one point AG codes. In particular, we can obtain the second Feng-Rao distance for the codes defined by asymptotically good towers of function fields whose Weierstrass semigroups are inductive. In addition, we compute the second Feng-Rao number, and provide some examples and comparisons with previous results on this topic. These calculations rely on Apéry sets, and thus several results concerning Apéry sets of Arf semigroups are presented.
• Feb 28 2017 cs.IT math.IT arXiv:1702.08152v1
The Korkine-Zolotareff (KZ) reduction is one of the often used reduction strategies for decoding lattices. A KZ reduction algorithm involves solving shortest vector problems (SVPs) and basis expansion. In this paper, first we improve the commonly used Schnorr-Euchner search strategy for solving SVPs. Then, we derive upper bounds on the magnitudes of the entries of any solution of a SVP when its basis matrix is LLL reduced. These upper bounds only involve the parameter of the LLL reduction and the dimension of the solution and they are useful for analyzing the complexity of the basis expansion in a KZ reduction algorithm. Finally, we modify the basis expansion method proposed by Zhang et al. and combine it with the improved Schnorr-Euchner search strategy to give a new KZ reduction algorithm. Simulation results show that the new KZ reduction algorithm is much faster and numerically more stable than the KZ reduction algorithm proposed by Zhang et al., especially when the basis matrix is ill conditioned.
• The capacity of a discrete-time multi-input multioutput (MIMO) Gaussian channel with output quantization is investigated for different analog receiver architectures. A general formulation of this problem is proposed in which the antenna outputs are processed by analog combiners while sign quantizers are used for analog/digital conversion. The configuration of the analog combiners is chosen as a function of the channel realization, so that capacity can be maximized over the set of available configurations for a fixed number of sign quantizers. To exemplify this approach, three analog receiver architectures of increasing generality are considered: (a) sign quantization of the antenna outputs, (b) antenna selection and multilevel quantization and (c) linear combining of the antenna outputs and multilevel quantization. By comparing the largest attainable rates for a given number of sign quantizers, it is possible to quantify the limiting rate advantages provided by each of the receiver analog architectures. In particular, it is shown that sign quantization of the antenna outputs is sufficient, among all possible receiver architectures studied, to attain the optimal high signal-to-noise ratio (SNR) performance for MIMO receiver in which the number of antennas is larger than the number of sign quantizers.
• In this paper, we develop a low-complexity channel estimation for hybrid millimeter wave (mmWave) systems, where the number of radio frequency (RF) chains is much less than the number of antennas equipped at each transceiver. The proposed channel estimation algorithm aims to estimate the strongest angle-of-arrivals (AoAs) at both the base station (BS) and the users. Then all the users transmit orthogonal pilot symbols to the BS via these estimated strongest AoAs to facilitate the channel estimation. The algorithm does not require any explicit channel state information (CSI) feedback from the users and the associated signalling overhead of the algorithm is only proportional to the number of users, which is significantly less compared to various existing schemes. Besides, the proposed algorithm is applicable to both non-sparse and sparse mmWave channel environments. Based on the estimated CSI, zero-forcing (ZF) precoding is adopted for multiuser downlink transmission. In addition, we derive a tight achievable rate upper bound of the system. Our analytical and simulation results show that the proposed scheme offer a considerable achievable rate gain compared to fully digital systems, where the number of RF chains equipped at each transceiver is equal to the number of antennas. Furthermore, the achievable rate performance gap between the considered hybrid mmWave systems and the fully digital system is characterized, which provides useful system design insights.
• For ergodic fading, a lattice coding and decoding strategy is proposed and its performance is analyzed for the single-input single-output (SISO) and multiple-input multiple-output (MIMO) point-to-point channel as well as the multiple-access channel (MAC), with channel state information available only at the receiver (CSIR). At the decoder a novel strategy is proposed consisting of a time-varying equalization matrix followed by decision regions that depend only on channel statistics, not individual realizations. Our encoder has a similar structure to that of Erez and Zamir. For the SISO channel, the gap to capacity is bounded by a constant under a wide range of fading distributions. For the MIMO channel under Rayleigh fading, the rate achieved is within a gap to capacity that does not depend on the signal-to-noise ratio (SNR), and diminishes with the number of receive antennas. The analysis is extended to the K-user MAC where similar results hold. Achieving a small gap to capacity while limiting the use of CSIR to the equalizer highlights the scope for efficient decoder implementations, since decision regions are fixed, i.e., independent of channel realizations.
• The Fisher information can be used to indicate the precision of parameter estimation by the quantum Cramér-Rao inequality. This paper presents an efficient numerical algorithm for the calculation of Fisher information based on quantum weak measurement. According to the quantum stochastic master equation, the Fisher information is expressed in the form of log-likelihood functions. Three main methods are employed in this algorithm: (i) we use the numerical differentiation approach to calculate the derivative of the log-likelihood function; (ii) we randomly generate a series of parameters of interest by the Metropolis Hastings (MH) algorithm; and (iii) the values of expectation can be approximated by the Markov chain Monte Carlo (MCMC) integration. Finally, as an example to testify the feasibility of the proposed algorithm, we consider the dissipation rates of the open quantum system as unknown parameters that need to be estimated. We show that the Fisher information can reach a precision close to the Heisenberg limit in the weak coupling condition. This again demonstrates the effectiveness of the new algorithm.
• Algorithmic statistics studies explanations of observed data that are good in the algorithmic sense: an explanation should be simple i.e. should have small Kolmogorov complexity and capture all the algorithmically discoverable regularities in the data. However this idea can not be used in practice because Kolmogorov complexity is not computable. In this paper we develop algorithmic statistics using space-bounded Kolmogorov complexity. We prove an analogue of one of the main result of `classic' algorithmic statistics (about the connection between optimality and randomness deficiences). The main tool of our proof is the Nisan-Wigderson generator.
• The topological interference management (TIM) problem studies partially-connected interference networks with no channel state information except for the network topology (i.e., connectivity graph) at the transmitters. In this paper, we consider a similar problem in the uplink cellular networks, while message passing is enabled at the receivers (e.g., base stations), so that the decoded messages can be routed to other receivers via backhaul links to help further improve network performance. For this TIM problem with decoded message passing (TIM-MP), we model the interference pattern by conflict digraphs, connect orthogonal access to the acyclic set coloring on conflict digraphs, and show that one-to-one interference alignment boils down to orthogonal access because of message passing. With the aid of polyhedral combinatorics, we identify the structural properties of certain classes of network topologies where orthogonal access achieves the optimal degrees-of-freedom (DoF) region in the information-theoretic sense. The relation to the conventional index coding with simultaneous decoding is also investigated by formulating a generalized index coding problem with successive decoding as a result of decoded message passing. The properties of reducibility and criticality are also studied, by which we are able to prove the linear optimality of orthogonal access in terms of symmetric DoF for the networks up to four users with all possible network topologies (218 instances). Practical issues of the tradeoff between the overhead of message passing and the achievable symmetric DoF are also discussed, in the hope of facilitating efficient backhaul utilization.
• Motivated by the question of whether the recently introduced Reduced Cutset Coding (RCC) offers rate-complexity performance benefits over conventional context-based conditional coding for sources with two-dimensional Markov structure, this paper compares several row-centric coding strategies that vary in the amount of conditioning as well as whether a model or an empirical table is used in the encoding of blocks of rows. The conclusion is that, at least for sources exhibiting low-order correlations, 1-sided model-based conditional coding is superior to the method of RCC for a given constraint on complexity, and conventional context-based conditional coding is nearly as good as the 1-sided model-based coding.
• In this paper, we aim to obtain the optimal delay-power tradeoff and the corresponding optimal scheduling policy for arbitrary i.i.d. arrival process and adaptive transmissions. The number of backlogged packets at the transmitter is known to a scheduler, who has to determine how many backlogged packets to transmit during each time slot. The power consumption is assumed to be convex in transmission rates. Hence, if the scheduler transmits faster, the delay will be reduced but with higher power consumption. To obtain the optimal delay-power tradeoff and the corresponding optimal policy, we model the problem as a Constrained Markov Decision Process (CMDP), where we minimize the average delay given an average power constraint. By steady-state analysis and Lagrangian relaxation, we can show that the optimal tradeoff curve is decreasing, convex, and piecewise linear, and the optimal policy is threshold-based. Based on the revealed properties of the optimal policy, we develop an algorithm to efficiently obtain the optimal tradeoff curve and the optimal policy. The complexity of our proposed algorithm is much lower than a general algorithm based on Linear Programming. We validate the derived results and the proposed algorithm through Linear Programming and simulations.
• Feb 28 2017 cs.IT math.IT arXiv:1702.08033v1
Linear codes with complementary duals (abbreviated LCD) are linear codes whose intersection with their dual is trivial. When they are binary, they play an important role in armoring implementations against side-channel attacks and fault injection attacks. Non-binary LCD codes in characteristic 2 can be transformed into binary LCD codes by expansion. On the other hand, being optimal codes, maximum distance separable codes (abbreviated MDS) have been of much interest from many researchers due to their theoretical significant and practical implications. However, little work has been done on LCD MDS codes. In particular, determining the existence of $q$-ary $[n,k]$ LCD MDS codes for various lengths $n$ and dimensions $k$ is a basic and interesting problem. In this paper, we firstly study the problem of the existence of $q$-ary $[n,k]$ LCD MDS codes and completely solve it for the Euclidean case. More specifically, we show that for $q>3$ there exists a $q$-ary $[n,k]$ Euclidean LCD MDS code, where $0\le k \le n\le q+1$, or, $q=2^{m}$, $n=q+2$ and $k= 3 \text{or} q-1$. Secondly, we investigate several constructions of new Euclidean and Hermitian LCD MDS codes. Our main techniques in constructing Euclidean and Hermitian LCD MDS codes use some linear codes with small dimension or codimension, self-orthogonal codes and generalized Reed-Solomon codes.
• This work contains two main contributions concerning the asymmetric broadcast channel. The first is an analysis of the exact random coding error exponents for both users, and the second is the derivation of universal decoders for both users. These universal decoders are certain variants of the maximum mutual information (MMI) universal decoder, which achieve the corresponding random coding exponents of optimal decoding. In addition, we introduce some lower bounds, which involve optimization over very few parameters, unlike the original, exact exponents, which involve minimizations over auxiliary probability distributions. Numerical results for the binary symmetric broadcast channel show improvements over previously derived error exponents for the same model.
• In this paper, a multi-scale approach to spectrum sensing in cognitive cellular networks is proposed. In order to overcome the huge cost incurred in the acquisition of full network state information, a hierarchical scheme is proposed, based on which local state estimates are aggregated up the hierarchy to obtain aggregate state information at multiple scales, which are then sent back to each cell for local decision making. Thus, each cell obtains fine-grained estimates of the channel occupancies of nearby cells, but coarse-grained estimates of those of distant cells. The performance of the aggregation scheme is studied in terms of the trade-off between the throughput achievable by secondary users and the interference generated by the activity of these secondary users to primary users. In order to account for the irregular structure of interference patterns arising from path loss, shadowing, and blockages, which are especially relevant in millimeter wave networks, a greedy algorithm is proposed to find a multi-scale aggregation tree to optimize the performance. It is shown numerically that this tailored hierarchy outperforms a regular tree construction by 60%.
• This paper considers the minimization of a general objective function $f(X)$ over the set of non-square $n\times m$ matrices where the optimal solution $X^\star$ is low-rank. To reduce the computational burden, we factorize the variable $X$ into a product of two smaller matrices and optimize over these two matrices instead of $X$. Despite the resulting nonconvexity, recent studies in matrix completion and sensing have shown that the factored problem has no spurious local minima and obeys the so-called strict saddle property (the function has a directional negative curvature at all critical points but local minima). We analyze the global geometry for a general and yet well-conditioned objective function $f(X)$ whose restricted strong convexity and restricted strong smoothness constants are comparable. In particular, we show that the reformulated objective function has no spurious local minima and obeys the strict saddle property. These geometric properties implies that a number of iterative optimization algorithms (such as gradient descent) can provably solve the factored problem with global convergence.
• In a projective plane $\Pi_{q}$ (not necessarily Desarguesian) of order $q$, a point subset $\mathcal{S}$ is saturating (or dense) if any point of $\Pi_{q}\setminus \mathcal{S}$ is collinear with two points in $\mathcal{S}$. Modifying an approach of [31], we proved the following upper bound on the smallest size $s(2,q)$ of a saturating set in $\Pi_{q}$: \beginequation* s(2,q)≤\sqrt(q+1)\left(3\ln q+\ln\ln q +\ln\frac34\right)+\sqrt\fracq3\ln q+3. \endequation* The bound holds for all q, not necessarily large. By using inductive constructions, upper bounds on the smallest size of a saturating set in the projective space $\mathrm{PG}(N,q)$ with even dimension $N$ are obtained. All the results are also stated in terms of linear covering codes.
• In this manuscript we study channel-aware decision fusion (DF) in a wireless sensor network (WSN) where: (i) the sensors transmit their decisions simultaneously for spectral efficiency purposes and the DF center (DFC) is equipped with multiple antennas; (ii) each sensor-DFC channel is described via a Rician model. As opposed to the existing literature, in order to account for stringent energy constraints in the WSN, only statistical channel information is assumed for the non-line-of sight (scattered) fading terms. For such a scenario, sub-optimal fusion rules are developed in order to deal with the exponential complexity of the likelihood ratio test (LRT) and impractical (complete) system knowledge. Furthermore, the considered model is extended to the case of (partially unknown) jamming-originated interference. Then the obtained fusion rules are modified with the use of composite hypothesis testing framework and generalized LRT. Coincidence and statistical equivalence among them are also investigated under some relevant simplified scenarios. Numerical results compare the proposed rules and highlight their jammingsuppression capability.
• In this paper, we analyze the performance of a time-slotted multi-antenna wireless powered communication (WPC) system, where a wireless device first harvests radio frequency (RF) energy from a power station (PS) in the downlink to facilitate information transfer to an information receiving station (IRS) in the uplink. The main goal of this paper is to provide insights and guidelines for the design of practical WPC systems. To this end, we adopt a recently proposed parametric non-linear RF energy harvesting (EH) model, which has been shown to accurately model the end-to-end non-linearity of practical RF EH circuits. In order to enhance the RF power transfer efficiency, maximum ratio transmission is adopted at the PS to focus the energy signals on the wireless device. Furthermore, at the IRS, maximum ratio combining is used. We analyze the outage probability and the average throughput of information transfer, assuming Nakagami-$m$ fading uplink and downlink channels. Moreover, we study the system performance as a function of the number of PS transmit antennas, the number of IRS receive antennas, the transmit power of the PS, the fading severity, the transmission rate of the wireless device, and the EH time duration. In addition, we obtain a fixed point equation for the optimal transmission rate and the optimal EH time duration that maximize the asymptotic throughput for high PS transmit powers. All analytical results are corroborated by simulations.
• Linear complementary-dual (LCD for short) codes are linear codes that intersect with their duals trivially. LCD codes have been used in certain communication systems. It is recently found that LCD codes can be applied in cryptography. This application of LCD codes renewed the interest in the construction of LCD codes having a large minimum distance. MDS codes are optimal in the sense that the minimum distance cannot be improved for given length and code size. Constructing LCD MDS codes is thus of significance in theory and practice. Recently, Jin (\citeJin, IEEE Trans. Inf. Theory, 2016) constructed several classes of LCD MDS codes through generalized Reed-Solomon codes. In this paper, a different approach is proposed to obtain new LCD MDS codes from generalized Reed-Solomon codes. Consequently, new code constructions are provided and certain previously known results in \citeJin are extended.
• We study the problem of using i.i.d. samples from an unknown multivariate probability distribution $p$ to estimate the mutual information of $p$. This problem has recently received attention in two settings: (1) where $p$ is assumed to be Gaussian and (2) where $p$ is assumed only to lie in a large nonparametric smoothness class. Estimators proposed for the Gaussian case converge in high dimensions when the Gaussian assumption holds, but are brittle, failing dramatically when $p$ is not Gaussian. Estimators proposed for the nonparametric case fail to converge with realistic sample sizes except in very low dimensions. As a result, there is a lack of robust mutual information estimators for many realistic data. To address this, we propose estimators for mutual information when $p$ is assumed to be a nonparanormal (a.k.a., Gaussian copula) model, a semiparametric compromise between Gaussian and nonparametric extremes. Using theoretical bounds and experiments, we show these estimators strike a practical balance between robustness and scaling with dimensionality.
• The speed at which two remote parties can exchange secret keys over a fixed-length fiber-optic cable in continuous-variable quantum key distribution (CV-QKD) is currently limited by the computational complexity of post-processing algorithms for key reconciliation. Multi-edge low-density parity-check (LDPC) codes with low code rates and long block lengths were proposed for CV-QKD, in order to extend the maximum reconciliation distance between the two remote parties. Key reconciliation over multiple dimensions has been shown to further improve the error-correction performance of multi-edge LDPC codes in CV-QKD, thereby increasing both the secret key rate and distance. However, the computational complexity of LDPC decoding for long block lengths on the order of 10^6 bits remains a challenge. This work introduces a quasi-cyclic code construction for multi-edge LDPC codes that is highly suitable for hardware-accelerated decoding on a modern graphics processing unit (GPU). When combined with an 8-dimensional reconciliation scheme, the LDPC decoder achieves a raw decoding throughput of 1.72Mbit/s and an information throughput of 7.16Kbit/s using an NVIDIA GeForce GTX 1080 GPU, at a maximum distance of 164km with an effective secret key rate of 8.24x10^-7 bits/pulse for a rate 0.02 multi-edge code with block length of 10^6 bits. For distances beyond 130km, the decoder delivers an information throughput between 2745x and 9138x higher than the maximum secret key rate achievable with a 1MHz light source, thereby showing that LDPC decoding is no longer the computational bottleneck in long-distance CV-QKD.
• This paper presents a survey on generalized Reed-Solomon codes and various decoding algorithms: Berlekamp-Massey decoding algorithms; Berlekamp-Welch decoding algorithms; Euclidean decoding algorithms; discrete Fourier decoding algorithms, Chien's search algorithm, and Forney's algorithm.

Samad Khabbazi Oskouei Sep 05 2016 11:34 UTC

I think that we have missed the "semi-" at the conclusion. Because, the proof of the theorem 4.3 is based on the using universal semi-density matrix concept which is not computable. The semi-computability concept used here is like the Kolmogorov complexity which is not computable and so the Cubic co

...(continued)
Toby Cubitt Sep 01 2016 11:14 UTC

I could well be missing something. But as far as I could tell from a rather quick read through the paper, all they show is that the quantum capacity of a channel with computable matrix elements is given by the regularised coherent information optimised over input ensembles with computable matrix ele

...(continued)
Māris Ozols Aug 30 2016 17:52 UTC

Do I understand correctly that this paper claims to show that quantum capacity is computable?

> After defining the algorithmic quantum capacity we have proved that it
> equals the standard one. Furthermore we have shown that it is
> computable.

Richard Kueng Jul 28 2015 07:01 UTC

fyi: our quantum implications are presented in Subsection 2.2 (pp 7-9).

Marco Tomamichel May 31 2015 22:07 UTC

Thanks for the comment! This is a good idea, I will do that in the next arXiv version.

Patrick Hayden May 28 2015 17:31 UTC

Wonderful! I've been waiting for a book like this for a while now! Thanks, Marco.

I do have one trivial comment from a 30 second preliminary scan, though: please consider typesetting the proofs with a font size matching the main text. If us readers are already squinting hard trying to understand

...(continued)
Marco Tomamichel Apr 02 2015 03:21 UTC

This is a preliminary version and I am happy to incorporate feedback I receive in the coming month. Any comments are welcome.

Māris Ozols Mar 17 2015 11:00 UTC

The strange equation is supposed to look like this:
$$f(\sqrt{a} X + \sqrt{1-a} Y) \geq a f(X) + (1-a) f(Y) \quad \forall a \in [0,1]$$

Yuanzhu Aug 02 2014 04:21 UTC

This algorithm is from Wu's list decoding algorithm.