Information Theory (math.IT)

  • PDF
    We study the consequences of 'super-quantum non-local correlations' as represented by the PR-box model of Popescu and Rohrlich, and show PR-boxes can enhance the capacity of noisy interference channels between two senders and two receivers. PR-box correlations violate Bell/CHSH inequalities and are thus stronger -- more non-local -- than quantum mechanics; yet weak enough to respect special relativity in prohibiting faster-than-light communication. Understanding their power will yield insight into the non-locality of quantum mechanics. We exhibit two proof-of-concept channels: first, we show a channel between two sender-receiver pairs where the senders are not allowed to communicate, for which a shared super-quantum bit (a PR-box) allows perfect communication. This feat is not achievable with the best classical (senders share no resources) or quantum entanglement-assisted (senders share entanglement) strategies. Second, we demonstrate a class of channels for which a tunable parameter achieves a double separation of capacities; for some range of \epsilon, the super-quantum assisted strategy does better than the entanglement-assisted strategy, which in turn does better than the classical one.
  • PDF
    Briët et al. showed that an efficient communication protocol implies a reliable XOR game protocol. In this work, we improve this relationship, and obtain a nontrivial lower bound $2\log3\approx 3.1699$ of XOR-amortized communication complexity of the equality function. The proof uses an elegant idea of Pawłowski et al. in a paper on information causality. Although the improvement of a lower bound of a communication complexity is at most a factor 2, all arguments and proofs in this work are quite simple and intuitive.
  • PDF
    We consider the dynamics of message passing for spatially coupled codes and, in particular, the set of density evolution equations that tracks the profile of decoding errors along the spatial direction of coupling. It is known that, for suitable boundary conditions and after a transient phase, the error profile exhibits a "solitonic behavior". Namely, a uniquely-shaped wavelike solution develops, that propagates with constant velocity. Under this assumption we derive an analytical formula for the velocity in the framework of a continuum limit of the spatially coupled system. The general formalism is developed for spatially coupled low-density parity-check codes on general binary memoryless symmetric channels which form the main system of interest in this work. We apply the formula for special channels and illustrate that it matches the direct numerical evaluation of the velocity for a wide range of noise values. A possible application of the velocity formula to the evaluation of finite size scaling law parameters is also discussed. We conduct a similar analysis for general scalar systems and illustrate the findings with applications to compressive sensing and generalized low-density parity-check codes on the binary erasure or binary symmetric channels.
  • PDF
    In this paper, we propose a generalized expectation consistent signal recovery algorithm to estimate the signal $\mathbf{x}$ from the nonlinear measurements of a linear transform output $\mathbf{z}=\mathbf{A}\mathbf{x}$. This estimation problem has been encountered in many applications, such as communications with front-end impairments, compressed sensing, and phase retrieval. The proposed algorithm extends the prior art called generalized turbo signal recovery from a partial discrete Fourier transform matrix $\mathbf{A}$ to a class of general matrices. Numerical results show the excellent agreement of the proposed algorithm with the theoretical Bayesian-optimal estimator derived using the replica method.
  • PDF
    We consider the design of wireless queueing network control policies with particular focus on combining stability with additional application-dependent requirements. Thereby, we consequently pursue a cost function based approach that provides the flexibility to incorporate constraints and requirements of particular services or applications. As typical examples of such requirements, we consider the reduction of buffer underflows in case of streaming traffic, and energy efficiency in networks of battery powered nodes. Compared to the classical throughput optimal control problem, such requirements significantly complicate the control problem. We provide easily verifyable theoretical conditions for stability, and, additionally, compare various candidate cost functions applied to wireless networks with streaming media traffic. Moreover, we demonstrate how the framework can be applied to the problem of energy efficient routing, and we demonstrate the aplication of our framework in cross-layer control problems for wireless multihop networks, using an advanced power control scheme for interference mitigation, based on successive convex approximation. In all scenarios, the performance of our control framework is evaluated using extensive numerical simulations.
  • PDF
    Feedback control actively dissipates uncertainty from a dynamical system by means of actuation. We develop a notion of "control capacity" that gives a fundamental limit (in bits) on the rate at which a controller can dissipate the uncertainty from a system, i.e. stabilize to a known fixed point. We give a computable single-letter characterization of control capacity for memoryless stationary scalar multiplicative actuation channels. Control capacity allows us to answer questions of stabilizability for scalar linear systems: a system with actuation uncertainty is stabilizable if and only if the control capacity is larger than the log of the unstable open-loop eigenvalue. For second-moment senses of stability, we recover the classic uncertainty threshold principle result. However, our definition of control capacity can quantify the stabilizability limits for any moment of stability. Our formulation parallels the notion of Shannon's communication capacity, and thus yields both a strong converse and a way to compute the value of side-information in control. The results in our paper are motivated by bit-level models for control that build on the deterministic models that are widely used to understand information flows in wireless network information theory.
  • PDF
    A secret sharing scheme (SSS) was introduced by Shamir in 1979 using polynomial interpolation. Later it turned out that it is equivalent to an SSS based on a Reed-Solomon code. SSSs based on linear codes have been studied by many researchers. However there is little research on SSSs based on additive codes. In this paper, we study SSSs based on additive codes over $GF(4)$ and show that they require at least two steps of calculations to reveal the secret. We also define minimal access structures of SSSs from additive codes over $GF(4)$ and describe SSSs using some interesting additive codes over $GF(4)$ which contain generalized 2-designs.
  • PDF
    As far as we know, there is no decoding algorithm of any binary self-dual $[40, 20, 8]$ code except for the syndrome decoding applied to the code directly. This syndrome decoding for a binary self-dual $[40,20,8]$ code is not efficient in the sense that it cannot be done by hand due to a large syndrome table. The purpose of this paper is to give two new efficient decoding algorithms for an extremal binary doubly-even self-dual $[40,20, 8]$ code $C_{40,1}^{DE}$ by hand with the help of a Hermitian self-dual $[10,5,4]$ code $E_{10}$ over $GF(4)$. The main idea of this decoding is to project codewords of $C_{40,1}^{DE}$ onto $E_{10}$ so that it reduces the complexity of the decoding of $C_{40,1}^{DE}$. The first algorithm is called the representation decoding algorithm. It is based on the pattern of codewords of $E_{10}$. Using certain automorphisms of $E_{10}$, we show that only eight types of codewords of $E_{10}$ can produce all the codewords of $E_{10}$. The second algorithm is called the syndrome decoding algorithm based on $E_{10}$. It first solves the syndrome equation in $E_{10}$ and finds a corresponding binary codeword of $C_{40,1}^{DE}$.
  • PDF
    A linear code with a complementary dual (or LCD code) is defined to be a linear code $C$ whose dual code $C^{\perp}$ satisfies $C \cap C^{\perp}$= $\left\{ \mathbf{0}\right\} $. Let $LCD{[}n,k{]}$ denote the maximum of possible values of $d$ among $[n,k,d]$ binary LCD codes. We give exact values of $LCD{[}n,k{]}$ for $1 \le k \le n \le 12$. We also show that $LCD[n,n-i]=2$ for any $i\geq2$ and $n\geq2^{i}$. Furthermore, we show that $LCD[n,k]\leq LCD[n,k-1]$ for $k$ odd and $LCD[n,k]\leq LCD[n,k-2]$ for $k$ even.
  • PDF
    We study broadcast capacity and minimum delay scaling laws for highly mobile wireless networks, in which each node has to disseminate or broadcast packets to all other nodes in the network. In particular, we consider a cell partitioned network under the simplified independent and identically distributed (IID) mobility model, in which each node chooses a new cell at random every time slot. We derive scaling laws for broadcast capacity and minimum delay as a function of the cell size. We propose a simple first-come-first-serve (FCFS) flooding scheme that nearly achieves both capacity and minimum delay scaling. Our results show that high mobility does not improve broadcast capacity, and that both capacity and delay improve with increasing cell sizes. In contrast to what has been speculated in the literature we show that there is (nearly) no tradeoff between capacity and delay. Our analysis makes use of the theory of Markov Evolving Graphs (MEGs) and develops two new bounds on flooding time in MEGs by relaxing the previously required expander property assumption.
  • PDF
    Reed-Solomon codes have found many applications in practical storage systems, but were until recently considered unsuitable for distributed storage applications due to the widely-held belief that they have poor repair bandwidth. The work of Guruswami and Wootters (STOC'16) has shown that one can actually perform bandwidth-efficient linear repair with Reed-Solomon codes: When the codes are over the field $\mathbb{F}_{q^t}$ and the number of parities $r \geq q^s$, where $(t-s)$ divides $t$, there exists a linear scheme that achieves a repair bandwidth of $(n-1)(t-s)\log_2 q$ bits. We extend this result by showing the existence of such a linear repair scheme for every $1 \leq s < t$. Moreover, our new schemes are optimal among all linear repair schemes for Reed-Solomon codes when $n = q^t$ and $r = q^s$. Additionally, we improve the lower bound on the repair bandwidth for Reed-Solomon codes, also established in the work of Guruswami and Wootters.
  • PDF
    The physical layer security in the up-link of the wireless communication systems is often modeled as the multiple access wiretap channel (MAC-WT), and recently it has received a lot attention. In this paper, the MAC-WT has been re-visited by considering the situation that the legitimate receiver feeds his received channel output back to the transmitters via two noiseless channels, respectively. This model is called the MAC-WT with noiseless feedback. Inner and outer bounds on the secrecy capacity region of this feedback model are provided. To be specific, we first present a decode-and-forward (DF) inner bound on the secrecy capacity region of this feedback model, and this bound is constructed by allowing each transmitter to decode the other one's transmitted message from the feedback, and then each transmitter uses the decoded message to re-encode his own messages, i.e., this DF inner bound allows the independent transmitters to co-operate with each other. Then, we provide a hybrid inner bound which is strictly larger than the DF inner bound, and it is constructed by using the feedback as a tool not only to allow the independent transmitters to co-operate with each other, but also to generate two secret keys respectively shared between the legitimate receiver and the two transmitters. Finally, we give a sato-type outer bound on the secrecy capacity region of this feedback model. The results of this paper are further explained via a Gaussian example.
  • PDF
    The majority of online content is written in languages other than English, and is most commonly encoded in UTF-8, the world's dominant Unicode character encoding. Traditional compression algorithms typically operate on individual bytes. While this approach works well for the single-byte ASCII encoding, it works poorly for UTF-8, where characters often span multiple bytes. Previous research has focused on developing Unicode compressors from scratch, which often failed to outperform established algorithms such as bzip2. We develop a technique to modify byte-based compressors to operate directly on Unicode characters, and implement variants of LZW and PPM that apply this technique. We find that our method substantially improves compression effectiveness on a UTF-8 corpus, with our PPM variant outperforming the state-of-the-art PPMII compressor. On ASCII and binary files, our variants perform similarly to the original unmodified compressors.
  • PDF
    Nowadays data compressors are applied to many problems of text analysis, but many such applications are developed outside of the framework of mathematical statistics. In this paper we overcome this obstacle and show how several methods of classical mathematical statistics can be developed based on applications of the data compressors.
  • PDF
    While all organisms on Earth descend from a common ancestor, there is no consensus on whether the origin of this ancestral self-replicator was a one-off event or whether it was only the final survivor of multiple origins. Here we use the digital evolution system Avida to study the origin of self-replicating computer programs. By using a computational system, we avoid many of the uncertainties inherent in any biochemical system of self-replicators (while running the risk of ignoring a fundamental aspect of biochemistry). We generated the exhaustive set of minimal-genome self-replicators and analyzed the network structure of this fitness landscape. We further examined the evolvability of these self-replicators and found that the evolvability of a self-replicator is dependent on its genomic architecture. We studied the differential ability of replicators to take over the population when competed against each other (akin to a primordial-soup model of biogenesis) and found that the probability of a self-replicator out-competing the others is not uniform. Instead, progenitor (most-recent common ancestor) genotypes are clustered in a small region of the replicator space. Our results demonstrate how computational systems can be used as test systems for hypotheses concerning the origin of life.
  • PDF
    Cyclic codes are an interesting type of linear codes and have wide applications in communication and storage systems due to their efficient encoding and decoding algorithms. It was proved that asymptotically good Hermitian LCD codes exist. The objective of this paper is to construct some cyclic Hermitian LCD codes over finite fields and analyse their parameters. The dimensions of these codes are settled and the lower bounds on their minimum distances are presented. Most Hermitian LCD codes presented in this paper are not BCH codes. In addition, we employ Hermitian LCD codes to propose a Hermitian orthogonal direct sum masking scheme that achieves protection against fault injection attacks. It is shown that the codes with great minimum distances are desired to improve the resistance.
  • PDF
    We describe a framework to build distances by measuring the tightness of inequalities, and introduce the notion of proper statistical divergences and improper pseudo-divergences. We then consider the Hölder ordinary and reverse inequalities, and present two novel classes of Hölder divergences and pseudo-divergences that both encapsulate the special case of the Cauchy-Schwarz divergence. We report closed-form formulas for those statistical dissimilarities when considering distributions belonging to the same exponential family provided that the natural parameter space is a cone (e.g., multivariate Gaussians), or affine (e.g., categorical distributions). Those new classes of Hölder distances are invariant to rescaling, and thus do not require distributions to be normalized. Finally, we show how to compute statistical Hölder centroids with respect to those divergences, and carry out center-based clustering toy experiments on a set of Gaussian distributions that demonstrate empirically that symmetrized Hölder divergences outperform the symmetric Cauchy-Schwarz divergence.
  • PDF
    Channel-reciprocity based key generation (CRKG) has gained significant importance as it has recently been proposed as a potential lightweight security solution for IoT devices. However, the impact of the attacker's position in close range has only rarely been evaluated in practice, posing an open research problem about the security of real-world realizations. Furthermore, this would further bridge the gap between theoretical channel models and their practice-oriented realizations. For security metrics, we utilize cross-correlation, mutual information, and a lower bound on secret-key capacity. We design a practical setup of three parties such that the channel statistics, although based on joint randomness, are always reproducible. We run experiments to obtain channel states and evaluate the aforementioned metrics for the impact of an attacker depending on his position. It turns out the attacker himself affects the outcome, which has not been adequately regarded yet in standard channel models.
  • PDF
    A cognitive radar adapts the transmit waveform in response to changes in the radar and target environment. In this work, we analyze the recently proposed sub-Nyquist cognitive radar wherein the total transmit power in a multi-band cognitive waveform remains the same as its full-band conventional counterpart. For such a system, we derive lower bounds on the mean-squared-error (MSE) of a single-target time delay estimate. We formulate a procedure to select the optimal bands, and recommend distribution of the total power in different bands to enhance the accuracy of delay estimation. In particular, using Cramér-Rao bounds, we show that equi-width subbands in cognitive radar always have better delay estimation than the conventional radar. Further analysis using Ziv-Zakai bound reveals that cognitive radar performs well in low signal-to-noise (SNR) regions.
  • PDF
    Permutation codes, in the form of rank modulation, have shown promise for applications such as flash memory. One of the metrics recently suggested as appropriate for rank modulation is the Ulam metric, which measures the minimum translocation distance between permutations. Multipermutation codes have also been proposed as a generalization of permutation codes that would improve code size (and consequently the code rate). In this paper we analyze the Ulam metric in the context of multipermutations, noting some similarities and differences between the Ulam metric in the context of permutations. We also consider sphere sizes for multipermutations under the Ulam metric and resulting bounds on code size.
  • PDF
    The promise of compressive sensing (CS) has been offset by two significant challenges. First, real-world data is not exactly sparse in a fixed basis. Second, current high-performance recovery algorithms are slow to converge, which limits CS to either non-real-time applications or scenarios where massive back-end computing is available. In this paper, we attack both of these challenges head-on by developing a new signal recovery framework we call \em DeepInverse that learns the inverse transformation from measurement vectors to signals using a \em deep convolutional network. When trained on a set of representative images, the network learns both a representation for the signals (addressing challenge one) and an inverse map approximating a greedy or convex recovery algorithm (addressing challenge two). Our experiments indicate that the DeepInverse network closely approximates the solution produced by state-of-the-art CS recovery algorithms yet is hundreds of times faster in run time. The tradeoff for the ultrafast run time is a computationally intensive, off-line training procedure typical to deep networks. However, the training needs to be completed only once, which makes the approach attractive for a host of sparse recovery problems.
  • PDF
    In this paper, we represent Raptor codes as multi-edge type low-density parity-check (MET-LDPC) codes, which gives a general framework to design them for higher-order modulation using MET density evolution. We then propose an efficient Raptor code design method for higher-order modulation, where we design distinct degree distributions for distinct bit levels. We consider a joint decoding scheme based on belief propagation for Raptor codes and also derive an exact expression for the stability condition. In several examples, we demonstrate that the higher-order modulated Raptor codes designed using the multi-edge framework outperform previously reported higher-order modulation codes in literature.
  • PDF
    In this paper, we establish new capacity bounds for the multi-sender unicast index-coding problem. We first revisit existing outer and inner bounds proposed by Sadeghi et al. and identify their suboptimality in general. We then propose a new multi-sender maximal-acyclic-induced-subgraph outer bound that improves upon the existing bound in two aspects. For inner bound, we identify shortcomings of the state-of-the-art partitioned Distributed Composite Coding (DCC) in the strategy of sender partitioning and in the implementation of multi-sender composite coding. We then modify the existing sender partitioning by a new joint link-and-sender partitioning technique, which allows each sender to split its link capacity so as to contribute to collaborative transmissions in multiple groups if necessary. This leads to a modified DCC (mDCC) scheme that is shown to outperform partitioned DCC and suffice to achieve optimality for some index-coding instances. We also propose cooperative compression of composite messages in composite coding to exploit the potential overlapping of messages at different senders to support larger composite rates than those by point-to-point compression in the existing DCC schemes. Combining joint partitioning and cooperative compression said, we develop a new multi-sender Cooperative Composite Coding (CCC) scheme for the problem. The CCC scheme improves upon partitioned DCC and mDCC in general, and is the key to achieve optimality for a number of index-coding instances. The usefulness of each scheme is illuminated via examples, and the capacity region is established for each example.
  • PDF
    Sequential estimation of the delay and Doppler parameters for sub-Nyquist radars by analog-to-information conversion (AIC) systems has received wide attention recently. However, the estimation methods reported are AIC-dependent and have poor performance for off-grid targets. This paper develops a general estimation scheme in the sense that it is applicable to all AICs regardless whether the targets are on or off the grids. The proposed scheme estimates the delay and Doppler parameters sequentially, in which the delay estimation is formulated into a beamspace direction-of- arrival problem and the Doppler estimation is translated into a line spectrum estimation problem. Then the well-known spatial and temporal spectrum estimation techniques are used to provide efficient and high-resolution estimates of the delay and Doppler parameters. In addition, sufficient conditions on the AIC to guarantee the successful estimation of off-grid targets are provided, while the existing conditions are mostly related to the on-grid targets. Theoretical analyses and numerical experiments show the effectiveness and the correctness of the proposed scheme.
  • PDF
    The theoretical analysis of detection and decoding of low-density parity-check (LDPC) codes transmitted over channels with two-dimensional (2D) interference and additive white Gaussian noise (AWGN) is provided in this paper. The detection and decoding system adopts the joint iterative detection and decoding scheme (JIDDS) in which the log-domain sum-product algorithm is adopted to decode the LDPC codes. The graph representations of the JIDDS are explained. Using the graph representations, we prove that the message-flow neighborhood of the detection and decoding system will be tree-like for a sufficiently long code length. We further confirm that the performance of the JIDDS will concentrate around the performance in which message-flow neighborhood is tree-like. Based on the tree-like message-flow neighborhood, we employ a modified density evolution algorithm to track the message densities during the iterations. A threshold is calculated using the density evolution algorithm which can be considered as the theoretical performance limit of the system. Simulation results demonstrate that the modified density evolution is effective in analyzing the performance of 2D interference systems.
  • PDF
    Cooperative relaying is often deployed to enhance the communication reliability (i.e., diversity order) and consequently the end-to-end achievable rate. However, this raises several security concerns when the relays are untrusted since they may have access to the relayed message. In this paper, we study the achievable secrecy diversity order of cooperative networks with untrusted relays. In particular, we consider a network with an N-antenna transmitter (Alice), K single-antenna relays, and a single-antenna destination (Bob). We consider the general scenario where there is no relation between N and K, and therefore K can be larger than N. Alice and Bob are assumed to be far away from each other, and all communication is done through the relays, i.e., there is no direct link. Providing secure communication while enhancing the diversity order has been shown to be very challenging. In fact, it has been shown in the literature that the maximum achievable secrecy diversity order for the adopted system model is one (while using artificial noise jamming). In this paper, we adopt a nonlinear interference alignment scheme that we have proposed recently to transmit the signals from Alice to Bob. We analyze the proposed scheme in terms of the achievable secrecy rate and secrecy diversity order. Assuming Gaussian inputs, we derive an explicit expression for the achievable secrecy rate and show analytically that a secrecy diversity order of up to min(N,K)-1 can be achieved using the proposed technique. We provide several numerical examples to validate the obtained analytical results and demonstrate the superiority of the proposed technique to its counterparts that exist in the literature.

Recent comments

Samad Khabbazi Oskouei Sep 05 2016 11:34 UTC

I think that we have missed the "semi-" at the conclusion. Because, the proof of the theorem 4.3 is based on the using universal semi-density matrix concept which is not computable. The semi-computability concept used here is like the Kolmogorov complexity which is not computable and so the Cubic co

...(continued)
Toby Cubitt Sep 01 2016 11:14 UTC

I could well be missing something. But as far as I could tell from a rather quick read through the paper, all they show is that the quantum capacity of a channel with computable matrix elements is given by the regularised coherent information optimised over input ensembles with computable matrix ele

...(continued)
Māris Ozols Aug 30 2016 17:52 UTC

Do I understand correctly that this paper claims to show that quantum capacity is computable?

> After defining the algorithmic quantum capacity we have proved that it
> equals the standard one. Furthermore we have shown that it is
> computable.

Richard Kueng Jul 28 2015 07:01 UTC

fyi: our quantum implications are presented in Subsection 2.2 (pp 7-9).

Marco Tomamichel May 31 2015 22:07 UTC

Thanks for the comment! This is a good idea, I will do that in the next arXiv version.

Patrick Hayden May 28 2015 17:31 UTC

Wonderful! I've been waiting for a book like this for a while now! Thanks, Marco.

I do have one trivial comment from a 30 second preliminary scan, though: please consider typesetting the proofs with a font size matching the main text. If us readers are already squinting hard trying to understand

...(continued)
Marco Tomamichel Apr 02 2015 03:21 UTC

This is a preliminary version and I am happy to incorporate feedback I receive in the coming month. Any comments are welcome.

Māris Ozols Mar 17 2015 11:00 UTC

The strange equation is supposed to look like this:
$$f(\sqrt{a} X + \sqrt{1-a} Y) \geq a f(X) + (1-a) f(Y) \quad \forall a \in [0,1]$$

Yuanzhu Aug 02 2014 04:21 UTC

This algorithm is from Wu's list decoding algorithm.