Information Theory (math.IT)

  • PDF
    We obtain the first polynomial-time algorithm for exact tensor completion that improves over the bound implied by reduction to matrix completion. The algorithm recovers an unknown 3-tensor with $r$ incoherent, orthogonal components in $\mathbb R^n$ from $r\cdot \tilde O(n^{1.5})$ randomly observed entries of the tensor. This bound improves over the previous best one of $r\cdot \tilde O(n^{2})$ by reduction to exact matrix completion. Our bound also matches the best known results for the easier problem of approximate tensor completion (Barak & Moitra, 2015). Our algorithm and analysis extends seminal results for exact matrix completion (Candes & Recht, 2009) to the tensor setting via the sum-of-squares method. The main technical challenge is to show that a small number of randomly chosen monomials are enough to construct a degree-3 polynomial with a precisely planted orthogonal global optima over the sphere and that this fact can be certified within the sum-of-squares proof system.
  • PDF
    We focus on energy harvesting (EH) two-hop communications since they are the essential building blocks of more complicated multi-hop networks. The scenario consists of three nodes, where an EH transmitter wants to send data to a receiver through an EH relay. The harvested energy is used exclusively for data transmission and we address the problem of how to efficiently use it. As in practical scenarios, we assume only causal knowledge at the EH nodes, i.e., in each time interval, the transmitter and the relay know their own current and past amounts of incoming energy, battery levels, data buffer levels and channel coefficients for their own transmit channels. Our goal is to find transmission policies which aim at maximizing the throughput considering that the EH nodes fully cooperate with each other to exchange their causal knowledge during a signaling phase. We model the problem as a Markov game and propose a multi-agent reinforcement learning algorithm to find the transmission policies. Furthermore, we show the trade-off between the achievable throughput and the signaling required, and provide convergence guarantees for the proposed algorithm. Results show that even when the signaling overhead is taken into account, the proposed algorithm outperforms other approaches that do not consider cooperation among the nodes.
  • PDF
    A number of fundamental quantities in statistical signal processing and information theory can be expressed as integral functions of two probability density functions. Such quantities are called density functionals as they map density functions onto the real line. For example, information divergence functions measure the dissimilarity between two probability density functions and are particularly useful in a number of applications. Typically, estimating these quantities requires complete knowledge of the underlying distribution followed by multi-dimensional integration. Existing methods make parametric assumptions about the data distribution or use non-parametric density estimation followed by high-dimensional integration. In this paper, we propose a new alternative. We introduce the concept of "data-driven" basis functions - functions of distributions whose value we can estimate given only samples from the underlying distributions without requiring distribution fitting or direct integration. We derive a new data-driven complete basis that is similar to the deterministic Bernstein polynomial basis and develop two methods for performing basis expansions of functionals of two distributions. We also show that the new basis set allows us to approximate functions of distributions as closely as desired. Finally, we evaluate the methodology by developing data driven estimators for the Kullback-Leibler divergences and the Hellinger distance and by constructing tight data-driven bounds on the Bayes Error Rate.
  • PDF
    In this paper, we propose a method for acquiring accurate and timely channel state information (CSI) by leveraging full-duplex transmission. Specifically, we propose a mobile communication system in which base stations continuously transmit a pilot sequence in the uplink frequency band, while terminals use self-interference cancellation capabilities to obtain CSI at any time. Our proposal outperforms its half-duplex counterpart by at least 50% in terms of throughput while ensuring the same (or even lower) outage probability. Remarkably, it also outperforms using full duplex for downlink data transmission for low values of downlink bandwidth and received power.
  • PDF
    Several new constructions of 3-dimensional optical orthogonal codes are presented here. In each case the codes have ideal autocorrelation $\mathbf{ \lambda_a=0} $, and in all but one case a cross correlation of $ \mathbf{\lambda_c=1} $. All codes produced are optimal with respect to the applicable Johnson bound either presented or developed here. Thus, on one hand the codes are as large as possible, and on the other, the bound(s) are shown to be tight. All codes are constructed by using a particular automorphism (a Singer cycle) of $ \mathbf{ PG(k,q)} $, the finite projective geometry of dimension $ k $ over the field of order $ \mathbf{q} $, or by using an affine analogue in $ AG(k,q) $.
  • PDF
    The construction of permutation trinomials over finite fields attracts people's interest recently due to their simple form and some additional properties. Motivated by some results on the construction of permutation trinomials with Niho exponents, by constructing some new fractional polynomials that permute the set of the $(q+1)$-th roots of unity in $\mathbb F_{q^2}$, we present several classes of permutation trinomials with Niho exponents over $\mathbb F_{q^2}$, where $q=5^k$.
  • PDF
    Sampling in shift-invariant spaces is a realistic model for signals with smooth spectrum. In this paper, we consider phaseless sampling and reconstruction of real-valued signals in a shift-invariant space from their magnitude measurements on the whole Euclidean space and from their phaseless samples taken on a discrete set with finite sampling density. We introduce an undirected graph to a signal and use connectivity of the graph to characterize whether the signal can be determined, up to a sign, from its magnitude measurements on the whole Euclidean space. Under the local complement property assumption on a shift-invariant space, we find a discrete set with finite sampling density such that signals in the shift-invariant space, that are determined from their magnitude measurements on the whole Euclidean space, can be reconstructed in a stable way from their phaseless samples taken on that discrete set. In this paper, we also propose a reconstruction algorithm which provides a suboptimal approximation to the original signal when its noisy phaseless samples are available only. Finally, numerical simulations are performed to demonstrate the robust reconstruction of box spline signals from their noisy phaseless samples.
  • PDF
    We study a spectral initialization method that serves as a key ingredient in recent work on using efficient iterative algorithms for estimating signals in nonconvex settings. Unlike previous analysis in the literature, which is restricted to the phase retrieval setting and which provides only performance bounds, we consider arbitrary generalized linear sensing models and present a precise asymptotic characterization of the performance of the spectral method in the high-dimensional regime. Our analysis reveals a phase transition phenomenon that depends on the sampling ratio. When the ratio is below a minimum threshold, the estimates given by the spectral method are no better than a random guess drawn uniformly from the hypersphere; above a maximum threshold, however, the estimates become increasingly aligned with the target signal. The computational complexity of the spectral method is also markedly different in the two phases. Worked examples and numerical results are provided to illustrate and verify the analytical predictions. In particular, simulations show that our asymptotic formulas provide accurate predictions even at moderate signal dimensions.
  • PDF
    In this paper, energy-efficient transmission schemes achieving maximal throughput over a finite time interval are studied in a problem setting including energy harvests, data arrivals and channel variation. The goal is to express the offline optimal policy in a way that facilitates a good online solution. We express any throughput maximizing energy efficient offline schedule (EE-TM-OFF) explicitly in terms of water levels. This allows per-slot real-time evaluation of transmit power and rate decisions, using estimates of the associated offline water levels. To compute the online power level, we construct a stochastic dynamic program that incorporates the offline optimal solution as a stochastic process. We introduce the "Immediate Fill" metric which provides a lower bound on the efficiency of any online policy with respect to the corresponding optimal offline solution. The online algorithms obtained this way exhibit performance close to the offline optimal, not only in the long run but also in short problem horizons, deeming them suitable for practical implementations.
  • PDF
    We propose an intelligent proactive content caching scheme to reduce the energy consumption in wireless downlink. We consider an online social network (OSN) setting where new contents are generated over time, and remain \textitrelevant to the user for a random lifetime. Contents are downloaded to the user equipment (UE) through a time-varying wireless channel at an energy cost that depends on the channel state and the number of contents downloaded. The user accesses the OSN at random time instants, and consumes all the relevant contents. To reduce the energy consumption, we propose \textitproactive caching of contents under favorable channel conditions to a finite capacity cache memory. Assuming that the channel quality (or equivalently, the cost of downloading data) is memoryless over time slots, we show that the optimal caching policy, which may replace contents in the cache with shorter remaining lifetime with contents at the server that remain relevant longer, has certain threshold structure with respect to the channel quality. Since the optimal policy is computationally demanding in practice, we introduce a simplified caching scheme and optimize its parameters using policy search. We also present two lower bounds on the energy consumption. We demonstrate through numerical simulations that the proposed caching scheme significantly reduces the energy consumption compared to traditional reactive caching tools, and achieves close-to-optimal performance for a wide variety of system parameters.
  • PDF
    This paper considers the channel estimation (CE) and multi-user detection (MUD) problems in cloud radio access network (C-RAN). Assuming that active users are sparse in the network, we solve CE and MUD problems with compressed sensing (CS) technology to greatly reduce the long identification pilot overhead. A mixed L2,1-regularization functional for extended sparse group-sparsity recovery is proposed to exploit the inherently sparse property existing both in user activities and remote radio heads (RRHs) that active users are attached to. Empirical and theoretical guidelines are provided to help choosing tuning parameters which have critical effect on the performance of the penalty functional. To speed up the processing procedure, based on alternating direction method of multipliers and variable splitting strategy, an efficient algorithm is formulated which is guaranteed to be convergent. Numerical results are provided to illustrate the effectiveness of the proposed functional and efficient algorithm.
  • PDF
    We analyze and optimize a wireless system with energy transfer in the downlink and information transfer in the uplink, under quasi-static Nakagami-m fading. We consider ultra-reliable communication scenarios representative of the fifth-generation of wireless systems, with strict error and latency requirements. The error probability and delay are investigated, and an approximation for the former is given and validated through simulations. The numerical results demonstrate that there are optimum numbers of channels uses for both energy and information transfer for a given message length.
  • PDF
    We introduce an inequality which may be viewed as a generalization of both the Brascamp-Lieb inequality and its reverse (Barthe's inequality), and prove its information-theoretic (i.e.\ entropic) formulation. This result leads to a unified approach to functional inequalities such as the variational formula of Rényi entropy, hypercontractivity and its reverse, strong data processing inequalities, and transportation-cost inequalities, whose utility in the proofs of various coding theorems has gained growing popularity recently. We show that our information-theoretic setting is convenient for proving properties such as data processing, tensorization, convexity (Riesz-Thorin interpolation) and Gaussian optimality. In particular, we elaborate on a "doubling trick" used by Lieb and Geng-Nair to prove several results on Gaussian optimality. Several applications are discussed, including a generalization of the Brascamp-Lieb inequality involving Gaussian random transformations, the determination of Wyner's common information of vector Gaussian sources, and the achievable rate region of certain key generation problems in the case of vector Gaussian sources.
  • PDF
    Existing works on building a soliton transmission system only encode information using the imaginary part of the eigenvalue, which fails to make full use of the signal degree-of-freedoms. Motivated by this observation, we make the first step of encoding information using (discrete) spectral amplitudes by proposing analytical noise models for the spectral amplitudes of $N$-solitons ($N\geq 1$). To our best knowledge, this is the first work in building an analytical noise model for spectral amplitudes, which leads to many interesting information theoretic questions, such as channel capacity analysis, and has a potential of increasing the transmission rate. The noise statistics of the spectral amplitude of a soliton are also obtained without the Gaussian approximation.
  • PDF
    This paper concerns the problem of recovering an unknown but structured signal $x \in R^n$ from $m$ quadratic measurements of the form $y_r=|<a_r,x>|^2$ for $r=1,2,...,m$. We focus on the under-determined setting where the number of measurements is significantly smaller than the dimension of the signal ($m<<n$). We formulate the recovery problem as a nonconvex optimization problem where prior structural information about the signal is enforced through constrains on the optimization variables. We prove that projected gradient descent, when initialized in a neighborhood of the desired signal, converges to the unknown signal at a linear rate. These results hold for any constraint set (convex or nonconvex) providing convergence guarantees to the global optimum even when the objective function and constraint set is nonconvex. Furthermore, these results hold with a number of measurements that is only a constant factor away from the minimal number of measurements required to uniquely identify the unknown signal. Our results provide the first provably tractable algorithm for this data-poor regime, breaking local sample complexity barriers that have emerged in recent literature. In a companion paper we demonstrate favorable properties for the optimization problem that may enable similar results to continue to hold more globally (over the entire ambient space). Collectively these two papers utilize and develop powerful tools for uniform convergence of empirical processes that may have broader implications for rigorous understanding of constrained nonconvex optimization heuristics. The mathematical results in this paper also pave the way for a new generation of data-driven phase-less imaging systems that can utilize prior information to significantly reduce acquisition time and enhance image reconstruction, enabling nano-scale imaging at unprecedented speeds and resolutions.
  • PDF
    Millimeter wave communications rely on narrow-beam transmissions to cope with the strong signal attenuation at these frequencies, thus demanding precise beam alignment between transmitter and receiver. The communication overhead incurred to achieve beam alignment may become a severe impairment in mobile networks. This paper addresses the problem of optimizing beam alignment acquisition, with the goal of maximizing throughput. Specifically, the algorithm jointly determines the portion of time devoted to beam alignment acquisition, as well as, within this portion of time, the optimal beam search parameters, using the framework of Markov decision processes. It is proved that a bisection search algorithm is optimal, and that it outperforms exhaustive and iterative search algorithms proposed in the literature. The duration of the beam alignment phase is optimized so as to maximize the overall throughput. The numerical results show that the throughput, optimized with respect to the duration of the beam alignment phase, achievable under the exhaustive algorithm is 88.3% lower than that achievable under the bisection algorithm. Similarly, the throughput achievable by the iterative search algorithm for a division factor of 4 and 8 is, respectively, 12.8% and 36.4% lower than that achievable by the bisection algorithm.
  • PDF
    An inequality is derived for the correlation of two univariate functions operating on symmetric bivariate normal random variables. The inequality is a simple consequence of the Cauchy-Schwarz inequality.
  • PDF
    An outlier-resistance phase retrieval algorithm based on alternating direction method of multipliers (ADMM) is devised in this letter. Instead of the widely used least squares criterion that is only optimal for Gaussian noise environment, we adopt the least absolute deviation criterion to enhance the robustness against outliers. Considering both intensity- and amplitude-based observation models, the framework of ADMM is developed to solve the resulting non-differentiable optimization problems. It is demonstrated that the core subproblem of ADMM is the proximity operator of the L1-norm, which can be computed efficiently by soft-thresholding in each iteration. Simulation results are provided to validate the accuracy and efficiency of the proposed approach compared to the existing schemes.

Recent comments

Samad Khabbazi Oskouei Sep 05 2016 11:34 UTC

I think that we have missed the "semi-" at the conclusion. Because, the proof of the theorem 4.3 is based on the using universal semi-density matrix concept which is not computable. The semi-computability concept used here is like the Kolmogorov complexity which is not computable and so the Cubic co

...(continued)
Toby Cubitt Sep 01 2016 11:14 UTC

I could well be missing something. But as far as I could tell from a rather quick read through the paper, all they show is that the quantum capacity of a channel with computable matrix elements is given by the regularised coherent information optimised over input ensembles with computable matrix ele

...(continued)
Māris Ozols Aug 30 2016 17:52 UTC

Do I understand correctly that this paper claims to show that quantum capacity is computable?

> After defining the algorithmic quantum capacity we have proved that it
> equals the standard one. Furthermore we have shown that it is
> computable.

Richard Kueng Jul 28 2015 07:01 UTC

fyi: our quantum implications are presented in Subsection 2.2 (pp 7-9).

Marco Tomamichel May 31 2015 22:07 UTC

Thanks for the comment! This is a good idea, I will do that in the next arXiv version.

Patrick Hayden May 28 2015 17:31 UTC

Wonderful! I've been waiting for a book like this for a while now! Thanks, Marco.

I do have one trivial comment from a 30 second preliminary scan, though: please consider typesetting the proofs with a font size matching the main text. If us readers are already squinting hard trying to understand

...(continued)
Marco Tomamichel Apr 02 2015 03:21 UTC

This is a preliminary version and I am happy to incorporate feedback I receive in the coming month. Any comments are welcome.

Māris Ozols Mar 17 2015 11:00 UTC

The strange equation is supposed to look like this:
$$f(\sqrt{a} X + \sqrt{1-a} Y) \geq a f(X) + (1-a) f(Y) \quad \forall a \in [0,1]$$

Yuanzhu Aug 02 2014 04:21 UTC

This algorithm is from Wu's list decoding algorithm.