# Information Theory (cs.IT)

• We combine conditional variational autoencoders (VAE) with adversarial censoring in order to learn invariant representations that are disentangled from nuisance/sensitive variations. In this method, an adversarial network attempts to recover the nuisance variable from the representation, which the VAE is trained to prevent. Conditioning the decoder on the nuisance variable enables clean separation of the representation, since they are recombined for model learning and data reconstruction. We show this natural approach is theoretically well-founded with information-theoretic arguments. Experiments demonstrate that this method achieves invariance while preserving model learning performance, and results in visually improved performance for style transfer and generative sampling tasks.
• Visible light communications (VLC) is an emerging field in technology and research. Estimating the channel taps is a major requirement for designing reliable communication systems. Due to the nonlinear characteristics of the VLC channel those parameters cannot be derived easily. They can be calculated by means of software simulation. In this work, a novel methodology is proposed for the prediction of channel parameters using neural networks. Measurements conducted in a controlled experimental setup are used to train neural networks for channel tap prediction. Our experiment results indicate that neural networks can be effectively trained to predict channel taps under different environmental conditions.
• Hybrid precoding design is challenging for millimeter-wave (mmWave) massive MIMO. Most prior hybrid precoding schemes are designed to maximize the sum spectral efficiency (SSE), while seldom investigate the bit-error-rate (BER). Therefore, we propose an over-sampling codebook (OSC)-based hybrid minimum sum-mean-square-error (min-SMSE) precoding scheme for mmWave multi-user three-dimensional (3D)-MIMO systems to optimize the BER, where multi-user transmission with multiple data streams for each user is considered. Specifically, given the effective baseband channel consisting of the real channel and analog precoding, we first design the digital precoder/combiner based on min-SMSE criterion to optimize the BER. To further reduce the SMSE between the transmit and receive signals, we propose an OSC-based joint analog precoder/combiner (JAPC) design. Simulation results show that the proposed scheme can achieve the better performance than its conventional counterparts.
• May 22 2018 cs.IT cs.LG math.IT stat.ML arXiv:1805.07631v1
In this paper we consider Multiple-Input-Multiple-Output (MIMO) detection using deep neural networks. We introduce two different deep architectures: a standard fully connected multi-layer network, and a Detection Network (DetNet) which is specifically designed for the task. The structure of DetNet is obtained by unfolding the iterations of a projected gradient descent algorithm into a network. We compare the accuracy and runtime complexity of the purposed approaches and achieve state-of-the-art performance while maintaining low computational requirements. Furthermore, we manage to train a single network to detect over an entire distribution of channels. Finally, we consider detection with soft outputs and show that the networks can easily be modified to produce soft decisions.
• Network embeddings map the nodes of a given network into $d$-dimensional Euclidean space $\mathbb{R}^d$. Ideally, this mapping is such that similar' nodes are mapped onto nearby points, such that the embedding can be used for purposes such as link prediction (if similar' means being more likely to be connected') or classification (if similar' means `being more likely to have the same label'). In recent years various methods for network embedding have been introduced. These methods all follow a similar strategy, defining a notion of similarity between nodes (typically deeming nodes more similar if they are nearby in the network in some metric), a distance measure in the embedding space, and minimizing a loss function that penalizes large distances for similar nodes or small distances for dissimilar nodes. A difficulty faced by existing methods is that certain networks are fundamentally hard to embed due to their structural properties, such as (approximate) multipartiteness, certain degree distributions, or certain kinds of assortativity. Overcoming this difficulty, we introduce a conceptual innovation to the literature on network embedding, proposing to create embeddings that maximally add information with respect to such structural properties (e.g. node degrees, block densities, etc.). We use a simple Bayesian approach to achieve this, and propose a block stochastic gradient descent algorithm for fitting it efficiently. Finally, we demonstrate that the combination of information such structural properties and a Euclidean embedding provides superior performance across a range of link prediction tasks. Moreover, we demonstrate the potential of our approach for network visualization.
• May 22 2018 cs.IT math.IT arXiv:1805.08144v1
For a Distributed Storage System (DSS), the \textitFractional Repetition (FR) code is a class in which replicas of encoded data packets are stored on distributed chunk servers, where the encoding is done using the Maximum Distance Separable (MDS) code. The FR codes allow for exact uncoded repair with minimum repair bandwidth. In this paper, FR codes are constructed using finite binary sequences. The condition for universally good FR codes is calculated on such sequences. For some sequences, the universally good FR codes are explored.
• In this paper, we introduce a new measure of correlation for bipartite quantum states. This measure depends on a parameter $\alpha$, and is defined in terms of vector-valued $L_p$-norms. The measure is within a constant of the exponential of $\alpha$-Rényi mutual information, and reduces to the trace norm (total variation distance) for $\alpha=1$. We will prove some decoupling type theorems in terms of this measure of correlation, and present some applications in privacy amplification as well as in bounding the random coding exponents. In particular, we establish a bound on the secrecy exponent of the wiretap channel (under the total variation metric) in terms of the $\alpha$-Rényi mutual information according to \emphCsiszár's proposal.
• This letter studies the practical design of secure transmissions without knowing eavesdropper's channel state information (ECSI). An ECSI-irrelevant metric is introduced to quantize the intrinsic anti-eavesdropping ability (AEA) that the transmitter has on confronting the eavesdropper via secrecy encoding together with artificial-noise-aided signaling. Non-adaptive and adaptive transmission schemes are proposed to maximize the AEA with the optimal encoding rates and power allocation presented in closed-form expressions. Analyses and numerical results show that maximizing the AEA is equivalent to minimizing the secrecy outage probability (SOP) for the worst case by ignoring eavesdropper's receiver noise. Therefore, the AEA is a useful alternative to the SOP for assessing and designing secure transmissions when the ECSI cannot be prior known.
• Light-fidelity (LiFi) is a networked optical wireless communication (OWC) solution for high-speed indoor connectivity for fixed and mobile optical communications. Unlike conventional radio frequency wireless systems, the OWC channel is not isotropic meaning that the device orientation affects the channel gain significantly particularly for mobile users. However, due to the lack of a proper model for device orientation, many studies have assumed that the receiver is vertically upward and fixed. In this paper, a novel model for device orientation based on experimental measurements of forty participants has been proposed. It is shown that the probability density function (PDF) of the polar angle can be modeled either based on Laplace (for static users) or Gaussian (for mobile users) distribution. In addition, a closed-form expression is obtained for the PDF of the cosine of the incidence angle based on which the line-of-sight (LOS) channel gain is described in OWC channels. An approximation of this PDF based on the truncated Laplace is proposed and the accuracy of this approximation is confirmed by the Kolmogorov-Smirnov distance (KSD). Moreover, the statistics of the LOS channel gain are calculated and the random orientation of a UE is modeled as slow fading. The influence of the random orientation on signal-to-noise-ratio (SNR) performance of OWC systems has been evaluated. Finally, an orientation-based random waypoint (ORWP) mobility model is proposed by considering the random orientation of the UE during the user's movement. The performance of ORWP is assessed on the handover rate and it is shown that it is important to take the random orientation into account.
• This paper considers multi-cell Massive MIMO (multiple-input multiple-output) systems where the channels are spatially correlated Rician fading. The channel model is composed of a deterministic line-of-sight (LoS) path and a stochastic non-line-of-sight (NLoS) component describing a practical spatially correlated multipath environment. We derive the statistical properties of the minimum mean squared error (MMSE), element-wise MMSE (EW-MMSE), and least-square (LS) channel estimates for this model. Using these estimates for maximum ratio (MR) combining and precoding, rigorous closed-form uplink (UL) and downlink (DL) spectral efficiency (SE) expressions are derived and analyzed. The asymptotic SE behavior when using the different channel estimators are also analyzed. Numerical results show that the SE is higher when using the MMSE estimator than the other estimators, and the performance gap increases with the number of antennas.
• This paper considers the uplink (UL) of a multi-cell Massive MIMO (multiple-input multiple-output) system with spatially correlated Rician fading channels. The channel model is composed of a deterministic line-of-sight (LoS) path and a stochastic non-line-of-sight (NLoS) component describing a spatially correlated multipath environment. We derive the statistical properties of the minimum mean squared error (MMSE) and least-square (LS) channel estimates for this model. Using these estimates for maximum ratio (MR) combining, rigorous closed-form UL spectral efficiency (SE) expressions are derived. Numerical results show that the SE is higher when using the MMSE estimator than the LS estimator, and the performance gap increases with the number of antennas. Moreover, Rician fading provides higher achievable SEs than Rayleigh fading since the LoS path improves the sum SE.
• This paper analyzes how the distortion created by hardware impairments in a multiple-antenna base station affects the uplink spectral efficiency (SE), with focus on Massive MIMO. The distortion is correlated across the antennas, but has been often approximated as uncorrelated to facilitate (tractable) SE analysis. To determine when this approximation is accurate, basic properties of the distortion correlation are first uncovered. Then, we focus on third-order non-linearities and prove analytically and numerically that the correlation can be neglected in the SE analysis when there are many users. In i.i.d. Rayleigh fading with equal signal-to-noise ratios, this occurs when having five users.
• In many applications of neural network, it is common to introduce huge amounts of input categorical features, as well as output labels. However, since the required network size should have rapid growth with respect to the dimensions of input and output space, there exists huge cost in both computation and memory resources. In this paper, we present a novel method called category coding (CC), where the design philosophy follows the principle of minimal collision to reduce the input and output dimension effectively. In addition, we introduce three types of category coding based on different Euclidean domains. Experimental results show that all three proposed methods outperform the existing state-of-the-art coding methods, such as standard cut-off and error-correct output coding (ECOC) methods.
• In this paper, we consider the generalized phase retrieval from affine measurements. This problem aims to recover signals ${\mathbf x} \in {\mathbb F}^d$ from the affine measurements $y_j=\norm{M_j^*\vx +{\mathbb b}_j}^2,\; j=1,\ldots,m,$ where $M_j \in {\mathbb F}^{d\times r}, {\mathbf b}_j\in {\mathbb F}^{r}, {\mathbb F}\in \{{\mathbb R},{\mathbb C}\}$ and we call it as \em generalized affine phase retrieval. We develop a framework for generalized affine phase retrieval with presenting necessary and sufficient conditions for $\{(M_j,{\mathbf b}_j)\}_{j=1}^m$ having generalized affine phase retrieval property. We also establish results on minimal measurement number for generalized affine phase retrieval. Particularly, we show if $\{(M_j,{\mathbf b}_j)\}_{j=1}^m \subset {\mathbb F}^{d\times r}\times {\mathbb F}^{r}$ has generalized affine phase retrieval property, then $m\geq d+\floor{d/r}$ for ${\mathbb F}={\mathbb R}$ ($m\geq 2d+\floor{d/r}$ for ${\mathbb F}={\mathbb C}$ ). We also show that the bound is tight provided $r\mid d$. These results imply that one can reduce the measurement number by raising $r$, i.e. the rank of $M_j$. This highlights a notable difference between generalized affine phase retrieval and generalized phase retrieval. Furthermore, using tools of algebraic geometry, we show that $m\geq 2d$ (resp. $m\geq 4d-1$) generic measurements ${\mathcal A}=\{(M_j,b_j)\}_{j=1}^m$ have the generalized phase retrieval property for ${\mathbb F}={\mathbb R}$ (resp. ${\mathbb F}={\mathbb C}$).
• We introduce a novel blind (noncoherent) communication scheme, called modulation on conjugate-reciprocal zeros (MOCZ), to reliably transmit short binary packets over unknown finite impulse response systems as used, for example, to model underspread wireless multipath channels. In MOCZ, the information is modulated onto the zeros of the transmitted signals $z-$transform. In the absence of additive noise, the zero structure of the signal is perfectly preserved at the receiver, no matter what the channel impulse response (CIR) is. Furthermore, by a proper selection of the zeros, we show that MOCZ is not only invariant to the CIR, but also robust against additive noise. Starting with the maximum-likelihood estimator, we define a low complexity and reliable decoder and compare it to various state-of-the art noncoherent schemes.
• May 22 2018 cs.IT math.IT arXiv:1805.07822v1
Multi-way and device-to-device (D2D) communications are currently considered for the design of future communication systems. Unmanned aerial vehicles (UAVs) can be effectively deployed to extend the communication range of D2D networks. To model the UAV-D2D interaction, we study a multi-antenna multi-way channel with two D2D users and an intermittently available UAV node. The performance in terms of sum-rate of various transmission schemes is compared. Numerical results show that for different ground environments, the scheme based on a combination of interference alignment, zero-forcing and erasure-channel treatment outperforms other schemes at low, medium and high SNRs and thus represents a viable transmission strategy for UAV-aided multi-way D2D networks.
• In this paper we consider the uplink of a massive MIMO communication system using 5G New Radio-compliant multiple access, which is to co-exist with a radar system using the same frequency band. We propose a system model taking into account the reverberation (clutter) produced by the radar system at the massive MIMO receiver. Then, we propose several linear receivers for uplink data-detection, ranging by the simple channel-matched beamformer to the zero-forcing and linear minimum mean square error receivers for clutter disturbance rejection. Our results show that the clutter may have a strong effect on the performance of the cellular communication system, but the use of large-scale antenna arrays at the base station is key to provide increased robustness against it, at least as far as data-detection is concerned.
• One-bit compressive sensing is an extended version of compressed sensing in which the sparse signal of interest can be recovered from extremely quantized measurements. Namely, only the sign of each measurement is available to us. There exist may practical application in which the underlying signal is not sparse directly, but it can be represented in a redundant dictionary. Apart from that, one can refine the sampling procedure by using profitable information lying in previous samples. this information can be employed to reduce the required number of measurements for exact recovery by adaptive sampling schemes. In this work, we proposed an adaptive algorithm that exploits the available information in previous samples. The proof uses the recent geometric concepts in high dimensional estimation. we show through rigorous and numerical analysis that our algorithm considerably outperforms non-adaptive approaches. Further, it reaches the optimal error rate from quantized measurements.
• Circularly pulse-shaped orthogonal frequency division multiplexing (CPS-OFDM) is one of the most promising 5G waveforms that addresses two physical layer signal requirements of low out-of-subband emission (OSBE) and low peak-to-average power ratio (PAPR) with flexibility in parameter adaptation. In this paper, a constellation shaping optimization method is proposed to further reduce the cubic metric (CM) of CPS-OFDM signals for the case that demands rather high power amplifier (PA) efficiency at the transmitter. Simulation results demonstrate the superiority of the proposed scheme in CM reduction, and the corresponding benefits of spectral regrowth mitigation and spectral efficiency improvement.
• Approximate outage probability expressions are derived for systems employing maximum ratio combining, when both the desired signal and the interfering signals are subjected to $\eta-\mu$ fading, with the interferers having unequal power. The approximations are in terms of the Appell Function and Gauss hypergeometric function. A close match is observed between the outage probability result obtained through the derived analytical expression and the one obtained through Monte-Carlo simulations.
• In this letter, the performance analysis of physical layer security over Fisher-Snedecor $\mathcal{F}$ fading channels is investigated. In particular, the average secrecy capacity (ASC), the secure outage probability (SOP), the lower bound of the SOP (SOP$^L$), and the strictly positive secure capacity (SPSC) are derived in exact closed-from expressions. The Fisher-Snedecor $\mathcal{F}$ fading channel is a composite of multipath/shadowed fading that are represented by the Nakagami-$m$ distribution. Moreover, it provides close results to the practical measurements than the generalised $K$ ($K_G$) fading channels. To validate our analysis, the numerical results are affirmed by the Monte Carlo simulations.
• This paper proposes efficient algorithms for accurate recovery of direction-of-arrival (DoA) of sources from single-snapshot measurements using compressed beamforming (CBF). In CBF, the conventional sensor array signal model is cast as an underdetermined complex-valued linear regression model and sparse signal recovery methods are used for solving the DoA finding problem. We develop a complex-valued pathwise weighted elastic net (c-PW-WEN) algorithm that finds solutions at knots of penalty parameter values over a path (or grid) of EN tuning parameter values. c-PW-WEN also computes Lasso or weighted Lasso in its path. We then propose a sequential adaptive EN (SAEN) method that is based on c-PW-WEN algorithm with adaptive weights that depend on the previous solution. Extensive simulation studies illustrate that SAEN improves the probability of exact recovery of true support compared to conventional sparse signal recovery approaches such as Lasso, elastic net or orthogonal matching pursuit in several challenging multiple target scenarios. The effectiveness of SAEN is more pronounced in the presence of high mutual coherence.
• We propose a tunable location-dependent base station (BS) cooperation scheme by partitioning the plane into three regions: the cell centers, cell edges and cell corners. The area fraction of each region is tuned by the cooperation level $\gamma$ ranging from 0 to 1. Depending on the region a user resides in, he/she receives no cooperation, two-BS cooperation or three-BS cooperation. Here, we use a Poisson point process (PPP) to model BS locations and study a non-coherent joint transmission scheme, $\textit{i.e.}$, selected BSs jointly serve one user in the absence of channel state information (CSI). For the proposed scheme, we examine its performance as a function of the cooperation level using tools from stochastic geometry. We derive an analytical expression for the signal-to-interference ratio (SIR) distribution and its approximation based on the asymptotic SIR gain, along with the characterization of the normalized spectral efficiency per BS. Our result suggests that the proposed scheme with a moderate cooperation level can improve the SIR performance while maintaining the normalized spectral efficiency.
• Delay-coordinate embedding is a powerful, time-tested mathematical framework for reconstructing the dynamics of a system from a series of scalar observations. Most of the associated theory and heuristics are overly stringent for real-world data, however, and real-time use is out of the question due to the expert human intuition needed to use these heuristics correctly. The approach outlined in this thesis represents a paradigm shift away from that traditional approach. I argue that perfect reconstructions are not only unnecessary for the purposes of delay-coordinate based forecasting, but that they can often be less effective than reduced-order versions of those same models. I demonstrate this using a range of low- and high-dimensional dynamical systems, showing that forecast models that employ imperfect reconstructions of the dynamics---i.e., models that are not necessarily true embeddings---can produce surprisingly accurate predictions of the future state of these systems. I develop a theoretical framework for understanding why this is so. This framework, which combines information theory and computational topology, also allows one to quantify the amount of predictive structure in a given time series, and even to choose which forecast method will be the most effective for those data.
• Time-division multiplexed (TDM) channel sounders, in which a single RF chain is connected sequentially via an electronic switch to different elements of an array, are widely used for the measurement of double-directional/MIMO propagation channels. This paper investigates the impact of array switching patterns on the accuracy of parameter estimation of multipath components (MPC) for a time-division multiplexed (TDM) channel sounder. The commonly-used sequential (uniform) switching pattern poses a fundamental limit on the number of antennas that a TDM channel sounder can employ in fast time-varying channels. We thus aim to design improved patterns that relax these constraints. To characterize the performance, we introduce a novel spatio-temporal ambiguity function, which can handle the non-idealities of real-word arrays. We formulate the sequence design problem as an optimization problem and propose an algorithm based on simulated annealing to obtain the optimal sequence. As a result we can extend the estimation range of Doppler shifts by eliminating ambiguities in parameter estimation. We show through Monte Carlo simulations that the root mean square errors of both direction of departure and Doppler are reduced significantly with the new switching sequence. Results are also verified with actual vehicle-to-vehicle (V2V) channel measurements.
• Error-controlled lossy compression has been studied for years because of extremely large volumes of data being produced by today's scientific simulations. None of existing lossy compressors, however, allow users to fix the peak signal-to-noise ratio (PSNR) during compression, although PSNR has been considered as one of the most significant indicators to assess compression quality. In this paper, we propose a novel technique providing a fixed-PSNR lossy compression for scientific data sets. We implement our proposed method based on the SZ lossy compression framework and release the code as an open-source toolkit. We evaluate our fixed-PSNR compressor on three real-world high-performance computing data sets. Experiments show that our solution has a high accuracy in controlling PSNR, with an average deviation of 0.1 ~ 5.0 dB on the tested data sets.
• Region-based classification of PolSAR data can be effectively performed by seeking for the assignment that minimizes a distance between prototypes and segments. Silva et al (2013) used stochastic distances between complex multivariate Wishart models which, differently from other measures, are computationally tractable. In this work we assess the robustness of such approach with respect to errors in the training stage, and propose an extension that alleviates such problems. We introduce robustness in the process by incorporating a combination of radial basis kernel functions and stochastic distances with Support Vector Machines (SVM). We consider several stochastic distances between Wishart: Bhatacharyya, Kullback-Leibler, Chi-Square, Rényi, and Hellinger. We perform two case studies with PolSAR images, both simulated and from actual sensors, and different classification scenarios to compare the performance of Minimum Distance and SVM classification frameworks. With this, we model the situation of imperfect training samples. We show that SVM with the proposed kernel functions achieves better performance with respect to Minimum Distance, at the expense of more computational resources and the need of parameter tuning. Code and data are provided for reproducibility.

gae Jul 26 2017 21:19 UTC

For those interested in the literature on teleportation simulation of quantum channels, a detailed and *comprehensive* review is provided in Supplementary Note 8 of https://images.nature.com/original/nature-assets/ncomms/2017/170426/ncomms15043/extref/ncomms15043-s1.pdf
The note describes well the t

...(continued)
Stefano Pirandola May 05 2017 05:45 UTC

Today I have seen on the arXiv the version 2 of this paper on quantum reading. I am sorry to say that this revision still misses to acknowledge important contributions from previous works, especially in relation to the methods on channel simulation and teleportation that are crucial for its claims.

...(continued)
Samad Khabbazi Oskouei Sep 05 2016 11:34 UTC

I think that we have missed the "semi-" at the conclusion. Because, the proof of the theorem 4.3 is based on the using universal semi-density matrix concept which is not computable. The semi-computability concept used here is like the Kolmogorov complexity which is not computable and so the Cubic co

...(continued)
Toby Cubitt Sep 01 2016 11:14 UTC

I could well be missing something. But as far as I could tell from a rather quick read through the paper, all they show is that the quantum capacity of a channel with computable matrix elements is given by the regularised coherent information optimised over input ensembles with computable matrix ele

...(continued)
Māris Ozols Aug 30 2016 17:52 UTC

Do I understand correctly that this paper claims to show that quantum capacity is computable?

> After defining the algorithmic quantum capacity we have proved that it
> equals the standard one. Furthermore we have shown that it is
> computable.

Richard Kueng Jul 28 2015 07:01 UTC

fyi: our quantum implications are presented in Subsection 2.2 (pp 7-9).

Marco Tomamichel May 31 2015 22:07 UTC

Thanks for the comment! This is a good idea, I will do that in the next arXiv version.

Patrick Hayden May 28 2015 17:31 UTC

Wonderful! I've been waiting for a book like this for a while now! Thanks, Marco.

I do have one trivial comment from a 30 second preliminary scan, though: please consider typesetting the proofs with a font size matching the main text. If us readers are already squinting hard trying to understand

...(continued)
Marco Tomamichel Apr 02 2015 03:21 UTC

This is a preliminary version and I am happy to incorporate feedback I receive in the coming month. Any comments are welcome.