results for au:Shi_Z in:cs

- Though deep neural networks (DNNs) achieve remarkable performances in many artificial intelligence tasks, the lack of training instances remains a notorious challenge. As the network goes deeper, the generalization accuracy decays rapidly in the situation of lacking massive amounts of training data. In this paper, we propose novel deep neural network structures that can be inherited from all existing DNNs with almost the same level of complexity, and develop simple training algorithms. We show our paradigm successfully resolves the lack of data issue. Tests on the CIFAR10 and CIFAR100 image recognition datasets show that the new paradigm leads to 20$\%$ to $30\%$ relative error rate reduction compared to their base DNNs. The intuition of our algorithms for deep residual network stems from theories of the partial differential equation (PDE) control problems. Code will be made available.
- In this paper, a novel weighted nonlocal total variation (WNTV) method is proposed. Compared to the classical nonlocal total variation methods, our method modifies the energy functional to introduce a weight to balance between the labeled sets and unlabeled sets. With extensive numerical examples in semi-supervised clustering, image inpainting and image colorization, we demonstrate that WNTV provides an effective and efficient method in many image processing and machine learning problems.
- J-vector has been proved to be very effective in text-dependent speaker verification with short-duration speech. However, the current state-of-the-art back-end classifiers, e.g. joint Bayesian model, cannot make full use of such deep features. In this paper, we generalize the standard joint Bayesian approach to model the multi-faceted information in the j-vector explicitly and jointly. In our generalization, the j-vector was modeled as a result derived by a generative Double Joint Bayesian (DoJoBa) model, which contains several kinds of latent variables. With DoJoBa, we are able to explicitly build a model that can combine multiple heterogeneous information from the j-vectors. In verification step, we calculated the likelihood to describe whether the two j-vectors having consistent labels or not. On the public RSR2015 data corpus, the experimental results showed that our approach can achieve 0.02\% EER and 0.02\% EER for impostor wrong and impostor correct cases respectively.
- Nov 20 2017 cs.CV arXiv:1711.06382v1This paper proposes a generalized framework with joint normalization which learns lower-dimensional subspaces with maximum discriminative power by making use of the Riemannian geometry. In particular, we model the similarity/dissimilarity between subspaces using various metrics defined on Grassmannian and formulate dimen-sionality reduction as a non-linear constraint optimization problem considering the orthogonalization. To obtain the linear mapping, we derive the components required to per-form Riemannian optimization (e.g., Riemannian conju-gate gradient) from the original Grassmannian through an orthonormal projection. We respect the Riemannian ge-ometry of the Grassmann manifold and search for this projection directly from one Grassmann manifold to an-other face-to-face without any additional transformations. In this natural geometry-aware way, any metric on the Grassmann manifold can be resided in our model theoreti-cally. We have combined five metrics with our model and the learning process can be treated as an unconstrained optimization problem on a Grassmann manifold. Exper-iments on several datasets demonstrate that our approach leads to a significant accuracy gain over state-of-the-art methods.
- Nov 08 2017 cs.CV arXiv:1711.02254v3Hand gesture recognition has long been a hot topic in human computer interaction. Traditional camera-based hand gesture recognition systems cannot work properly under dark circumstances. In this paper, a Doppler Radar based hand gesture recognition system using convolutional neural networks is proposed. A cost-effective Doppler radar sensor with dual receiving channels at 5.8GHz is used to acquire a big database of four standard gestures. The received hand gesture signals are then processed with time-frequency analysis. Convolutional neural networks are used to classify different gestures. Experimental results verify the effectiveness of the system with an accuracy of 98%. Besides, related factors such as recognition distance and gesture scale are investigated.
- Traditional vision-based hand gesture recognition systems is limited under dark circumstances. In this paper, we build a hand gesture recognition system based on microwave transceiver and deep learning algorithm. A Doppler radar sensor with dual receiving channels at 5.8GHz is used to acquire a big database of hand gestures signals. The received hand gesture signals are then processed with time-frequency analysis. Based on these big databases of hand gesture, we propose a new machine learning architecture called deformable deep convolutional generative adversarial network. Experimental results show the new architecture can upgrade the recognition rate by 10% and the deformable kernel can reduce the testing time cost by 30%.
- Nov 02 2017 cs.CR arXiv:1711.00232v1Wearable devices enable users to collect health data and share them with healthcare providers for improved health service. Since health data contain privacy-sensitive information, unprotected data release system may result in privacy leakage problem. Most of the existing work use differential privacy for private data release. However, they have limitations in healthcare scenarios because they do not consider the unique features of health data being collected from wearables, such as continuous real-time collection and pattern preservation. In this paper, we propose Re-DPoctor, a real-time health data releasing scheme with $w$-day differential privacy where the privacy of health data collected from any consecutive $w$ days is preserved. We improve utility by using a specially-designed partition algorithm to protect the health data patterns. Meanwhile, we improve privacy preservation by applying newly proposed adaptive sampling technique and budget allocation method. We prove that Re-DPoctor satisfies $w$-day differential privacy. Experiments on real health data demonstrate that our method achieves better utility with strong privacy guarantee than existing state-of-the-art methods.
- Based on a natural connection between ResNet and transport equation or its characteristic equation, we propose a continuous flow model for both ResNet and plain net. Through this continuous model, a ResNet can be explicitly constructed as a refinement of a plain net. The flow model provides an alternative perspective to understand phenomena in deep neural networks, such as why it is necessary and sufficient to use 2-layer blocks in ResNets, why deeper is better, and why ResNets are even deeper, and so on. It also opens a gate to bring in more tools from the huge area of differential equations.
- In this paper, we formulate the deep residual network (ResNet) as a control problem of transport equation. In ResNet, the transport equation is solved along the characteristics. Based on this observation, deep neural network is closely related to the control problem of PDEs on manifold. We propose several models based on transport equation and Hamilton-Jacobi equation. The discretization of these PDEs on point cloud is also discussed.
- Aug 09 2017 cs.CV arXiv:1708.02459v1We propose to model complex visual scenes using a non-parametric Bayesian model learned from weakly labelled images abundant on media sharing sites such as Flickr. Given weak image-level annotations of objects and attributes without locations or associations between them, our model aims to learn the appearance of object and attribute classes as well as their association on each object instance. Once learned, given an image, our model can be deployed to tackle a number of vision problems in a joint and coherent manner, including recognising objects in the scene (automatic object annotation), describing objects using their attributes (attribute prediction and association), and localising and delineating the objects (object detection and semantic segmentation). This is achieved by developing a novel Weakly Supervised Markov Random Field Stacked Indian Buffet Process (WS-MRF-SIBP) that models objects and attributes as latent factors and explicitly captures their correlations within and across superpixels. Extensive experiments on benchmark datasets demonstrate that our weakly supervised model significantly outperforms weakly supervised alternatives and is often comparable with existing strongly supervised models on a variety of tasks including semantic segmentation, automatic image annotation and retrieval based on object-attribute associations.
- This paper develops an interference aware design for cooperative hybrid automatic repeat request (HARQ) assisted non-orthogonal multiple access (NOMA) scheme for large-scale device-to-device (D2D) networks. Specifically, interference aware rate selection and power allocation are considered to maximize long term average throughput (LTAT) and area spectral efficiency (ASE). The design framework is based on stochastic geometry that jointly accounts for the spatial interference correlation at the NOMA receivers as well as the temporal interference correlation across HARQ transmissions. It is found that ignoring the effect of the aggregate interference, or overlooking the spatial and temporal correlation in interference, highly overestimates the NOMA performance and produces misleading design insights. An interference oblivious selection for the power and/or transmission rates leads to violating the network outage constraints. To this end, the results demonstrate the effectiveness of NOMA transmission and manifest the importance of the cooperative HARQ to combat the negative effect of the network aggregate interference. For instance, comparing to the non-cooperative HARQ assisted NOMA, the proposed scheme can yield an outage probability reduction by $32$%. Furthermore, an interference aware optimal design that maximizes the LTAT given outage constraints leads to $47$% throughput improvement over HARQ-assisted orthogonal multiple access (OMA) scheme.
- Jul 04 2017 cs.CL arXiv:1707.00248v1Neural word segmentation has attracted more and more research interests for its ability to alleviate the effort of feature engineering and utilize the external resource by the pre-trained character or word embeddings. In this paper, we propose a new neural model to incorporate the word-level information for Chinese word segmentation. Unlike the previous word-based models, our model still adopts the framework of character-based sequence labeling, which has advantages on both effectiveness and efficiency at the inference stage. To utilize the word-level information, we also propose a new long short-term memory (LSTM) architecture over directed acyclic graph (DAG). Experimental results demonstrate that our model leads to better performances than the baseline models.
- Jun 20 2017 cs.CV arXiv:1706.05952v1We address the problem of localisation of objects as bounding boxes in images and videos with weak labels. This weakly supervised object localisation problem has been tackled in the past using discriminative models where each object class is localised independently from other classes. In this paper, a novel framework based on Bayesian joint topic modelling is proposed, which differs significantly from the existing ones in that: (1) All foreground object classes are modelled jointly in a single generative model that encodes multiple object co-existence so that "explaining away" inference can resolve ambiguity and lead to better learning and localisation. (2) Image backgrounds are shared across classes to better learn varying surroundings and "push out" objects of interest. (3) Our model can be learned with a mixture of weakly labelled and unlabelled data, allowing the large volume of unlabelled images on the Internet to be exploited for learning. Moreover, the Bayesian formulation enables the exploitation of various types of prior knowledge to compensate for the limited supervision offered by weakly labelled data, as well as Bayesian domain adaptation for transfer learning. Extensive experiments on the PASCAL VOC, ImageNet and YouTube-Object videos datasets demonstrate the effectiveness of our Bayesian joint model for weakly supervised object localisation.
- Jun 13 2017 cs.CV arXiv:1706.03725v1Learning semantic attributes for person re-identification and description-based person search has gained increasing interest due to attributes' great potential as a pose and view-invariant representation. However, existing attribute-centric approaches have thus far underperformed state-of-the-art conventional approaches. This is due to their non-scalable need for extensive domain (camera) specific annotation. In this paper we present a new semantic attribute learning approach for person re-identification and search. Our model is trained on existing fashion photography datasets -- either weakly or strongly labelled. It can then be transferred and adapted to provide a powerful semantic description of surveillance person detections, without requiring any surveillance domain supervision. The resulting representation is useful for both unsupervised and supervised person re-identification, achieving state-of-the-art and near state-of-the-art performance respectively. Furthermore, as a semantic representation it allows description-based person search to be integrated within the same framework.
- In this paper, outage performance of hybrid automatic repeat request with incremental redundancy (HARQ-IR) is analyzed. Unlike prior analyses, time-correlated Nakagami-$m$ fading channel is considered. The outage analysis thus involves the probability distribution analysis of a product of multiple correlated shifted Gamma random variables and is more challenging than prior analyses. Based on the finding of the conditional independence of the received signal-to-noise ratios (SNRs), the outage probability is exactly derived by using conditional Mellin transform. Specifically, the outage probability of HARQ-IR under time-correlated Nakagami-$m$ fading channels can be written as a weighted sum of outage probabilities of HARQ-IR over independent Nakagami fading channels, where the weightings are determined by a negative multinomial distribution. This result enables not only an efficient truncation approximation of the outage probability with uniform convergence but also asymptotic outage analysis to further extract clear insights which have never been discovered for HARQ-IR even under fast fading channels. The asymptotic outage probability is then derived in a simple form which clearly quantifies the impacts of transmit powers, channel time correlation and information transmission rate. It is proved that the asymptotic outage probability is an inverse power function of the product of transmission powers in all HARQ rounds, an increasing function of the channel time correlation coefficients, and a monotonically increasing and convex function of information transmission rate. The simple expression of the asymptotic result enables optimal power allocation and optimal rate selection of HARQ-IR with low complexity. Finally, numerical results are provided to verify our analytical results and justify the application of the asymptotic result for optimal system design.
- May 10 2017 cs.CV arXiv:1705.03372v1We address the problem of localisation of objects as bounding boxes in images with weak labels. This weakly supervised object localisation problem has been tackled in the past using discriminative models where each object class is localised independently from other classes. We propose a novel framework based on Bayesian joint topic modelling. Our framework has three distinctive advantages over previous works: (1) All object classes and image backgrounds are modelled jointly together in a single generative model so that "explaining away" inference can resolve ambiguity and lead to better learning and localisation. (2) The Bayesian formulation of the model enables easy integration of prior knowledge about object appearance to compensate for limited supervision. (3) Our model can be learned with a mixture of weakly labelled and unlabelled data, allowing the large volume of unlabelled images on the Internet to be exploited for learning. Extensive experiments on the challenging VOC dataset demonstrate that our approach outperforms the state-of-the-art competitors.
- May 03 2017 cs.CV arXiv:1705.00873v1Most existing approaches to training object detectors rely on fully supervised learning, which requires the tedious manual annotation of object location in a training set. Recently there has been an increasing interest in developing weakly supervised approach to detector training where the object location is not manually annotated but automatically determined based on binary (weak) labels indicating if a training image contains the object. This is a challenging problem because each image can contain many candidate object locations which partially overlaps the object of interest. Existing approaches focus on how to best utilise the binary labels for object location annotation. In this paper we propose to solve this problem from a very different perspective by casting it as a transfer learning problem. Specifically, we formulate a novel transfer learning based on learning to rank, which effectively transfers a model for automatic annotation of object location from an auxiliary dataset to a target dataset with completely unrelated object categories. We show that our approach outperforms existing state-of-the-art weakly supervised approach to annotating objects in the challenging VOC dataset.
- Apr 26 2017 cs.CL arXiv:1704.07556v1Different linguistic perspectives causes many diverse segmentation criteria for Chinese word segmentation (CWS). Most existing methods focus on improve the performance for each single criterion. However, it is interesting to exploit these different criteria and mining their common underlying knowledge. In this paper, we propose adversarial multi-criteria learning for CWS by integrating shared knowledge from multiple heterogeneous segmentation criteria. Experiments on eight corpora with heterogeneous segmentation criteria show that the performance of each corpus obtains a significant improvement, compared to single-criterion learning. Source codes of this paper are available on Github.
- Apr 21 2017 cs.LG arXiv:1704.06061v4Multi-view feature has been proved to be very effective in many multimedia applications. However, the current back-end classifiers cannot make full use of such features. In this paper, we propose a method to model the multi-faceted information in the multi-view features explicitly and jointly. In our approach, the feature was modeled as a result derived by a generative multi-view (joint\footnotemark[1]) Probability Linear Discriminant Analysis (PLDA) model, which contains multiple kinds of latent variables. The usual PLDA model only considers one single label. However, in practical use, when using multi-task learned network as feature extractor, the extracted feature are always attached to several labels. This type of feature is called multi-view feature. With multi-view (joint) PLDA, we are able to explicitly build a model that can combine multiple heterogeneous information from the multi-view features. In verification step, we calculated the likelihood to describe whether the two features having consistent labels or not. This likelihood are used in the following decision-making. Experiments have been conducted on large scale verification task. On the public RSR2015 data corpus, the results showed that our approach can achieve 0.02\% EER and 0.09\% EER for impostor wrong and impostor correct cases respectively.
- Mar 29 2017 cs.CV arXiv:1703.09625v4Existing RNN-based approaches for action recognition from depth sequences require either skeleton joints or hand-crafted depth features as inputs. An end-to-end manner, mapping from raw depth maps to action classes, is non-trivial to design due to the fact that: 1) single channel map lacks texture thus weakens the discriminative power; 2) relatively small set of depth training data. To address these challenges, we propose to learn an RNN driven by privileged information (PI) in three-steps: An encoder is pre-trained to learn a joint embedding of depth appearance and PI (i.e. skeleton joints). The learned embedding layers are then tuned in the learning step, aiming to optimize the network by exploiting PI in a form of multi-task loss. However, exploiting PI as a secondary task provides little help to improve the performance of a primary task (i.e. classification) due to the gap between them. Finally, a bridging matrix is defined to connect two tasks by discovering latent PI in the refining step. Our PI-based classification loss maintains a consistency between latent PI and predicted distribution. The latent PI and network are iteratively estimated and updated in an expectation-maximization procedure. The proposed learning process provides greater discriminative power to model subtle depth difference, while helping avoid overfitting the scarcer training data. Our experiments show significant performance gains over state-of-the-art methods on three public benchmark datasets and our newly collected Blanket dataset.
- Oct 06 2016 cs.SY arXiv:1610.01455v1This paper studies the operation and scheduling of electric loads in micro-grid, a highly automated and distributed cyber-physical energy system (CPES). We establish rigorous mathematical expressions for electric loads and battery banks in the micro-grid by considering their characteristics and constraints. Based on these mathematical models, we propose a novel real-time scheduling analysis method for priority-based energy management in micro-grid, named Significant Moments Analysis (SMA). SMA pinpoints all the crucial moments when electrical operations are requested among the micro-grid and establishes a dynamic model to describe the scheduling behavior of electric loads. Using SMA, we can check the scheduling feasibility and predict whether the micro-grid can generate enough power to support the execution of electric loads. In the case where the power is insufficient to supply load demands, SMA can provide accurate information about the amount of insufficient power and the time when the insufficiency happens. Simulated results are presented to show the effectiveness of the proposed analysis method.
- Kerr nonlinear cavities displaying optical thresholding have been proposed for the realization of ultra-low power photonic logic gates. In the ultra-low photon number regime, corresponding to energy levels in the attojoule scale, quantum input-output models become important to study the effect of unavoidable quantum fluctuations on the performance of such logic gates. However, being a quantum anharmonic oscillator, a Kerr-cavity has an infinite dimensional Hilbert space spanned by the Fock states of the oscillator. This poses a challenge to simulate and analyze photonic logic gates and circuits composed of multiple Kerr nonlinearities. For simulation, the Hilbert of the oscillator is typically truncated to the span of only a finite number of Fock states. This paper develops a quasi-principal components approach to identify important subspaces of a Kerr-cavity Hilbert space and exploits it to construct an approximate reduced model of the Kerr-cavity on a smaller Hilbert space. Using this approach, we find a reduced dimension model with a Hilbert space dimension of 15 that can closely match the magnitudes of the mean transmitted and reflected output fields of a conventional truncated Fock state model of dimension 75, when driven by an input coherent field that switches between two levels. For the same input, the reduced model also closely matches the magnitudes of the mean output fields of Kerr-cavity-based AND and NOT gates and a NAND latch obtained from simulation of the full 75 dimension model.
- Jul 26 2016 cs.CV arXiv:1607.06972v3In this paper, we tackle the problem of 24 hours-monitoring patient actions in a ward such as "stretching an arm out of the bed", "falling out of the bed", where temporal movements are subtle or significant. In the concerned scenarios, the relations between scene layouts and body kinematics (skeletons) become important cues to recognize actions; however they are hard to be secured at a testing stage. To address this problem, we propose a kinematic-layout-aware random forest which takes into account the kinematic-layout (\ie layout and skeletons), to maximize the discriminative power of depth image appearance. We integrate the kinematic-layout in the split criteria of random forests to guide the learning process by 1) determining the switch to either the depth appearance or the kinematic-layout information, and 2) implicitly closing the gap between two distributions obtained by the kinematic-layout and the appearance, when the kinematic-layout appears useful. The kinematic-layout information is not required for the test data, thus called "privileged information prior". The proposed method has also been testified in cross-view settings, by the use of view-invariant features and enforcing the consistency among synthetic-view data. Experimental evaluations on our new dataset PATIENT, CAD-60 and UWA3D (multiview) demonstrate that our method outperforms various state-of-the-arts.
- We present the application of a low dimensional manifold model (LDMM) on hyperspectral image (HSI) reconstruction. An important property of hyperspectral images is that the patch manifold, which is sampled by the three-dimensional blocks in the data cube, is generally of a low dimensional nature. This is a generalization of low-rank models in that hyperspectral images with nonlinear mixing terms can also fit in this framework. The point integral method (PIM) is used to solve a Laplace-Beltrami equation over a point cloud sampling the patch manifold in LDMM. Both numerical simulations and theoretical analysis show that the sample points constraint is correctly enforced by PIM. The framework is demonstrated by experiments on the reconstruction of both linear and nonlinear mixed hyperspectral images with a significant number of missing voxels and several entirely missing spectral bands.
- Apr 19 2016 cs.LG arXiv:1604.05024v1PROXTONE is a novel and fast method for optimization of large scale non-smooth convex problem \citeshi2015large. In this work, we try to use PROXTONE method in solving large scale \emphnon-smooth non-convex problems, for example training of sparse deep neural network (sparse DNN) or sparse convolutional neural network (sparse CNN) for embedded or mobile device. PROXTONE converges much faster than first order methods, while first order method is easy in deriving and controlling the sparseness of the solutions. Thus in some applications, in order to train sparse models fast, we propose to combine the merits of both methods, that is we use PROXTONE in the first several epochs to reach the neighborhood of an optimal solution, and then use the first order method to explore the possibility of sparsity in the following training. We call such method PROXTONE plus (PROXTONE$^+$). Both PROXTONE and PROXTONE$^+$ are tested in our experiments, and which demonstrate both methods improved convergence speed twice as fast at least on diverse sparse model learning problems, and at the same time reduce the size to 0.5\% for DNN models. The source of all the algorithms is available upon request.
- This paper presents formulae for Einstein-Podolsky-Rosen (EPR) entanglement generated from $N$ nondegenerate optical parametric amplifiers (NOPAs) interconnected in a linear coherent feedback (CFB) chain in the idealized lossless scenario and infinite bandwidth limit. The lossless scenario sets the ultimate EPR entanglement (two-mode squeezing) that can be achieved by this linear chain of NOPAs while the infinite bandwidth limit simplifies the analysis but gives an accurate approximation to the EPR entanglement at low frequencies of interest. Two adjustable phase shifts are placed at the outputs of the system to achieve the best EPR entanglement by selecting appropriate quadratures of the output fields.
- Feb 17 2016 cs.LG arXiv:1602.05127v1We present a new perspective on graph-based methods for collaborative ranking for recommender systems. Unlike user-based or item-based methods that compute a weighted average of ratings given by the nearest neighbors, or low-rank approximation methods using convex optimization and the nuclear norm, we formulate matrix completion as a series of semi-supervised learning problems, and propagate the known ratings to the missing ones on the user-user or item-item graph globally. The semi-supervised learning problems are expressed as Laplace-Beltrami equations on a manifold, or namely, harmonic extension, and can be discretized by a point integral method. We show that our approach does not impose a low-rank Euclidean subspace on the data points, but instead minimizes the dimension of the underlying manifold. Our method, named LDM (low dimensional manifold), turns out to be particularly effective in generating rankings of items, showing decent computational efficiency and robust ranking quality compared to state-of-the-art methods.
- In this paper, we consider the harmonic extension problem, which is widely used in many applications of machine learning. We find that the transitional method of graph Laplacian fails to produce a good approximation of the classical harmonic function. To tackle this problem, we propose a new method called the point integral method (PIM). We consider the harmonic extension problem from the point of view of solving PDEs on manifolds. The basic idea of the PIM method is to approximate the harmonicity using an integral equation, which is easy to be discretized from points. Based on the integral equation, we explain the reason why the transitional graph Laplacian may fail to approximate the harmonicity in the classical sense and propose a different approach which we call the volume constraint method (VCM). Theoretically, both the PIM and the VCM computes a harmonic function with convergence guarantees, and practically, they are both simple, which amount to solve a linear system. One important application of the harmonic extension in machine learning is semi-supervised learning. We run a popular semi-supervised learning algorithm by Zhu et al. over a couple of well-known datasets and compare the performance of the aforementioned approaches. Our experiments show the PIM performs the best.
- The purpose of this paper is to prove a local optimality property of a recently proposed coherent feedback configuration for distributed generation of EPR entanglement using two nondegenerate optical parametric amplifiers (NOPAs) in the idealized infinite bandwidth limit. This local optimality is with respect to a class of similar coherent feedback configurations but employing different unitary scattering matrices, representing different scattering of propagating signals within the network. The infinite bandwidth limit is considered as it significantly simplifies the analysis, allowing local optimality criteria to be explicitly verified. Nonetheless, this limit is relevant for the finite bandwidth scenario as it provides an accurate approximation to the EPR entanglement in the low frequency region where EPR entanglement exists.
- In this paper, we consider multiple signals sharing same instantaneous frequencies. This kind of data is very common in scientific and engineering problems. To take advantage of this special structure, we modify our data-driven time-frequency analysis by updating the instantaneous frequencies simultaneously. Moreover, based on the simultaneously sparsity approximation and fast Fourier transform, some efficient algorithms is developed. Since the information of multiple signals is used, this method is very robust to the perturbation of noise. And it is applicable to the general nonperiodic signals even with missing samples or outliers. Several synthetic and real signals are used to test this method. The performances of this method are very promising.
- Eigenvectors and eigenvalues of discrete graph Laplacians are often used for manifold learning and nonlinear dimensionality reduction. It was previously proved by Belkin and Niyogi that the eigenvectors and eigenvalues of the graph Laplacian converge to the eigenfunctions and eigenvalues of the Laplace-Beltrami operator of the manifold in the limit of infinitely many data points sampled independently from the uniform distribution over the manifold. Recently, we introduced Point Integral method (PIM) to solve elliptic equations and corresponding eigenvalue problem on point clouds. We have established a unified framework to approximate the elliptic differential operators on point clouds. In this paper, we prove that the eigenvectors and eigenvalues obtained by PIM converge in the limit of infinitely many random samples independently from a distribution (not necessarily to be uniform distribution). Moreover, one estimate of the rate of the convergence is also given.
- This paper is concerned with linear quantum networks of $N$ nondegenerate optical parametric amplifiers (NOPAs), with $N$ up to 6, which are interconnected in a coherent feedback chain. Each network connects two communicating parties (Alice and Bob) over two transmission channels. In previous work we have shown that a dual-NOPA coherent feedback network generates better Einstein-Podolsky-Rosen (EPR) entanglement (i.e., more two-mode squeezing) between its two outgoing Gaussian fields than a single NOPA, when the same total pump power is consumed and the systems undergo the same transmission losses over the same distance. This paper aims to analyze stability, EPR entanglement between two outgoing fields of interest, and bipartite entanglement of two-mode Gaussian states of cavity modes of the $N$-NOPA networks under the effect of transmission and amplification losses, as well as time delays. It is numerically shown that, in the absence of losses and delays, the network with more NOPAs in the chain requires less total pump power to generate the same degree of EPR entanglement. Moreover, we report on the internal entanglement synchronization that occurs in the steady state between certain pairs of Gaussian oscillator modes inside the NOPA cavities of the networks.
- Apr 28 2015 cs.SY arXiv:1504.06663v1Fundamental theory on battery-powered cyber-physical systems (CPS) calls for dynamic models that are able to describe and predict the status of processors and batteries at any given time. We believe that the idealized system of single processor powered by single battery (SPSB) can be viewed as a generic case for the modeling effort. This paper introduces a dynamic model for multiple aperiodic tasks on a SPSB system under a scheduling algorithm that resembles the rate monotonic scheduling (RMS) within finite time windows. The model contains two major modules. The first module is an online battery capacity model based on the Rakhmatov-Vrudhula-Wallach (RVW) model. This module provides predictions of remaining battery capacity based on the knowledge of the battery discharging current. The second module is a dynamical scheduling model that can predict the scheduled behavior of tasks within any finite time window, without the need to store all past information about each task before the starting time of the finite time window. The module provides a complete analytical description of the relationship among tasks and it delineates all possible modes of the processor utilization as square-wave functions of time. The two modules i.e. the scheduling model and the battery model are integrated to obtain a hybrid scheduling model that describes the dynamic behaviors of the SPSB system. Our effort may have demonstrated that through dynamic modeling, different components of CPS may be integrated under a unified theoretical framework centered around hybrid systems theory.
- Apr 02 2015 cs.CV arXiv:1504.00045v1When humans describe images they tend to use combinations of nouns and adjectives, corresponding to objects and their associated attributes respectively. To generate such a description automatically, one needs to model objects, attributes and their associations. Conventional methods require strong annotation of object and attribute locations, making them less scalable. In this paper, we model object-attribute associations from weakly labelled images, such as those widely available on media sharing sites (e.g. Flickr), where only image-level labels (either object or attributes) are given, without their locations and associations. This is achieved by introducing a novel weakly supervised non-parametric Bayesian model. Once learned, given a new image, our model can describe the image, including objects, attributes and their associations, as well as their locations and segmentation. Extensive experiments on benchmark datasets demonstrate that our weakly supervised model performs at par with strongly supervised models on tasks such as image description and retrieval based on object-attribute associations.
- Mar 11 2015 cs.OH arXiv:1503.02747v1This paper addresses the problem of rate selection for the cooperative hybrid automatic repeat request with chase combination (HARQ-CC) system, where time correlated Nakagami-m fading channels are considered. To deal with this problem, the closed-form cumulative distribution function (CDF) for the combine SNRs through maximal ratio combining (MRC) is first derived as a generalized Fox's $\bar H$ function. By using this result, outage probability and delay-limited throughput (DLT) are derived in closed forms, which then enables the rate selection for maximum DLT. These analytical results are validated via Monte Carlo simulations. The impacts of time correlation and channel fading-order parameter $m$ upon outage probability, DLT and the optimal rate are investigated thoroughly. It is found that the system can achieve more diversity gain from less correlated channels, and the outage probability of cooperative HARQ-CC system decreases with the increase of $m$, and etc. Furthermore, the optimal rate increases with the number of retransmissions, while it decreases with the increase of the channel time correlation.
- Mar 10 2015 cs.SY arXiv:1503.02300v2When multiple model predictive controllers are implemented on a shared control area network (CAN), their performance may degrade due to the inhomogeneous timing and delays among messages. The priority based real-time scheduling of messages on the CAN introduces complex timing of events, especially when the types and number of messages change at runtime. This paper introduces a novel hybrid timing model to make runtime predictions on the timing of the messages for a finite time window. Controllers can be designed using the optimization algorithms for model predictive control by considering the timing as optimization constraints. This timing model allows multiple controllers to share a CAN without significant degradation in the controller performance. The timing model also provides a convenient way to check the schedulability of messages on the CAN at runtime. Simulation results demonstrate that the timing model is accurate and computationally efficient to meet the needs of real-time implementation. Simulation results also demonstrate that model predictive controllers designed when considering the timing constraints have superior performance than the controllers designed without considering the timing constraints.
- Recent theoretical investigations on quantum coherent feedback networks have found that with the same pump power, the Einstein-Podolski-Rosen (EPR)-like entanglement generated via a dual nondegenerate optical parametric amplifier (NOPA) system placed in a certain coherent feedback loop is stronger than the EPR-like entangled pairs produced by a single NOPA. In this paper, we present a linear quantum system consisting of two NOPAs and a static linear passive network of optical devices. The network has six inputs and six outputs, among which four outputs and four inputs are connected in a coherent feedback loop with the two NOPAs. This passive network is represented by a $6 \times 6$ complex unitary matrix. A modified steepest descent method is used to find a passive complex unitary matrix at which the entanglement of this dual-NOPA network is locally maximized. Here we choose the matrix corresponding to a dual-NOPA coherent feedback network from our previous work as a starting point for the modified steepest descent algorithm. By decomposing the unitary matrix obtained by the algorithm as the product of so-called two-level unitary matrices, we find an optimized configuration in which the complex matrix is realized by a static optical network made of beam splitters.
- In this paper, we consider signals with intra-wave frequency modulation. To handle this kind of signals effectively, we generalize our data-driven time-frequency analysis by using a shape function to describe the intra-wave frequency modulation. The idea of using a shape function in time-frequency analysis was first proposed by Wu. A shape function could be any periodic function. Based on this model, we propose to solve an optimization problem to extract the shape function. By exploring the fact that s is a periodic function, we can identify certain low rank structure of the signal. This structure enables us to extract the shape function from the signal. To test the robustness of our method, we apply our method on several synthetic and real signals. The results are very encouraging.
- In this paper, we analyze the uniqueness of the sparse time frequency decomposition and investigate the efficiency of the nonlinear matching pursuit method. Under the assumption of scale separation, we show that the sparse time frequency decomposition is unique up to an error that is determined by the scale separation property of the signal. We further show that the unique decomposition can be obtained approximately by the sparse time frequency decomposition using nonlinear matching pursuit.
- Recent work has shown that deploying two nondegenerate optical parametric amplifiers (NOPAs) separately at two distant parties in a coherent feedback loop generates stronger Einstein-Podolski-Rosen (EPR) entanglement between two propagating continuous-mode output fields than a single NOPA under same pump power, decay rate and transmission losses. The purpose of this paper is to investigate the stability and EPR entanglement of a dual-NOPA coherent feedback system under the effect of phase shifts in the transmission channel between two distant parties. It is shown that, in the presence of phase shifts, EPR entanglement worsens or can vanish, but can be improved to some extent in certain scenarios by adding a phase shifter at each output with a certain value of phase shift. In ideal cases, in the absence of transmission and amplification losses, existence of EPR entanglement and whether the original EPR entanglement can be recovered by the additional phase shifters are decided by values of the phase shifts in the path.
- We provide two algorithms counting the number of minimum Roman dominating functions of a graph on n vertices in O(1.5673^n) time and polynomial space. We also show that the time complexity can be reduced to O(1.5014^n) if exponential space is used. Our result is obtained by transforming the Roman domination problem into other combinatorial problems on graphs for which exact algorithms already exist.
- In this paper, we establish a connection between the recently developed data-driven time-frequency analysis \citeHS11,HS13-1 and the classical second order differential equations. The main idea of the data-driven time-frequency analysis is to decompose a multiscale signal into a sparsest collection of Intrinsic Mode Functions (IMFs) over the largest possible dictionary via nonlinear optimization. These IMFs are of the form $a(t) \cos(\theta(t))$ where the amplitude $a(t)$ is positive and slowly varying. The non-decreasing phase function $\theta(t)$ is determined by the data and in general depends on the signal in a nonlinear fashion. One of the main results of this paper is that we show that each IMF can be associated with a solution of a second order ordinary differential equation of the form $\ddot{x}+p(x,t)\dot{x}+q(x,t)=0$. Further, we propose a localized variational formulation for this problem and develop an effective $l^1$-based optimization method to recover $p(x,t)$ and $q(x,t)$ by looking for a sparse representation of $p$ and $q$ in terms of the polynomial basis. Depending on the form of nonlinearity in $p(x,t)$ and $q(x,t)$, we can define the degree of nonlinearity for the associated IMF. %and the corresponding coefficients for the associated highest order nonlinear terms. This generalizes a concept recently introduced by Prof. N. E. Huang et al. \citeHuang11. Numerical examples will be provided to illustrate the robustness and stability of the proposed method for data with or without noise. This manuscript should be considered as a proof of concept.
- Most existing work on predicting NCAAB matches has been developed in a statistical context. Trusting the capabilities of ML techniques, particularly classification learners, to uncover the importance of features and learn their relationships, we evaluated a number of different paradigms on this task. In this paper, we summarize our work, pointing out that attributes seem to be more important than models, and that there seems to be an upper limit to predictive quality.
- In this paper, we propose a time-frequency analysis method to obtain instantaneous frequencies and the corresponding decomposition by solving an optimization problem. In this optimization problem, the basis to decompose the signal is not known. Instead, it is adapted to the signal and is determined as part of the optimization problem. In this sense, this optimization problem can be seen as a dictionary learning problem. This dictionary learning problem is solved by using the Augmented Lagrangian Multiplier method (ALM) iteratively. We further accelerate the convergence of the ALM method in each iteration by using the fast wavelet transform. We apply our method to decompose several signals, including signals with poor scale separation, signals with outliers and polluted by noise and a real signal. The results show that this method can give accurate recovery of both the instantaneous frequencies and the intrinsic mode functions.
- Online and stochastic learning has emerged as powerful tool in large scale optimization. In this work, we generalize the Douglas-Rachford splitting (DRs) method for minimizing composite functions to online and stochastic settings (to our best knowledge this is the first time DRs been generalized to sequential version). We first establish an $O(1/\sqrt{T})$ regret bound for batch DRs method. Then we proved that the online DRs splitting method enjoy an $O(1)$ regret bound and stochastic DRs splitting has a convergence rate of $O(1/\sqrt{T})$. The proof is simple and intuitive, and the results and technique can be served as a initiate for the research on the large scale machine learning employ the DRs method. Numerical experiments of the proposed method demonstrate the effectiveness of the online and stochastic update rule, and further confirm our regret and convergence analysis.
- In a recent paper, Hou and Shi introduced a new adaptive data analysis method to analyze nonlinear and non-stationary data. The main idea is to look for the sparsest representation of multiscale data within the largest possible dictionary consisting of intrinsic mode functions of the form $\{a(t) \cos(\theta(t))\}$, where $a \in V(\theta)$, $V(\theta)$ consists of the functions smoother than $\cos(\theta(t))$ and $\theta'\ge 0$. This problem was formulated as a nonlinear $L^0$ optimization problem and an iterative nonlinear matching pursuit method was proposed to solve this nonlinear optimization problem. In this paper, we prove the convergence of this nonlinear matching pursuit method under some sparsity assumption on the signal. We consider both well-resolved and sparse sampled signals. In the case without noise, we prove that our method gives exact recovery of the original signal.
- Mixture models have been widely used in modeling of continuous observations. For the possibility to estimate the parameters of a mixture model consistently on the basis of observations from the mixture, identifiability is a necessary condition. In this study, we give some results on the identifiability of multivariate logistic mixture models.
- This paper studies the recovery guarantees of the models of minimizing $\|\mathcal{X}\|_*+\frac{1}{2\alpha}\|\mathcal{X}\|_F^2$ where $\mathcal{X}$ is a tensor and $\|\mathcal{X}\|_*$ and $\|\mathcal{X}\|_F$ are the trace and Frobenius norm of respectively. We show that they can efficiently recover low-rank tensors. In particular, they enjoy exact guarantees similar to those known for minimizing $\|\mathcal{X}\|_*$ under the conditions on the sensing operator such as its null-space property, restricted isometry property, or spherical section property. To recover a low-rank tensor $\mathcal{X}^0$, minimizing $\|\mathcal{X}\|_*+\frac{1}{2\alpha}\|\mathcal{X}\|_F^2$ returns the same solution as minimizing $\|\mathcal{X}\|_*$ almost whenever $\alpha\geq10\mathop {\max}\limits_{i}\|X^0_{(i)}\|_2$.
- In this paper, a novel framework based on trace norm minimization for audio segment is proposed. In this framework, both the feature extraction and classification are obtained by solving corresponding convex optimization problem with trace norm regularization. For feature extraction, robust principle component analysis (robust PCA) via minimization a combination of the nuclear norm and the $\ell_1$-norm is used to extract low-rank features which are robust to white noise and gross corruption for audio segments. These low-rank features are fed to a linear classifier where the weight and bias are learned by solving similar trace norm constrained problems. For this classifier, most methods find the weight and bias in batch-mode learning, which makes them inefficient for large-scale problems. In this paper, we propose an online framework using accelerated proximal gradient method. This framework has a main advantage in memory cost. In addition, as a result of the regularization formulation of matrix classification, the Lipschitz constant was given explicitly, and hence the step size estimation of general proximal gradient method was omitted in our approach. Experiments on real data sets for laugh/non-laugh and applause/non-applause classification indicate that this novel framework is effective and noise robust.
- This paper establishes a novel analytical approach to quantify robustness of scheduling and battery management for battery supported cyber-physical systems. A dynamic schedulability test is introduced to determine whether tasks are schedulable within a finite time window. The test is used to measure robustness of a real-time scheduling algorithm by evaluating the strength of computing time perturbations that break schedulability at runtime. Robustness of battery management is quantified analytically by an adaptive threshold on the state of charge. The adaptive threshold significantly reduces the false alarm rate for battery management algorithms to decide when a battery needs to be replaced.
- In this paper we propose an algorithm to classify tensor data. Our methodology is built on recent studies about matrix classification with the trace norm constrained weight matrix and the tensor trace norm. Similar to matrix classification, the tensor classification is formulated as a convex optimization problem which can be solved by using the off-the-shelf accelerated proximal gradient (APG) method. However, there are no analytic solutions as the matrix case for the updating of the weight tensors via the proximal gradient. To tackle this problem, the Douglas-Rachford splitting technique and the alternating direction method of multipliers (ADM) used in tensor completion are adapted to update the weight tensors. Further more, due to the demand of real applications, we also propose its online learning approaches. Experiments demonstrate the efficiency of the methods.
- The Immersed Boundary method has evolved into one of the most useful computational methods in studying fluid structure interaction. On the other hand, the Immersed Boundary method is also known to suffer from a severe timestep stability restriction when using an explicit time discretization. In this paper, we propose several efficient semi-implicit schemes to remove this stiffness from the Immersed Boundary method for the two-dimensional Stokes flow. First, we obtain a novel unconditionally stable semi-implicit discretization for the immersed boundary problem. Using this unconditionally stable discretization as a building block, we derive several efficient semi-implicit schemes for the immersed boundary problem by applying the Small Scale Decomposition to this unconditionally stable discretization. Our stability analysis and extensive numerical experiments show that our semi-implicit schemes offer much better stability property than the explicit scheme. Unlike other implicit or semi-implicit schemes proposed in the literature, our semi-implicit schemes can be solved explicitly in the spectral space. Thus the computational cost of our semi-implicit schemes is comparable to that of an explicit scheme, but with a much better stability property.
- The subset sum problem (SSP) can be briefly stated as: given a target integer $E$ and a set $A$ containing $n$ positive integer $a_j$, find a subset of $A$ summing to $E$. The \textitdensity $d$ of an SSP instance is defined by the ratio of $n$ to $m$, where $m$ is the logarithm of the largest integer within $A$. Based on the structural and statistical properties of subset sums, we present an improved enumeration scheme for SSP, and implement it as a complete and exact algorithm (EnumPlus). The algorithm always equivalently reduces an instance to be low-density, and then solve it by enumeration. Through this approach, we show the possibility to design a sole algorithm that can efficiently solve arbitrary density instance in a uniform way. Furthermore, our algorithm has considerable performance advantage over previous algorithms. Firstly, it extends the density scope, in which SSP can be solved in expected polynomial time. Specifically, It solves SSP in expected $O(n\log{n})$ time when density $d \geq c\cdot \sqrt{n}/\log{n}$, while the previously best density scope is $d \geq c\cdot n/(\log{n})^{2}$. In addition, the overall expected time and space requirement in the average case are proven to be $O(n^5\log n)$ and $O(n^5)$ respectively. Secondly, in the worst case, it slightly improves the previously best time complexity of exact algorithms for SSP. Specifically, the worst-case time complexity of our algorithm is proved to be $O((n-6)2^{n/2}+n)$, while the previously best result is $O(n2^{n/2})$.
- This paper is aimed at designing efficient parallel matrix-product algorithms for heterogeneous master-worker platforms. While matrix-product is well-understood for homogeneous 2D-arrays of processors (e.g., Cannon algorithm and ScaLAPACK outer product algorithm), there are three key hypotheses that render our work original and innovative: - Centralized data. We assume that all matrix files originate from, and must be returned to, the master. - Heterogeneous star-shaped platforms. We target fully heterogeneous platforms, where computational resources have different computing powers. - Limited memory. Because we investigate the parallelization of large problems, we cannot assume that full matrix panels can be stored in the worker memories and re-used for subsequent updates (as in ScaLAPACK). We have devised efficient algorithms for resource selection (deciding which workers to enroll) and communication ordering (both for input and result messages), and we report a set of numerical experiments on various platforms at Ecole Normale Superieure de Lyon and the University of Tennessee. However, we point out that in this first version of the report, experiments are limited to homogeneous platforms.