State-of-the-art results on neural machine translation often use attentional sequence-to-sequence models with some form of convolution or recursion. Vaswani et al. (2017) propose a new architecture that avoids recurrence and convolution completely. Instead, it uses only self-attention and feed-forward layers. While the proposed architecture achieves state-of-the-art results on several machine translation tasks, it requires a large number of parameters and training iterations to converge. We propose Weighted Transformer, a Transformer with modified attention layers, that not only outperforms the baseline network in BLEU score but also converges 15-40% faster. Specifically, we replace the multi-head attention by multiple self-attention branches that the model learns to combine during the training process. Our model improves the state-of-the-art performance by 0.5 BLEU points on the WMT 2014 English-to-German translation task and by 0.4 on the English-to-French translation task.
While much of the work in the design of convolutional networks over the last five years has revolved around the empirical investigation of the importance of depth, filter sizes, and number of feature channels, recent studies have shown that branching, i.e., splitting the computation along parallel but distinct threads and then aggregating their outputs, represents a new promising dimension for significant improvements in performance. To combat the complexity of design choices in multi-branch architectures, prior work has adopted simple strategies, such as a fixed branching factor, the same input being fed to all parallel branches, and an additive combination of the outputs produced by all branches at aggregation points. In this work we remove these predefined choices and propose an algorithm to learn the connections between branches in the network. Instead of being chosen a priori by the human designer, the multi-branch connectivity is learned simultaneously with the weights of the network by optimizing a single loss function defined with respect to the end task. We demonstrate our approach on the problem of multi-class image classification using three different datasets where it yields consistently higher accuracy compared to the state-of-the-art "ResNeXt" multi-branch network given the same learning capacity.
Apr 21 2017 cs.CV
We introduce an architecture for large-scale image categorization that enables the end-to-end learning of separate visual features for the different classes to distinguish. The proposed model consists of a deep CNN shaped like a tree. The stem of the tree includes a sequence of convolutional layers common to all classes. The stem then splits into multiple branches implementing parallel feature extractors, which are ultimately connected to the final classification layer via learned gated connections. These learned gates determine for each individual class the subset of features to use. Such a scheme naturally encourages the learning of a heterogeneous set of specialized features through the separate branches and it allows each class to use the subset of features that are optimal for its recognition. We show the generality of our proposed method by reshaping several popular CNNs from the literature into our proposed architecture. Our experiments on the CIFAR100, CIFAR10, and Synth datasets show that in each case our resulting model yields a substantial improvement in accuracy over the original CNN. Our empirical analysis also suggests that our scheme acts as a form of beneficial regularization improving generalization performance.
Oct 11 2016 cs.CV
Content-Based Image Retrieval (CBIR) systems have been widely used for a wide range of applications such as Art collections, Crime prevention and Intellectual property. In this paper, a novel CBIR system, which utilizes visual contents (color, texture and shape) of an image to retrieve images, is proposed. The proposed system builds three feature vectors and stores them into MySQL database. The first feature vector uses descriptive statistics to describe the distribution of data in each channel of RGB channels of the image. The second feature vector describes the texture using eigenvalues of the 39 sub-bands that are generated after applying four levels 2D DWT in each channel (red, green and blue channels) of the image. These wavelets sub-bands perfectly describes the horizontal, vertical and diagonal edges that exist in the multi-resolution analysis of the image. The third feature vector describes the basic shapes that exist in the skeletonization version of the black and white representation of the image. Experimental results on a private MYSQL database that consists of 10000 images, using color, texture, shape and stored relevance feedbacks, showed 96.4% average correct retrieval rate in an efficient recovery time.
Apr 22 2016 cs.CV
We present a tree-structured network architecture for large scale image classification. The trunk of the network contains convolutional layers optimized over all classes. At a given depth, the trunk splits into separate branches, each dedicated to discriminate a different subset of classes. Each branch acts as an expert classifying a set of categories that are difficult to tell apart, while the trunk provides common knowledge to all experts in the form of shared features. The training of our "network of experts" is completely end-to-end: the partition of categories into disjoint subsets is learned simultaneously with the parameters of the network trunk and the experts are trained jointly by minimizing a single learning objective over all classes. The proposed structure can be built from any existing convolutional neural network (CNN). We demonstrate its generality by adapting 4 popular CNNs for image categorization into the form of networks of experts. Our experiments on CIFAR100 and ImageNet show that in every case our method yields a substantial improvement in accuracy over the base CNN, and gives the best result achieved so far on CIFAR100. Finally, the improvement in accuracy comes at little additional cost: compared to the base network, the training time is only moderately increased and the number of parameters is comparable or in some cases even lower.
V-BLAST detection method suffers large computational complexity due to its successive detection of symbols. In this paper, we propose a modified V-BLAST algorithm to decrease the computational complexity by reducing the number of detection iterations required in MIMO communication systems. We begin by showing the existence of a maximum number of iterations, beyond which, no significant improvement is obtained. We establish a criterion for the number of maximum effective iterations. We propose a modified algorithm that uses the measured SNR to dynamically set the number of iterations to achieve an acceptable bit-error rate. Then, we replace the feedback algorithm with an approximate linear function to reduce the complexity. Simulations show that significant reduction in computational complexity is achieved compared to the ordinary V-BLAST, while maintaining a good BER performance.
Nov 04 2014 cs.CY
Data will soon become one of the most precious treasures we have ever had, 43 trillion gigabytes of data will be created by 2020 according to a study made by Mckinsey Global Institute, it is estimated that 2.3 trillion gigabytes of data is created each day and most companies in the US have 100.000 gigabytes of data stored. Data is recorded, stored and analyzed to enable technology and services that the world relies on every day, this technology is getting smarter and we will be soon living in a world of smart services or what is called smart cities. This article presents an overview of the topic pointing to its actual status and forecasting the crucial roles it will play in the future, we will define big data analytics and smart cities and talk about their potential contributions in changing our way of living and finally we will discuss the possible down side of this upcoming technologies and how it can fool us, violate our privacy and turn us into puppets or technology slaves.
Jan 22 2014 cs.SE
Behavior Engineering (BE) provides a rigorous way to derive a formal specification of a software system from the requirements written in natural language. Its graphical specification language, Behavior Tree (BT), has been used with success in industry to systematically translate large, complex, and often erroneous requirements into an integrated model of the software system. BE's process, the Behavior Modeling Process (BMP), allows requirements to be translated into individual requirement BTs one at a time, which are then integrated to form a holistic view of the system. The integrated BT then goes through a series of modifications to construct a specification BT, which is used for validation and verification. The BMP also addresses different types of defects in the requirements throughout its process. However, BT itself is a graphical modeling notation, and the types of integration relations, how they correspond to particular issues, how they should be integrated and how to get formal specification have not been clearly defined. As a result, the BMP is informal, and provides guidelines to perform all these tasks on an ad-hoc basis. In this paper, we first introduce a mathematical framework which defines the graphical form of BTs which we use to define the integration relationships of BTs and to formalize the integration strategy of the BMP. We then formulate semi-automated requirements defects detection techniques by utilizing this underlying mathematical framework, which may be extended to formalize the BMP, develop change management framework for it, build techniques for round-trip engineering and so on.
Oct 26 2010 cs.NI
The natural or man-made disaster demands an efficient communication and coordination among first responders to save life and other community resources. Normally, the traditional communication infrastructures such as land line or cellular networks are damaged and don't provide adequate communication services to first responders for exchanging emergency related information. Wireless ad hoc networks such as mobile ad hoc networks, wireless sensor networks and wireless mesh networks are the promising alternatives in such type of situations. The security requirements for emergency response communications include privacy, data integrity, authentication, key management, access control and availability. Various ad hoc communication frameworks have been proposed for emergency response situations. The majority of the proposed frameworks don't provide adequate security services for reliable and secure information exchange. This paper presents a survey of the proposed emergency response communication frameworks and the potential security services required by them to provide reliable and secure information exchange during emergency situations.