今日学术视野(2017.06.29)

astro-ph.IM - 仪器仪表和天体物理学方法
cond-mat.stat-mech - 统计数学
cs.AI - 人工智能
cs.CL - 计算与语言
cs.CV - 机器视觉与模式识别
cs.CY - 计算与社会
cs.IR - 信息检索
cs.IT - 信息论
cs.LG - 自动学习
cs.MS - 数学软件
cs.NE - 神经与进化计算
cs.NI - 网络和互联网体系结构
cs.RO - 机器人学
cs.SD - 声音处理
cs.SE - 软件工程
cs.SI - 社交网络与信息网络
math.NA - 数值分析
math.ST - 统计理论
q-bio.PE - 人口与发展
quant-ph - 量子物理
stat.AP - 应用统计
stat.CO - 统计计算
stat.ME - 统计方法论
stat.ML - (统计)机器学习

• [astro-ph.IM]The Fog of War: A Machine Learning Approach to Forecasting Weather on Mars
• [cond-mat.stat-mech]An Isomorphism between Lyapunov Exponents and Shannon's Channel Capacity
• [cs.AI]Relating Complexity-theoretic Parameters with SAT Solver Performance
• [cs.AI]SUNNY-CP and the MiniZinc Challenge
• [cs.CL]Memory-augmented Chinese-Uyghur Neural Machine Translation
• [cs.CL]Neural Question Answering at BioASQ 5B
• [cs.CL]The Minor Fall, the Major Lift: Inferring Emotional Valence of Musical Chords through Lyrics
• [cs.CV]A Unified approach for Conventional Zero-shot, Generalized Zero-shot and Few-shot Learning
• [cs.CV]Approximate Reflection Symmetry in a Point Set: Theory and Algorithm with an Application
• [cs.CV]Auto-Encoder Guided GAN for Chinese Calligraphy Synthesis
• [cs.CV]Cross-Country Skiing Gears Classification using Deep Learning
• [cs.CV]Dense Non-rigid Structure-from-Motion Made Easy - A Spatial-Temporal Smoothness based Solution
• [cs.CV]Detecting Small Signs from Large Images
• [cs.CV]Do Deep Neural Networks Suffer from Crowding?
• [cs.CV]Fast and accurate classification of echocardiograms using deep learning
• [cs.CV]Hierarchical Model for Long-term Video Prediction
• [cs.CV]Illuminating Pedestrians via Simultaneous Detection & Segmentation
• [cs.CV]Independent motion detection with event-driven cameras
• [cs.CV]Large-scale Datasets: Faces with Partial Occlusions and Pose Variations in the Wild
• [cs.CV]Recurrent Residual Learning for Action Recognition
• [cs.CV]Robust Sonar ATR Through Bayesian Pose Corrected Sparse Classification
• [cs.CV]Robust Video-Based Eye Tracking Using Recursive Estimation of Pupil Characteristics
• [cs.CV]Topometric Localization with Deep Learning
• [cs.CV]Training a Fully Convolutional Neural Network to Route Integrated Circuits
• [cs.CY]Democratizing Design for Future Computing Platforms
• [cs.IR]Classical Music Clustering Based on Acoustic Features
• [cs.IR]DE-PACRR: Exploring Layers Inside the PACRR Model
• [cs.IT]Beamforming and Scheduling for mmWave Downlink Sparse Virtual Channels With Non-Orthogonal and Orthogonal Multiple Access
• [cs.IT]Beyond Moore-Penrose Part II: The Sparse Pseudoinverse
• [cs.IT]Centralized and Distributed Sparsification for Low-Complexity Message Passing Algorithm in C-RAN Architectures
• [cs.IT]Constant composition codes derived from linear codes
• [cs.IT]DFE/THP duality for FBMC with highly frequency selective channels
• [cs.IT]Fountain Codes under Maximum Likelihood Decoding
• [cs.IT]Invariant components of synergy, redundancy, and unique information among three variables
• [cs.IT]MMSE precoder for massive MIMO using 1-bit quantization
• [cs.IT]Minimum BER Precoding in 1-Bit Massive MIMO Systems
• [cs.IT]NOMA based Random Access with Multichannel ALOHA
• [cs.IT]NOMA: Principles and Recent Results
• [cs.IT]PSK Precoding in Multi-User MISO Systems
• [cs.IT]Power- and Spectral Efficient Communication System Design Using 1-Bit Quantization
• [cs.IT]Spatial Coding Based on Minimum BER in 1-Bit Massive MIMO Systems
• [cs.IT]Spectral shaping with low resolution signals
• [cs.LG]Exploring Generalization in Deep Learning
• [cs.LG]Fast and robust tensor decomposition with applications to dictionary learning
• [cs.LG]Forecasting and Granger Modelling with Non-linear Dynamical Dependencies
• [cs.LG]GANs Trained by a Two Time-Scale Update Rule Converge to a Nash Equilibrium
• [cs.LG]Gradient Episodic Memory for Continuum Learning
• [cs.LG]Learning Local Feature Aggregation Functions with Backpropagation
• [cs.LG]Preserving Differential Privacy in Convolutional Deep Belief Networks
• [cs.LG]Reexamining Low Rank Matrix Factorization for Trace Norm Regularization
• [cs.MS]Parareal Algorithm Implementation and Simulation in Julia
• [cs.NE]PasMoQAP: A Parallel Asynchronous Memetic Algorithm for solving the Multi-Objective Quadratic Assignment Problem
• [cs.NE]Proceedings of the First International Workshop on Deep Learning and Music
• [cs.NI]Rate-Distortion Classification for Self-Tuning IoT Networks
• [cs.NI]Self-Sustaining Caching Stations: Towards Cost-Effective 5G-Enabled Vehicular Networks
• [cs.RO]Controlled Tactile Exploration and Haptic Object Recognition
• [cs.RO]Material Recognition CNNs and Hierarchical Planning for Biped Robot Locomotion on Slippery Terrain
• [cs.SD]Gabor frames and deep scattering networks in audio processing
• [cs.SE]Developing Bug-Free Machine Learning Systems With Formal Mathematics
• [cs.SI]A preference elicitation interface for collecting dense recommender datasets with rich user information
• [cs.SI]Second-Order Moment-Closure for Tighter Epidemic Thresholds
• [cs.SI]Validation of a smartphone app to map social networks of proximity
• [cs.SI]White, Man, and Highly Followed: Gender and Race Inequalities in Twitter
• [math.NA]Using Frame Theoretic Convolutional Gridding for Robust Synthetic Aperture Sonar Imaging
• [math.ST]Coverage Probability Fails to Ensure Reliable Inference
• [math.ST]Empirical priors and posterior concentration rates for a monotone density
• [math.ST]Group Synchronization on Grids
• [math.ST]Laplace deconvolution in the presence of indirect long-memory data
• [math.ST]New insights into non-central beta distributions
• [math.ST]Robust Sparse Covariance Estimation by Thresholding Tyler's M-Estimator
• [q-bio.PE]Well-supported phylogenies using largest subsets of core-genes by discrete particle swarm optimization
• [quant-ph]Preservation of quantum Fisher information and geometric phase of a single qubit system in a dissipative reservoir through the addition of qubits
• [stat.AP]Robust and Efficient Parametric Spectral Estimation in Atomic Force Microscopy
• [stat.CO]archivist: An R Package for Managing, Recording and Restoring Data Analysis Results
• [stat.ME]Evaluating the hot hand phenomenon using predictive memory selection for multistep Markov Chains: LeBron James' error correcting free throws
• [stat.ME]Extrinsic Gaussian processes for regression and classification on manifolds
• [stat.ME]Invariant Causal Prediction for Nonlinear Models
• [stat.ME]Subspace Clustering with the Multivariate-t Distribution
• [stat.ML]Cognitive Psychology for Deep Neural Networks: A Shape Bias Case Study
• [stat.ML]Fast Algorithms for Learning Latent Variables in Graphical Models
• [stat.ML]Faster ICA by preconditioning with Hessian approximations
• [stat.ML]MolecuLeNet: A continuous-filter convolutional neural network for modeling quantum interactions
• [stat.ML]On conditional parity as a notion of non-discrimination in machine learning
• [stat.ML]Two-Stage Hybrid Day-Ahead Solar Forecasting
• [stat.ML]Unsupervised Feature Selection Based on Space Filling Concept
• [stat.ML]When Neurons Fail

·····································

• [astro-ph.IM]The Fog of War: A Machine Learning Approach to Forecasting Weather on Mars
Daniele Bellutta
http://arxiv.org/abs/1706.08915v1

For over a decade, scientists at NASA's Jet Propulsion Laboratory (JPL) have been recording measurements from the Martian surface as a part of the Mars Exploration Rovers mission. One quantity of interest has been the opacity of Mars's atmosphere for its importance in day-to-day estimations of the amount of power available to the rover from its solar arrays. This paper proposes the use of neural networks as a method for forecasting Martian atmospheric opacity that is more effective than the current empirical model. The more accurate prediction provided by these networks would allow operators at JPL to make more accurate predictions of the amount of energy available to the rover when they plan activities for coming sols.

• [cond-mat.stat-mech]An Isomorphism between Lyapunov Exponents and Shannon's Channel Capacity
Gerald Friedland, Alfredo Metere
http://arxiv.org/abs/1706.08638v1

We demonstrate that discrete Lyapunov exponents are isomorphic to numeric overflows of the capacity of an arbitrary noiseless and memoryless channel in a Shannon communication model with feedback. The isomorphism allows the understanding of Lyapunov exponents in terms of Information Theory, rather than the traditional definitions in chaos theory. The result also implies alternative approaches to the calculation of related quantities, such as the Kolmogorov Sinai entropy which has been linked to thermodynamic entropy. This work provides a bridge between fundamental physics and information theory. It suggests, among other things, that machine learning and other information theory methods can be employed at the core of physics simulations.

• [cs.AI]Relating Complexity-theoretic Parameters with SAT Solver Performance
Edward Zulkoski, Ruben Martins, Christoph Wintersteiger, Robert Robere, Jia Liang, Krzysztof Czarnecki, Vijay Ganesh
http://arxiv.org/abs/1706.08611v1

Over the years complexity theorists have proposed many structural parameters to explain the surprising efficiency of conflict-driven clause-learning (CDCL) SAT solvers on a wide variety of large industrial Boolean instances. While some of these parameters have been studied empirically, until now there has not been a unified comparative study of their explanatory power on a comprehensive benchmark. We correct this state of affairs by conducting a large-scale empirical evaluation of CDCL SAT solver performance on nearly 7000 industrial and crafted formulas against several structural parameters such as backdoors, treewidth, backbones, and community structure. Our study led us to several results. First, we show that while such parameters only weakly correlate with CDCL solving time, certain combinations of them yield much better regression models. Second, we show how some parameters can be used as a "lens" to better understand the efficiency of different solving heuristics. Finally, we propose a new complexity-theoretic parameter, which we call learning-sensitive with restarts (LSR) backdoors, that extends the notion of learning-sensitive (LS) backdoors to incorporate restarts and discuss algorithms to compute them. We mathematically prove that for certain class of instances minimal LSR-backdoors are exponentially smaller than minimal-LS backdoors.

• [cs.AI]SUNNY-CP and the MiniZinc Challenge
Roberto Amadini, Maurizio Gabbrielli, Jacopo Mauro
http://arxiv.org/abs/1706.08627v1

In Constraint Programming (CP) a portfolio solver combines a variety of different constraint solvers for solving a given problem. This fairly recent approach enables to significantly boost the performance of single solvers, especially when multicore architectures are exploited. In this work we give a brief overview of the portfolio solver sunny-cp, and we discuss its performance in the MiniZinc Challenge---the annual international competition for CP solvers---where it won two gold medals in 2015 and 2016.

• [cs.CL]Memory-augmented Chinese-Uyghur Neural Machine Translation
Shiyue Zhang, Gulnigar Mahmut, Dong Wang, Askar Hamdulla
http://arxiv.org/abs/1706.08683v1

Neural machine translation (NMT) has achieved notable performance recently. However, this approach has not been widely applied to the translation task between Chinese and Uyghur, partly due to the limited parallel data resource and the large proportion of rare words caused by the agglutinative nature of Uyghur. In this paper, we collect ~200,000 sentence pairs and show that with this middle-scale database, an attention-based NMT can perform very well on Chinese-Uyghur/Uyghur-Chinese translation. To tackle rare words, we propose a novel memory structure to assist the NMT inference. Our experiments demonstrated that the memory-augmented NMT (M-NMT) outperforms both the vanilla NMT and the phrase-based statistical machine translation (SMT). Interestingly, the memory structure provides an elegant way for dealing with words that are out of vocabulary.

• [cs.CL]Neural Question Answering at BioASQ 5B
Georg Wiese, Dirk Weissenborn, Mariana Neves
http://arxiv.org/abs/1706.08568v1

This paper describes our submission to the 2017 BioASQ challenge. We participated in Task B, Phase B which is concerned with biomedical question answering (QA). We focus on factoid and list question, using an extractive QA model, that is, we restrict our system to output substrings of the provided text snippets. At the core of our system, we use FastQA, a state-of-the-art neural QA system. We extended it with biomedical word embeddings and changed its answer layer to be able to answer list questions in addition to factoid questions. We pre-trained the model on a large-scale open-domain QA dataset, SQuAD, and then fine-tuned the parameters on the BioASQ training set. With our approach, we achieve state-of-the-art results on factoid questions and competitive results on list questions.

• [cs.CL]The Minor Fall, the Major Lift: Inferring Emotional Valence of Musical Chords through Lyrics
Artemy Kolchinsky, Nakul Dhande, Kengjeun Park, Yong-Yeol Ahn
http://arxiv.org/abs/1706.08609v1

We investigate the association between musical chords and lyrics by analyzing a large dataset of user-contributed guitar tablatures. Motivated by the idea that the emotional content of chords is reflected in the words used in corresponding lyrics, we analyze associations between lyrics and chord categories. We also examine the usage patterns of chords and lyrics in different musical genres, historical eras, and geographical regions. Our overall results confirms a previously known association between Major chords and positive valence. We also report a wide variation in this association across regions, genres, and eras. Our results suggest possible existence of different emotional associations for other types of chords.

• [cs.CV]A Unified approach for Conventional Zero-shot, Generalized Zero-shot and Few-shot Learning
Shafin Rahman, Salman H. Khan, Fatih Porikli
http://arxiv.org/abs/1706.08653v1

Prevalent techniques in zero-shot learning do not generalize well to other related problem scenarios. Here, we present a unified approach for conventional zero-shot, generalized zero-shot and few-shot learning problems. Our approach is based on a novel Class Adapting Principal Directions (CAPD) concept that allows multiple embeddings of image features into a semantic space. Given an image, our method produces one principal direction for each seen class. Then, it learns how to combine these directions to obtain the principal direction for each unseen class such that the CAPD of the test image is aligned with the semantic embedding of the true class, and opposite to the other classes. This allows efficient and class-adaptive information transfer from seen to unseen classes. In addition, we propose an automatic process for selection of the most useful seen classes for each unseen class to achieve robustness in zero-shot learning. Our method can update the unseen CAPD taking the advantages of few unseen images to work in a few-shot learning scenario. Furthermore, our method can generalize the seen CAPDs by estimating seen-unseen diversity that significantly improves the performance of generalized zero-shot learning. Our extensive evaluations demonstrate that the proposed approach consistently achieves superior performance in zero-shot, generalized zero-shot and few/one-shot learning problems.

• [cs.CV]Approximate Reflection Symmetry in a Point Set: Theory and Algorithm with an Application
Rajendra Nagar, Shanmuganathan Raman
http://arxiv.org/abs/1706.08801v1

We propose an algorithm to detect approximate reflection symmetry present in a set of volumetrically distributed points belonging to $\mathbb{R}^d$ containing a distorted reflection symmetry pattern. We pose the problem of detecting approximate reflection symmetry as the problem of establishing the correspondences between the points which are reflections of each other and determining the reflection symmetry transformation. We formulate an optimization framework in which the problem of establishing the correspondences amounts to solving a linear assignment problem and the problem of determining the reflection symmetry transformation amounts to an optimization problem on a smooth Riemannian product manifold. The proposed approach estimates the symmetry from the distribution of the points and is descriptor independent. We evaluate the robustness of our approach by varying the amount of distortion in a perfect reflection symmetry pattern where we perturb each point by a different amount of perturbation. We demonstrate the effectiveness of the method by applying it to the problem of 2-D reflection symmetry detection along with relevant comparisons.

• [cs.CV]Auto-Encoder Guided GAN for Chinese Calligraphy Synthesis
Pengyuan Lyu, Xiang Bai, Cong Yao, Zhen Zhu, Tengteng Huang, Wenyu Liu
http://arxiv.org/abs/1706.08789v1

In this paper, we investigate the Chinese calligraphy synthesis problem: synthesizing Chinese calligraphy images with specified style from standard font(eg. Hei font) images (Fig. 1(a)). Recent works mostly follow the stroke extraction and assemble pipeline which is complex in the process and limited by the effect of stroke extraction. We treat the calligraphy synthesis problem as an image-to-image translation problem and propose a deep neural network based model which can generate calligraphy images from standard font images directly. Besides, we also construct a large scale benchmark that contains various styles for Chinese calligraphy synthesis. We evaluate our method as well as some baseline methods on the proposed dataset, and the experimental results demonstrate the effectiveness of our proposed model.

• [cs.CV]Cross-Country Skiing Gears Classification using Deep Learning
Aliaa Rassem, Mohammed El-Beltagy, Mohamed Saleh
http://arxiv.org/abs/1706.08924v1

Human Activity Recognition has witnessed a significant progress in the last decade. Although a great deal of work in this field goes in recognizing normal human activities, few studies focused on identifying motion in sports. Recognizing human movements in different sports has high impact on understanding the different styles of humans in the play and on improving their performance. As deep learning models proved to have good results in many classification problems, this paper will utilize deep learning to classify cross-country skiing movements, known as gears, collected using a 3D accelerometer. It will also provide a comparison between different deep learning models such as convolutional and recurrent neural networks versus standard multi-layer perceptron. Results show that deep learning is more effective and has the highest classification accuracy.

• [cs.CV]Dense Non-rigid Structure-from-Motion Made Easy - A Spatial-Temporal Smoothness based Solution
Yuchao Dai, Huizhong Deng, Mingyi He
http://arxiv.org/abs/1706.08629v1

This paper proposes a simple spatial-temporal smoothness based method for solving dense non-rigid structure-from-motion (NRSfM). First, we revisit the temporal smoothness and demonstrate that it can be extended to dense case directly. Second, we propose to exploit the spatial smoothness by resorting to the Laplacian of the 3D non-rigid shape. Third, to handle real world noise and outliers in measurements, we robustify the data term by using the $L_1$ norm. In this way, our method could robustly exploit both spatial and temporal smoothness effectively and make dense non-rigid reconstruction easy. Our method is very easy to implement, which involves solving a series of least squares problems. Experimental results on both synthetic and real image dense NRSfM tasks show that the proposed method outperforms state-of-the-art dense non-rigid reconstruction methods.

• [cs.CV]Detecting Small Signs from Large Images
Zibo Meng, Xiaochuan Fan, Xin Chen, Min Chen, Yan Tong
http://arxiv.org/abs/1706.08574v1

In the past decade, Convolutional Neural Networks (CNNs) have been demonstrated successful for object detections. However, the size of network input is limited by the amount of memory available on GPUs. Moreover, performance degrades when detecting small objects. To alleviate the memory usage and improve the performance of detecting small traffic signs, we proposed an approach for detecting small traffic signs from large images under real world conditions. In particular, large images are broken into small patches as input to a Small-Object-Sensitive-CNN (SOS-CNN) modified from a Single Shot Multibox Detector (SSD) framework with a VGG-16 network as the base network to produce patch-level object detection results. Scale invariance is achieved by applying the SOS-CNN on an image pyramid. Then, image-level object detection is obtained by projecting all the patch-level detection results to the image at the original scale. Experimental results on a real-world conditioned traffic sign dataset have demonstrated the effectiveness of the proposed method in terms of detection accuracy and recall, especially for those with small sizes.

• [cs.CV]Do Deep Neural Networks Suffer from Crowding?
Anna Volokitin, Gemma Roig, Tomaso Poggio
http://arxiv.org/abs/1706.08616v1

Crowding is a visual effect suffered by humans, in which an object that can be recognized in isolation can no longer be recognized when other objects, called flankers, are placed close to it. In this work, we study the effect of crowding in artificial Deep Neural Networks for object recognition. We analyze both standard deep convolutional neural networks (DCNNs) as well as a new version of DCNNs which is 1) multi-scale and 2) with size of the convolution filters change depending on the eccentricity wrt to the center of fixation. Such networks, that we call eccentricity-dependent, are a computational model of the feedforward path of the primate visual cortex. Our results reveal that the eccentricity-dependent model, trained on target objects in isolation, can recognize such targets in the presence of flankers, if the targets are near the center of the image, whereas DCNNs cannot. Also, for all tested networks, when trained on targets in isolation, we find that recognition accuracy of the networks decreases the closer the flankers are to the target and the more flankers there are. We find that visual similarity between the target and flankers also plays a role and that pooling in early layers of the network leads to more crowding. Additionally, we show that incorporating the flankers into the images of the training set does not improve performance with crowding.

• [cs.CV]Fast and accurate classification of echocardiograms using deep learning
Ali Madani, Ramy Arnaout, Mohammad Mofrad, Rima Arnaout
http://arxiv.org/abs/1706.08658v1

Echocardiography is essential to modern cardiology. However, human interpretation limits high throughput analysis, limiting echocardiography from reaching its full clinical and research potential for precision medicine. Deep learning is a cutting-edge machine-learning technique that has been useful in analyzing medical images but has not yet been widely applied to echocardiography, partly due to the complexity of echocardiograms' multi view, multi modality format. The essential first step toward comprehensive computer assisted echocardiographic interpretation is determining whether computers can learn to recognize standard views. To this end, we anonymized 834,267 transthoracic echocardiogram (TTE) images from 267 patients (20 to 96 years, 51 percent female, 26 percent obese) seen between 2000 and 2017 and labeled them according to standard views. Images covered a range of real world clinical variation. We built a multilayer convolutional neural network and used supervised learning to simultaneously classify 15 standard views. Eighty percent of data used was randomly chosen for training and 20 percent reserved for validation and testing on never seen echocardiograms. Using multiple images from each clip, the model classified among 12 video views with 97.8 percent overall test accuracy without overfitting. Even on single low resolution images, test accuracy among 15 views was 91.7 percent versus 70.2 to 83.5 percent for board-certified echocardiographers. Confusional matrices, occlusion experiments, and saliency mapping showed that the model finds recognizable similarities among related views and classifies using clinically relevant image features. In conclusion, deep neural networks can classify essential echocardiographic views simultaneously and with high accuracy. Our results provide a foundation for more complex deep learning assisted echocardiographic interpretation.

• [cs.CV]Hierarchical Model for Long-term Video Prediction
Peter Wang, Zhongxia Yan, Jeff Zhang
http://arxiv.org/abs/1706.08665v1

Video prediction has been an active topic of research in the past few years. Many algorithms focus on pixel-level predictions, which generates results that blur and disintegrate within a few frames. In this project, we use a hierarchical approach for long-term video prediction. We aim at estimating high-level structure in the input frame first, then predict how that structure grows in the future. Finally, we use an image analogy network to recover a realistic image from the predicted structure. Our method is largely adopted from the work by Villegas et al. The method is built with a combination of LSTMs and analogy-based convolutional auto-encoder networks. Additionally, in order to generate more realistic frame predictions, we also adopt adversarial loss. We evaluate our method on the Penn Action dataset, and demonstrate good results on high-level long-term structure prediction.

• [cs.CV]Illuminating Pedestrians via Simultaneous Detection & Segmentation
Garrick Brazil, Xi Yin, Xiaoming Liu
http://arxiv.org/abs/1706.08564v1

Pedestrian detection is a critical problem in computer vision with significant impact on safety in urban autonomous driving. In this work, we explore how semantic segmentation can be used to boost pedestrian detection accuracy while having little to no impact on network efficiency. We propose a segmentation infusion network to enable joint supervision on semantic segmentation and pedestrian detection. When placed properly, the additional supervision helps guide features in shared layers to become more sophisticated and helpful for the downstream pedestrian detector. Using this approach, we find weakly annotated boxes to be sufficient for considerable performance gains. We provide an in-depth analysis to demonstrate how shared layers are shaped by the segmentation supervision. In doing so, we show that the resulting feature maps become more semantically meaningful and robust to shape and occlusion. Overall, our simultaneous detection and segmentation framework achieves a considerable gain over the state-of-the-art on the Caltech pedestrian dataset, competitive performance on KITTI, and executes 2x faster than competitive methods.

• [cs.CV]Independent motion detection with event-driven cameras
Valentina Vasco, Arren Glover, Elias Mueggler, Davide Scaramuzza, Lorenzo Natale, Chiara Bartolozzi
http://arxiv.org/abs/1706.08713v1

Unlike standard cameras that send intensity images at a constant frame rate, event-driven cameras asynchronously report pixel-level brightness changes, offering low latency and high temporal resolution (both in the order of micro-seconds). As such, they have great potential for fast and low power vision algorithms for robots. Visual tracking, for example, is easily achieved even for very fast stimuli, as only moving objects cause brightness changes. However, cameras mounted on a moving robot are typically non-stationary and the same tracking problem becomes confounded by background clutter events due to the robot ego-motion. In this paper, we propose a method for segmenting the motion of an independently moving object for event-driven cameras. Our method detects and tracks corners in the event stream and learns the statistics of their motion as a function of the robot's joint velocities when no independently moving objects are present. During robot operation, independently moving objects are identified by discrepancies between the predicted corner velocities from ego-motion and the measured corner velocities. We validate the algorithm on data collected from the neuromorphic iCub robot. We achieve a precision of ~ 90 and show that the method is robust to changes in speed of both the head and the target.

• [cs.CV]Large-scale Datasets: Faces with Partial Occlusions and Pose Variations in the Wild
Tarik Alafif, Zeyad Hailat, Melih Aslan, Xuewen Chen
http://arxiv.org/abs/1706.08690v1

Face detection methods have relied on face datasets for training. However, existing face datasets tend to be in small scales for face learning in both constrained and unconstrained environments. In this paper, we first introduce our large-scale image datasets, Large-scale Labeled Face (LSLF) and noisy Large-scale Labeled Non-face (LSLNF). Our LSLF dataset consists of a large number of unconstrained multi-view and partially occluded faces. The faces have many variations in color and grayscale, image quality, image resolution, image illumination, image background, image illusion, human face, cartoon face, facial expression, light and severe partial facial occlusion, make up, gender, age, and race. Many of these faces are partially occluded with accessories such as tattoos, hats, glasses, sunglasses, hands, hair, beards, scarves, microphones, or other objects or persons. The LSLF dataset is currently the largest labeled face image dataset in the literature in terms of the number of labeled images and the number of individuals compared to other existing labeled face image datasets. Second, we introduce our CrowedFaces and CrowedNonFaces image datasets. The crowedFaces and CrowedNonFaces datasets include faces and non-faces images from crowed scenes. These datasets essentially aim for researchers to provide a large number of training examples with many variations for large scale face learning and face recognition tasks.

• [cs.CV]Recurrent Residual Learning for Action Recognition
Ahsan Iqbal, Alexander Richard, Hilde Kuehne, Juergen Gall
http://arxiv.org/abs/1706.08807v1

Action recognition is a fundamental problem in computer vision with a lot of potential applications such as video surveillance, human computer interaction, and robot learning. Given pre-segmented videos, the task is to recognize actions happening within videos. Historically, hand crafted video features were used to address the task of action recognition. With the success of Deep ConvNets as an image analysis method, a lot of extensions of standard ConvNets were purposed to process variable length video data. In this work, we propose a novel recurrent ConvNet architecture called recurrent residual networks to address the task of action recognition. The approach extends ResNet, a state of the art model for image classification. While the original formulation of ResNet aims at learning spatial residuals in its layers, we extend the approach by introducing recurrent connections that allow to learn a spatio-temporal residual. In contrast to fully recurrent networks, our temporal connections only allow a limited range of preceding frames to contribute to the output for the current frame, enabling efficient training and inference as well as limiting the temporal context to a reasonable local range around each frame. On a large-scale action recognition dataset, we show that our model improves over both, the standard ResNet architecture and a ResNet extended by a fully recurrent layer.

• [cs.CV]Robust Sonar ATR Through Bayesian Pose Corrected Sparse Classification
John McKay, Vishal Monga, Raghu G. Raj
http://arxiv.org/abs/1706.08590v1

Sonar imaging has seen vast improvements over the last few decades due in part to advances in synthetic aperture Sonar (SAS). Sophisticated classification techniques can now be used in Sonar automatic target recognition (ATR) to locate mines and other threatening objects. Among the most promising of these methods is sparse reconstruction-based classification (SRC) which has shown an impressive resiliency to noise, blur, and occlusion. We present a coherent strategy for expanding upon SRC for Sonar ATR that retains SRC's robustness while also being able to handle targets with diverse geometric arrangements, bothersome Rayleigh noise, and unavoidable background clutter. Our method, pose corrected sparsity (PCS), incorporates a novel interpretation of a spike and slab probability distribution towards use as a Bayesian prior for class-specific discrimination in combination with a dictionary learning scheme for localized patch extractions. Additionally, PCS offers the potential for anomaly detection in order to avoid false identifications of tested objects from outside the training set with no additional training required. Compelling results are shown using a database provided by the United States Naval Surface Warfare Center.

• [cs.CV]Robust Video-Based Eye Tracking Using Recursive Estimation of Pupil Characteristics
Terence Brouns
http://arxiv.org/abs/1706.08189v2

Video-based eye tracking is a valuable technique in various research fields. Numerous open-source eye tracking algorithms have been developed in recent years, primarily designed for general application with many different camera types. These algorithms do not, however, capitalize on the high frame rate of eye tracking cameras often employed in psychophysical studies. We present a pupil detection method that utilizes this high-speed property to obtain reliable predictions through recursive estimation about certain pupil characteristics in successive camera frames. These predictions are subsequently used to carry out novel image segmentation and classification routines to improve pupil detection performance. Based on results from hand-labelled eye images, our approach was found to have a greater detection rate, accuracy and speed compared to other recently published open-source pupil detection algorithms. The program's source code, together with a graphical user interface, can be downloaded at https://github.com/tbrouns/eyestalker

• [cs.CV]Topometric Localization with Deep Learning
Gabriel L. Oliveira, Noha Radwan, Wolfram Burgard, Thomas Brox
http://arxiv.org/abs/1706.08775v1

Compared to LiDAR-based localization methods, which provide high accuracy but rely on expensive sensors, visual localization approaches only require a camera and thus are more cost-effective while their accuracy and reliability typically is inferior to LiDAR-based methods. In this work, we propose a vision-based localization approach that learns from LiDAR-based localization methods by using their output as training data, thus combining a cheap, passive sensor with an accuracy that is on-par with LiDAR-based localization. The approach consists of two deep networks trained on visual odometry and topological localization, respectively, and a successive optimization to combine the predictions of these two networks. We evaluate the approach on a new challenging pedestrian-based dataset captured over the course of six months in varying weather conditions with a high degree of noise. The experiments demonstrate that the localization errors are up to 10 times smaller than with traditional vision-based localization methods.

• [cs.CV]Training a Fully Convolutional Neural Network to Route Integrated Circuits
Sambhav R. Jain, Kye Okabe
http://arxiv.org/abs/1706.08948v1

We present a deep, fully convolutional neural network that learns to route a circuit layout net with appropriate choice of metal tracks and wire class combinations. Inputs to the network are the encoded layouts containing spatial location of pins to be routed. After 15 fully convolutional stages followed by a score comparator, the network outputs 8 layout layers (corresponding to 4 route layers, 3 via layers and an identity-mapped pin layer) which are then decoded to obtain the routed layouts. We formulate this as a binary segmentation problem on a per-pixel per-layer basis, where the network is trained to correctly classify pixels in each layout layer to be 'on' or 'off'. To demonstrate learnability of layout design rules, we train the network on a dataset of 50,000 train and 10,000 validation samples that we generate based on certain pre-defined layout constraints. Precision, recall and $F_1$ score metrics are used to track the training progress. Our network achieves $F_1\approx97%$ on the train set and $F_1\approx92%$ on the validation set. We use PyTorch for implementing our model.

• [cs.CY]Democratizing Design for Future Computing Platforms
Luis Ceze, Mark D. Hill, Karthikeyan Sankaralingam, Thomas F. Wenisch
http://arxiv.org/abs/1706.08597v1

Information and communications technology can continue to change our world. These advances will partially depend upon designs that synergistically combine software with specialized hardware. Today open-source software incubates rapid software-only innovation. The government can unleash software-hardware innovation with programs to develop open hardware components, tools, and design flows that simplify and reduce the cost of hardware design. Such programs will speed development for startup companies, established industry leaders, education, scientific research, and for government intelligence and defense platforms.

• [cs.IR]Classical Music Clustering Based on Acoustic Features
Xindi Wang, Syed Arefinul Haque
http://arxiv.org/abs/1706.08928v1

In this paper we cluster 330 classical music pieces collected from MusicNet database based on their musical note sequence. We use shingling and chord trajectory matrices to create signature for each music piece and performed spectral clustering to find the clusters. Based on different resolution, the output clusters distinctively indicate composition from different classical music era and different composing style of the musicians.

• [cs.IR]DE-PACRR: Exploring Layers Inside the PACRR Model
Andrew Yates, Kai Hui
http://arxiv.org/abs/1706.08746v1

Recent neural IR models have demonstrated deep learning's utility in ad-hoc information retrieval. However, deep models have a reputation for being black boxes, and the roles of a neural IR model's components may not be obvious at first glance. In this work, we attempt to shed light on the inner workings of a recently proposed neural IR model, namely the PACRR model, by visualizing the output of intermediate layers and by investigating the relationship between intermediate weights and the ultimate relevance score produced. We highlight several insights, hoping that such insights will be generally applicable.

• [cs.IT]Beamforming and Scheduling for mmWave Downlink Sparse Virtual Channels With Non-Orthogonal and Orthogonal Multiple Access
Alessandro Brighente, Stefano Tomasin
http://arxiv.org/abs/1706.08745v1

We consider the problem of scheduling and power allocation for the downlink of a 5G cellular system operating in the millimeter wave (mmWave) band and serving two sets of users: fix-rate (FR) users typically seen in device-to-device (D2D) communications, and variable-rate (VR) users, or high data rate services. The scheduling objective is the weighted sum-rate of both FR and VR users, and the constraints ensure that active FR users get the required rate. The weights of the objective function provide a trade-off between the number of served FR users and the resources allocated to VR users. For mmWave channels the virtual channel matrix obtained by applying fixed discrete-Fourier transform (DFT) beamformers at both the transmitter and the receiver is sparse. This results into a sparsity of the resulting multiple access channel, which is exploited to simplify scheduling, first establishing an interference graph among users and then grouping users according to their orthogonality. The original scheduling problem is solved using a graph-coloring algorithm on the interference graph in order to select sub-sets of orthogonal VR users. Two options are considered for FR users: either they are chosen orthogonal to VR users or non-orthogonal. A waterfilling algorithm is then used to allocate power to the FR users.

• [cs.IT]Beyond Moore-Penrose Part II: The Sparse Pseudoinverse
Ivan Dokmanić, Rémi Gribonval
http://arxiv.org/abs/1706.08701v1

This is the second part of a two-paper series on generalized inverses that minimize matrix norms. In Part II we focus on generalized inverses that are minimizers of entrywise p norms whose main representative is the sparse pseudoinverse for $p = 1$. We are motivated by the idea to replace the Moore-Penrose pseudoinverse by a sparser generalized inverse which is in some sense well-behaved. Sparsity implies that it is faster to apply the resulting matrix; well-behavedness would imply that we do not lose much in stability with respect to the least-squares performance of the MPP. We first address questions of uniqueness and non-zero count of (putative) sparse pseu-doinverses. We show that a sparse pseudoinverse is generically unique, and that it indeed reaches optimal sparsity for almost all matrices. We then turn to proving our main stability result: finite-size concentration bounds for the Frobenius norm of p-minimal inverses for $1 $\le$ p $\le$ 2$. Our proof is based on tools from convex analysis and random matrix theory, in particular the recently developed convex Gaussian min-max theorem. Along the way we prove several results about sparse representations and convex programming that were known folklore, but of which we could find no proof.

• [cs.IT]Centralized and Distributed Sparsification for Low-Complexity Message Passing Algorithm in C-RAN Architectures
Alessandro Brighente, Stefano Tomasin
http://arxiv.org/abs/1706.08762v1

Cloud radio access network (C-RAN) is a promising technology for fifth-generation (5G) cellular systems. However the burden imposed by the huge amount of data to be collected (in the uplink) from the radio remote heads (RRHs) and processed at the base band unit (BBU) poses serious challenges. In order to reduce the computation effort of minimum mean square error (MMSE) receiver at the BBU the Gaussian message passing (MP) together with a suitable sparsification of the channel matrix can be used. In this paper we propose two sets of solutions, either centralized or distributed ones. In the centralized solutions, we propose different approaches to sparsify the channel matrix, in order to reduce the complexity of MP. However these approaches still require that all signals reaching the RRH are conveyed to the BBU, therefore the communication requirements among the backbone network devices are unaltered. In the decentralized solutions instead we aim at reducing both the complexity of MP at the BBU and the requirements on the RRHs-BBU communication links by pre-processing the signals at the RRH and convey a reduced set of signals to the BBU.

• [cs.IT]Constant composition codes derived from linear codes
Long Yu, Xiusheng Liu
http://arxiv.org/abs/1706.08900v1

In this paper, we propose a class of linear codes and obtain their weight distribution. Some of these codes are almost optimal. Moreover, several classes of constant composition codes(CCCs) are constructed as subcodes of linear codes.

• [cs.IT]DFE/THP duality for FBMC with highly frequency selective channels
Hela Jedda, Leonardo G. Baltar, Oliver De Candido, Amine Mezghani, Josef A. Nossek
http://arxiv.org/abs/1706.08718v1

Filter bank based multicarrier with Offset-QAM systems (FBMC/OQAM) are strong candidates for the waveform of future 5-th generation (5G) wireless standards. These systems can achieve maximum spectral efficiency compared to other multicarrier schemes, particularly in highly frequency selective propagation conditions. In this case a multi-tap, fractionally spaced equalizer or precoder needs to be inserted in each subcarrier at the receiver or transmitter side to compensate inter-symbol interference (ISI) and inter-carrier interference (ICI). In this paper we propose a new Tomlinson-Harashima precoder (THP) design for FBMC/OQAM based on the mean squared error (MSE) duality from a minimum MSE (MMSE) designed decision feedback equalizer (DFE).

• [cs.IT]Fountain Codes under Maximum Likelihood Decoding
Francisco Lázaro
http://arxiv.org/abs/1706.08739v1

This dissertation focuses on fountain codes under maximum likelihood (ML) decoding. First LT codes are considered under a practical and widely used ML decoding algorithm known as inactivation decoding. Different analysis techniques are presented to characterize the decoding complexity. Next an upper bound to the probability of decoding failure of Raptor codes under ML decoding is provided. Then, the distance properties of an ensemble of fixed-rate Raptor codes with linear random outer codes are analyzed. Finally, a novel class of fountain codes is presented, which consists of a parallel concatenation of a block code with a linear random fountain code.

• [cs.IT]Invariant components of synergy, redundancy, and unique information among three variables
Giuseppe Pica, Eugenio Piasini, Daniel Chicharro, Stefano Panzeri
http://arxiv.org/abs/1706.08921v1

In a system of three stochastic variables, the Partial Information Decomposition (PID) of Williams and Beer dissects the information that two variables (sources) carry about a third variable (target) into nonnegative information atoms that describe redundant, unique, and synergistic modes of dependencies among the variables. However, the classification of the three variables into two sources and one target limits the dependency modes that can be quantitatively resolved, and does not naturally suit all systems. Here, we extend the PID to describe trivariate modes of dependencies in full generality, without introducing additional decomposition axioms or making assumptions about the target/source nature of the variables. By comparing different PID lattices of the same system, we unveil a finer PID structure made of seven nonnegative information subatoms that are invariant to different target/source classifications and that are sufficient to construct any PID lattice. This finer structure naturally splits redundant information into two nonnegative components: the source redundancy, which arises from the pairwise correlations between the source variables, and the non-source redundancy, which does not, and relates to the synergistic information the sources carry about the target. The invariant structure is also sufficient to construct the system's entropy, hence it characterizes completely all the interdependencies in the system.

• [cs.IT]MMSE precoder for massive MIMO using 1-bit quantization
Ovais Bin Usman, Hela Jedda, Amine Mezghani, Josef A. Nossek
http://arxiv.org/abs/1706.08717v1

We propose a novel linear minimum-mean-squared-error (MMSE) precoder design for a downlink (DL) massive multiple-input-multiple-output (MIMO) scenario. For economical and computational efficiency reasons low resolution 1-bit digital-to-analog (DAC) and analog-to-digital (ADC) converters are used. This comes at the cost of performance gain that can be recovered by the large number of antennas deployed at the base station (BS) and an appropiate precoder design to mitigate the distortions due to the coarse quantization. The proposed precoder takes the quantization non-linearities into account and is split into a digital precoder and an analog precoder. We formulate the two-stage precoding problem such that the MSE of the users is minimized under the 1-bit constraint. In the simulations, we compare the new optimized precoding scheme with previously proposed linear precoders in terms of uncoded bit error ratio (BER).

• [cs.IT]Minimum BER Precoding in 1-Bit Massive MIMO Systems
Hela Jedda, Josef A. Nossek, Amine Mezghani
http://arxiv.org/abs/1706.08708v1

1-bit digital-to-analog (DACs) and analog-to-digital converters (ADCs) are gaining more interest in massive MIMO systems for economical and computational efficiency. We present a new precoding technique to mitigate the inter-user-interference (IUI) and the channel distortions in a 1-bit downlink MUMISO system with QPSK symbols. The transmit signal vector is optimized taking into account the 1-bit quantization. We develop a sort of mapping based on a look-up table (LUT) between the input signal and the transmit signal. The LUT is updated for each channel realization. Simulation results show a significant gain in terms of the uncoded bit-error-ratio (BER) compared to the existing linear precoding techniques.

• [cs.IT]NOMA based Random Access with Multichannel ALOHA
Jinho Choi
http://arxiv.org/abs/1706.08799v1

In nonorthogonal multiple access (NOMA), the power difference of multiple signals is exploited for multiple access and successive interference cancellation (SIC) is employed at a receiver to mitigate co-channel interference. Thus, NOMA is usually employed for coordinated transmissions and mostly applied to downlink transmissions where a base station (BS) per- forms coordination for downlink transmissions with full channel state information (CSI). In this paper, however, we show that NOMA can also be employed for non-coordinated transmissions such as random access for uplink transmissions. We apply a NOMA scheme to multichannel ALOHA and show that the throughput can be improved. In particular, the resulting scheme is suitable for random access when the number of subchannels is limited since NOMA can effectively increase the number of subchannels without any bandwidth expansion.

• [cs.IT]NOMA: Principles and Recent Results
Jinho Choi
http://arxiv.org/abs/1706.08805v1

Although non-orthogonal multiple access (NOMA) is recently considered for cellular systems, its key ideas such as successive interference cancellation (SIC) and superposition coding have been well studied in information theory. In this paper, we overview principles of NOMA based on information theory and present some recent results. Under a single-cell environment, we mainly focus on fundamental issues, e.g., power allocation and beamforming for downlink NOMA and coordinated and uncoordinated transmissions for uplink NOMA.

• [cs.IT]PSK Precoding in Multi-User MISO Systems
Andreas Noll, Hela Jedda, Josef A. Nossek
http://arxiv.org/abs/1706.08707v1

We consider the downlink scenario of multiuser multiple-input-single-output (MU-MISO) communication systems with constant envelope (CE) signals emitted from each antenna. This results in energy efficient power amplifiers (PAs). We propose a holistic CE precoding scheme based on the symbol-wise minimum squared error (SMSE) criterion. Additionally, we analyze the distortions introduced by low-resolution quantization to PSK for higher energy efficiency reasons. We present three solution algorithms and examine their performance to decide for the best pick for different quantization resolutions. Our results show that good performance can be achieved with minimal loss compared to an ideal unquantized case. Finally, we analyze and discuss the results and consider the overall complexity of the precoder as well as implementation issues.

• [cs.IT]Power- and Spectral Efficient Communication System Design Using 1-Bit Quantization
Hela Jedda, Muhammad Mudussir Ayub, Jawad Munir, Amine Mezghani, Josef A. Nossek
http://arxiv.org/abs/1706.08709v1

Improving the power efficiency and spectral efficiency of communication systems has been one of the major research goals over the recent years. However, there is a tradeoff in achieving both goals at the same time. In this work, we consider the joint optimization of the power amplifier and a pulse shaping filter over a single-input single-output (SISO) additive white Gaussian noise (AWGN) channel using 1-bit analog-todigital (ADC) and digital-to-analog (DAC) converters. The goal of the optimization is the selection of the optimal system parameters in order to maximize the desired figure-of-merit (FOM) which is the product of power efficiency and spectral efficiency. Simulation results give an insight in choosing the optimal parameters of the pulse shaping filter and power amplifier to maximize the desired FOM.

• [cs.IT]Spatial Coding Based on Minimum BER in 1-Bit Massive MIMO Systems
Hela Jedda, Amine Mezghani, Jawad Munir, Fabian Steiner, Josef A. Nossek
http://arxiv.org/abs/1706.08719v1

We consider a downlink 1-bit quantized multiuser (MU) multiple-input-multiple-output (MIMO) system, where 1-bit digital-to-analog (DACs) and analog-to-digital converters (ADCs) are used at the transmitter and the receiver for economical and computational efficiency. We end up with a discrete memoryless channel with input and output vectors belonging to the QPSK constellation. In the context of massive (MIMO) systems the number of base station (BS) antennas is much larger than the number of receive antennas. This leads to high input cardinality of the channel. In this work we introduce a method to reduce the input set based on the mimimum bit-error-ratio (BER) criterion combined with a non-linear precoding technique. This method is denoted as spatial coding. Simulations show that this spatial coding improves the BER behavior significantly removing the error floor due to coarse quantization.

• [cs.IT]Spectral shaping with low resolution signals
Hela Jedda, Amine Mezghani, Josef A. Nossek
http://arxiv.org/abs/1706.08727v1

We aim at investigating the impact of low resolution digital-to-analog converters (DACs) at the transmitter and low resolution analog-to-digital converters (ADCs) at the receiver on the required bandwidth and the required signalto- noise ratio (SNR). In particular, we consider the extreme case of only 1-bit resolution (with oversampling), where we propose a single carrier system architecture for minimizing the spectral occupation and the required SNR of 1-bit signals. In addition, the receiver is optimized to take into account the effects of quantization at both ends. Through simulations, we show that despite of the coarse quantization, sufficient spectral confinement is still achievable.

• [cs.LG]Exploring Generalization in Deep Learning
Behnam Neyshabur, Srinadh Bhojanapalli, David McAllester, Nathan Srebro
http://arxiv.org/abs/1706.08947v1

With a goal of understanding what drives generalization in deep networks, we consider several recently suggested explanations, including norm-based control, sharpness and robustness. We study how these measures can ensure generalization, highlighting the importance of scale normalization, and making a connection between sharpness and PAC-Bayes theory. We then investigate how well the measures explain different observed phenomena.

• [cs.LG]Fast and robust tensor decomposition with applications to dictionary learning
Tselil Schramm, David Steurer
http://arxiv.org/abs/1706.08672v1

We develop fast spectral algorithms for tensor decomposition that match the robustness guarantees of the best known polynomial-time algorithms for this problem based on the sum-of-squares (SOS) semidefinite programming hierarchy. Our algorithms can decompose a 4-tensor with $n$-dimensional orthonormal components in the presence of error with constant spectral norm (when viewed as an $n^2$-by-$n2$ matrix). The running time is $n^5$ which is close to linear in the input size $n^4$. We also obtain algorithms with similar running time to learn sparsely-used orthogonal dictionaries even when feature representations have constant relative sparsity and non-independent coordinates. The only previous polynomial-time algorithms to solve these problem are based on solving large semidefinite programs. In contrast, our algorithms are easy to implement directly and are based on spectral projections and tensor-mode rearrangements. Or work is inspired by recent of Hopkins, Schramm, Shi, and Steurer (STOC'16) that shows how fast spectral algorithms can achieve the guarantees of SOS for average-case problems. In this work, we introduce general techniques to capture the guarantees of SOS for worst-case problems.

• [cs.LG]Forecasting and Granger Modelling with Non-linear Dynamical Dependencies
Magda Gregorová, Alexandros Kalousis, Stéphane Marchand-Maillet
http://arxiv.org/abs/1706.08811v1

Traditional linear methods for forecasting multivariate time series are not able to satisfactorily model the non-linear dependencies that may exist in non-Gaussian series. We build on the theory of learning vector-valued functions in the reproducing kernel Hilbert space and develop a method for learning prediction functions that accommodate such non-linearities. The method not only learns the predictive function but also the matrix-valued kernel underlying the function search space directly from the data. Our approach is based on learning multiple matrix-valued kernels, each of those composed of a set of input kernels and a set of output kernels learned in the cone of positive semi-definite matrices. In addition to superior predictive performance in the presence of strong non-linearities, our method also recovers the hidden dynamic relationships between the series and thus is a new alternative to existing graphical Granger techniques.

• [cs.LG]GANs Trained by a Two Time-Scale Update Rule Converge to a Nash Equilibrium
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, Günter Klambauer, Sepp Hochreiter
http://arxiv.org/abs/1706.08500v2

Generative Adversarial Networks (GANs) excel at creating realistic images with complex models for which maximum likelihood is infeasible. However, the convergence of GAN training has still not been proved. We propose a two time-scale update rule (TTUR) for training GANs with stochastic gradient descent that has an individual learning rate for both the discriminator and the generator. We prove that the TTUR converges under mild assumptions to a stationary Nash equilibrium. The convergence carries over to the popular Adam optimization, for which we prove that it follows the dynamics of a heavy ball with friction and thus prefers flat minima in the objective landscape. For the evaluation of the performance of GANs at image generation, we introduce the "Fr'echet Inception Distance" (FID) which captures the similarity of generated images to real ones better than the Inception Score. In experiments, TTUR improves learning for DCGANs, improved Wasserstein GANs, and BEGANs, outperforming conventional GAN training on CelebA, Billion Word Benchmark, and LSUN bedrooms. Implementations are available at: https://github.com/bioinf-jku/TTUR.

• [cs.LG]Gradient Episodic Memory for Continuum Learning
David Lopez-Paz, Marc'Aurelio Ranzato
http://arxiv.org/abs/1706.08840v1

One major obstacle towards artificial intelligence is the poor ability of models to quickly solve new problems, without forgetting previously acquired knowledge. To better understand this issue, we study the problem of learning over a continuum of data, where the model observes, once and one by one, examples concerning an ordered sequence of tasks. First, we propose a set of metrics to evaluate models learning over a continuum of data. These metrics characterize models not only by their test accuracy, but also in terms of their ability to transfer knowledge across tasks. Second, we propose a model to learn over continuums of data, called Gradient of Episodic Memory (GEM), which alleviates forgetting while allowing beneficial transfer of knowledge to previous tasks. Our experiments on variants of the MNIST and CIFAR-100 datasets demonstrate the strong performance of GEM when compared to the state-of-the-art.

• [cs.LG]Learning Local Feature Aggregation Functions with Backpropagation
Angelos Katharopoulos, Despoina Paschalidou, Christos Diou, Anastasios Delopoulos
http://arxiv.org/abs/1706.08580v1

This paper introduces a family of local feature aggregation functions and a novel method to estimate their parameters, such that they generate optimal representations for classification (or any task that can be expressed as a cost function minimization problem). To achieve that, we compose the local feature aggregation function with the classifier cost function and we backpropagate the gradient of this cost function in order to update the local feature aggregation function parameters. Experiments on synthetic datasets indicate that our method discovers parameters that model the class-relevant information in addition to the local feature space. Further experiments on a variety of motion and visual descriptors, both on image and video datasets, show that our method outperforms other state-of-the-art local feature aggregation functions, such as Bag of Words, Fisher Vectors and VLAD, by a large margin.

• [cs.LG]Preserving Differential Privacy in Convolutional Deep Belief Networks
NhatHai Phan, Xintao Wu, Dejing Dou
http://arxiv.org/abs/1706.08839v1

The remarkable development of deep learning in medicine and healthcare domain presents obvious privacy issues, when deep neural networks are built on users' personal and highly sensitive data, e.g., clinical records, user profiles, biomedical images, etc. However, only a few scientific studies on preserving privacy in deep learning have been conducted. In this paper, we focus on developing a private convolutional deep belief network (pCDBN), which essentially is a convolutional deep belief network (CDBN) under differential privacy. Our main idea of enforcing epsilon-differential privacy is to leverage the functional mechanism to perturb the energy-based objective functions of traditional CDBNs, rather than their results. One key contribution of this work is that we propose the use of Chebyshev expansion to derive the approximate polynomial representation of objective functions. Our theoretical analysis shows that we can further derive the sensitivity and error bounds of the approximate polynomial representation. As a result, preserving differential privacy in CDBNs is feasible. We applied our model in a health social network, i.e., YesiWell data, and in a handwriting digit dataset, i.e., MNIST data, for human behavior prediction, human behavior classification, and handwriting digit recognition tasks. Theoretical analysis and rigorous experimental evaluations show that the pCDBN is highly effective. It significantly outperforms existing solutions.

• [cs.LG]Reexamining Low Rank Matrix Factorization for Trace Norm Regularization
Carlo Ciliberto, Dimitris Stamos, Massimiliano Pontil
http://arxiv.org/abs/1706.08934v1

Trace norm regularization is a widely used approach for learning low rank matrices. A standard optimization strategy is based on formulating the problem as one of low rank matrix factorization which, however, leads to a non-convex problem. In practice this approach works well, and it is often computationally faster than standard convex solvers such as proximal gradient methods. Nevertheless, it is not guaranteed to converge to a global optimum, and the optimization can be trapped at poor stationary points. In this paper we show that it is possible to characterize all critical points of the non-convex problem. This allows us to provide an efficient criterion to determine whether a critical point is also a global minimizer. Our analysis suggests an iterative meta-algorithm that dynamically expands the parameter space and allows the optimization to escape any non-global critical point, thereby converging to a global minimizer. The algorithm can be applied to problems such as matrix completion or multitask learning, and our analysis holds for any random initialization of the factor matrices. Finally, we confirm the good performance of the algorithm on synthetic and real datasets.

• [cs.MS]Parareal Algorithm Implementation and Simulation in Julia
Tyler M. Masthay, Saverio Perugini
http://arxiv.org/abs/1706.08569v1

We present a full implementation of the parareal algorithm---an integration technique to solve differential equations in parallel---in the Julia programming language for a fully general, first-order, initial-value problem. Our implementation accepts both coarse and fine integrators as functional arguments. We use Euler's method and another Runge-Kutta integration technique as the integrators in our experiments. We also present a simulation of the algorithm for purposes of pedagogy.

• [cs.NE]PasMoQAP: A Parallel Asynchronous Memetic Algorithm for solving the Multi-Objective Quadratic Assignment Problem
Claudio Sanhueza, Francia Jimenez, Regina Berretta, Pablo Moscato
http://arxiv.org/abs/1706.08700v1

Multi-Objective Optimization Problems (MOPs) have attracted growing attention during the last decades. Multi-Objective Evolutionary Algorithms (MOEAs) have been extensively used to address MOPs because are able to approximate a set of non-dominated high-quality solutions. The Multi-Objective Quadratic Assignment Problem (mQAP) is a MOP. The mQAP is a generalization of the classical QAP which has been extensively studied, and used in several real-life applications. The mQAP is defined as having as input several flows between the facilities which generate multiple cost functions that must be optimized simultaneously. In this study, we propose PasMoQAP, a parallel asynchronous memetic algorithm to solve the Multi-Objective Quadratic Assignment Problem. PasMoQAP is based on an island model that structures the population by creating sub-populations. The memetic algorithm on each island individually evolve a reduced population of solutions, and they asynchronously cooperate by sending selected solutions to the neighboring islands. The experimental results show that our approach significatively outperforms all the island-based variants of the multi-objective evolutionary algorithm NSGA-II. We show that PasMoQAP is a suitable alternative to solve the Multi-Objective Quadratic Assignment Problem.

• [cs.NE]Proceedings of the First International Workshop on Deep Learning and Music
Dorien Herremans, Ching-Hua Chuan
http://arxiv.org/abs/1706.08675v1

Proceedings of the First International Workshop on Deep Learning and Music, joint with IJCNN, Anchorage, US, May 17-18, 2017

• [cs.NI]Rate-Distortion Classification for Self-Tuning IoT Networks
Davide Zordan, Michele Rossi, Michele Zorzi
http://arxiv.org/abs/1706.08877v1

Many future wireless sensor networks and the Internet of Things are expected to follow a software defined paradigm, where protocol parameters and behaviors will be dynamically tuned as a function of the signal statistics. New protocols will be then injected as a software as certain events occur. For instance, new data compressors could be (re)programmed on-the-fly as the monitored signal type or its statistical properties change. We consider a lossy compression scenario, where the application tolerates some distortion of the gathered signal in return for improved energy efficiency. To reap the full benefits of this paradigm, we discuss an automatic sensor profiling approach where the signal class, and in particular the corresponding rate-distortion curve, is automatically assessed using machine learning tools (namely, support vector machines and neural networks). We show that this curve can be reliably estimated on-the-fly through the computation of a small number (from ten to twenty) of statistical features on time windows of a few hundreds samples.

• [cs.NI]Self-Sustaining Caching Stations: Towards Cost-Effective 5G-Enabled Vehicular Networks
Shan Zhang, Ning Zhang, Xiaojie Fang, Peng Yang, Xuemin, Shen
http://arxiv.org/abs/1706.08628v1

In this article, we investigate the cost-effective 5G-enabled vehicular networks to support emerging vehicular applications, such as autonomous driving, in-car infotainment and location-based road services. To this end, self-sustaining caching stations (SCSs) are introduced to liberate on-road base stations from the constraints of power lines and wired backhauls. Specifically, the cache-enabled SCSs are powered by renewable energy and connected to core networks through wireless backhauls, which can realize "drop-and-play" deployment, green operation, and low-latency services. With SCSs integrated, a 5G-enabled heterogeneous vehicular networking architecture is further proposed, where SCSs are deployed along roadside for traffic offloading while conventional macro base stations (MBSs) provide ubiquitous coverage to vehicles. In addition, a hierarchical network management framework is designed to deal with high dynamics in vehicular traffic and renewable energy, where content caching, energy management and traffic steering are jointly investigated to optimize the service capability of SCSs with balanced power demand and supply in different time scales. Case studies are provided to illustrate SCS deployment and operation designs, and some open research issues are also discussed.

• [cs.RO]Controlled Tactile Exploration and Haptic Object Recognition
Massimo Regoli, Nawid Jamali, Giorgio Metta, Lorenzo Natale
http://arxiv.org/abs/1706.08697v1

In this paper we propose a novel method for in-hand object recognition. The method is composed of a grasp stabilization controller and two exploratory behaviours to capture the shape and the softness of an object. Grasp stabilization plays an important role in recognizing objects. First, it prevents the object from slipping and facilitates the exploration of the object. Second, reaching a stable and repeatable position adds robustness to the learning algorithm and increases invariance with respect to the way in which the robot grasps the object. The stable poses are estimated using a Gaussian mixture model (GMM). We present experimental results showing that using our method the classifier can successfully distinguish 30 objects.We also compare our method with a benchmark experiment, in which the grasp stabilization is disabled. We show, with statistical significance, that our method outperforms the benchmark method.

• [cs.RO]Material Recognition CNNs and Hierarchical Planning for Biped Robot Locomotion on Slippery Terrain
Martim Brandao, Yukitoshi Minami Shiguematsu, Kenji Hashimoto, Atsuo Takanishi
http://arxiv.org/abs/1706.08685v1

In this paper we tackle the problem of visually predicting surface friction for environments with diverse surfaces, and integrating this knowledge into biped robot locomotion planning. The problem is essential for autonomous robot locomotion since diverse surfaces with varying friction abound in the real world, from wood to ceramic tiles, grass or ice, which may cause difficulties or huge energy costs for robot locomotion if not considered. We propose to estimate friction and its uncertainty from visual estimation of material classes using convolutional neural networks, together with probability distribution functions of friction associated with each material. We then robustly integrate the friction predictions into a hierarchical (footstep and full-body) planning method using chance constraints, and optimize the same trajectory costs at both levels of the planning method for consistency. Our solution achieves fully autonomous perception and locomotion on slippery terrain, which considers not only friction and its uncertainty, but also collision, stability and trajectory cost. We show promising friction prediction results in real pictures of outdoor scenarios, and planning experiments on a real robot facing surfaces with different friction.

• [cs.SD]Gabor frames and deep scattering networks in audio processing
Roswitha Bammer, Monika Dörfler
http://arxiv.org/abs/1706.08818v1

In this paper a feature extractor based on Gabor frames and Mallat's scattering transform, called Gabor scattering, is introduced. This feature extractor is applied to a simple signal model for audio signals, i.e. a class of tones consisting of fundamental frequency and its multiples and an according envelope. Within different layers, different invariances to certain signal features occur. In this paper we give a mathematical explanation for the first and the second layer which are illustrated by numerical examples. Deformation stability of this feature extractor will be shown by using a decoupling technique, previously suggested for the scattering transform of Cartoon functions. Here it is used to see if the feature extractor is robust to changes in spectral shape and frequency modulation.

• [cs.SE]Developing Bug-Free Machine Learning Systems With Formal Mathematics
Daniel Selsam, Percy Liang, David L. Dill
http://arxiv.org/abs/1706.08605v1

Noisy data, non-convex objectives, model misspecification, and numerical instability can all cause undesired behaviors in machine learning systems. As a result, detecting actual implementation errors can be extremely difficult. We demonstrate a methodology in which developers use an interactive proof assistant to both implement their system and to state a formal theorem defining what it means for their system to be correct. The process of proving this theorem interactively in the proof assistant exposes all implementation errors since any error in the program would cause the proof to fail. As a case study, we implement a new system, Certigrad, for optimizing over stochastic computation graphs, and we generate a formal (i.e. machine-checkable) proof that the gradients sampled by the system are unbiased estimates of the true mathematical gradients. We train a variational autoencoder using Certigrad and find the performance comparable to training the same model in TensorFlow.

• [cs.SI]A preference elicitation interface for collecting dense recommender datasets with rich user information
Pantelis P. Analytis, Tobias Schnabel, Stefan Herzog, Daniel Barkoczi, Thorsten Joachims
http://arxiv.org/abs/1706.08184v2

We present an interface that can be leveraged to quickly and effortlessly elicit people's preferences for visual stimuli, such as photographs, visual art and screensavers, along with rich side-information about its users. We plan to employ the new interface to collect dense recommender datasets that will complement existing sparse industry-scale datasets. The new interface and the collected datasets are intended to foster integration of research in recommender systems with research in social and behavioral sciences. For instance, we will use the datasets to assess the diversity of human preferences in different domains of visual experience. Further, using the datasets we will be able to measure crucial psychological effects, such as preference consistency, scale acuity and anchoring biases. Last, we the datasets will facilitate evaluation in counterfactual learning experiments.

• [cs.SI]Second-Order Moment-Closure for Tighter Epidemic Thresholds
Masaki Ogura, Victor M. Preciado
http://arxiv.org/abs/1706.08602v1

In this paper, we study the dynamics of contagious spreading processes taking place in complex contact networks. We specifically present a tight lower-bound on the decay rate of the number of nodes infected by a susceptible-infected-susceptible (SIS) stochastic spreading process. A precise quantification of this decay rate is crucial for designing efficient strategies to contain epidemic outbreaks. However, existing lower-bounds on the decay rate based on first-order mean field approximations are often accompanied by a large error resulting in inefficient containment strategies. To overcome this deficiency, we derive a novel and tight lower-bound based on a second-order moment-closure of the stochastic SIS processes. The proposed second-order bound is theoretically guaranteed to be tighter than existing first-order bounds. We also present various numerical simulations to illustrate how our lower-bound drastically improves the performance of existing first-order lower-bounds in practical scenarios, resulting in more efficient strategies for epidemic containment.

• [cs.SI]Validation of a smartphone app to map social networks of proximity
Tjeerd W. Boonstra, Mark E. Larsen, Samuel Townsend, Helen Christensen
http://arxiv.org/abs/1706.08777v1

Social network analysis is a prominent approach to investigate interpersonal relationships. Most studies use self-report data to quantify the connections between participants and construct social networks. In recent years smartphones have been used as an alternative to map networks by assessing the proximity between participants based on Bluetooth and GPS data. While most studies have handed out specially programmed smartphones to study participants, we developed an application for iOS and Android to collect Bluetooth data from participants own smartphones. In this study, we compared the networks estimated with the smartphone app to those obtained from sociometric badges and self-report data. Participants (n=21) installed the app on their phone and wore a sociometric badge during office hours. Proximity data was collected for 4 weeks. A contingency table revealed a significant association between proximity data (rho = 0.17, p<0.0001), but the marginal odds were higher for the app (8.6%) than for the badges (1.3%), indicating that dyads were more often detected by the app. We then compared the networks that were estimated using the proximity and self-report data. All three networks were significantly correlated, although the correlation with self-reported data was lower for the app (rho = 0.25) than for badges (rho = 0.67). The scanning rates of the app varied considerably between devices and was lower on iOS than on Android. The association between the app and the badges increased when the network was estimated between participants whose app recorded more regularly. These findings suggest that the accuracy of proximity networks can be further improved by reducing missing data and restricting the interpersonal distance at which interactions are detected.

• [cs.SI]White, Man, and Highly Followed: Gender and Race Inequalities in Twitter
Johnnatan Messias, Pantelis Vikatos, Fabricio Benevenuto
http://arxiv.org/abs/1706.08619v1

Social media is considered a democratic space in which people connect and interact with each other regardless of their gender, race, or any other demographic factor. Despite numerous efforts that explore demographic factors in social media, it is still unclear whether social media perpetuates old inequalities from the offline world. In this paper, we attempt to identify gender and race of Twitter users located in U.S. using advanced image processing algorithms from Face++. Then, we investigate how different demographic groups (i.e. male/female, Asian/Black/White) connect with other. We quantify to what extent one group follow and interact with each other and the extent to which these connections and interactions reflect in inequalities in Twitter. Our analysis shows that users identified as White and male tend to attain higher positions in Twitter, in terms of the number of followers and number of times in user's lists. We hope our effort can stimulate the development of new theories of demographic information in the online space.

• [math.NA]Using Frame Theoretic Convolutional Gridding for Robust Synthetic Aperture Sonar Imaging
John McKay, Anne Gelb, Vishal Monga, Raghu Raj
http://arxiv.org/abs/1706.08575v1

Recent progress in synthetic aperture sonar (SAS) technology and processing has led to significant advances in underwater imaging, outperforming previously common approaches in both accuracy and efficiency. There are, however, inherent limitations to current SAS reconstruction methodology. In particular, popular and efficient Fourier domain SAS methods require a 2D interpolation which is often ill conditioned and inaccurate, inevitably reducing robustness with regard to speckle and inaccurate sound-speed estimation. To overcome these issues, we propose using the frame theoretic convolution gridding (FTCG) algorithm to handle the non-uniform Fourier data. FTCG extends upon non-uniform fast Fourier transform (NUFFT) algorithms by casting the NUFFT as an approximation problem given Fourier frame data. The FTCG has been show to yield improved accuracy at little more computational cost. Using simulated data, we outline how the FTCG can be used to enhance current SAS processing.

• [math.ST]Coverage Probability Fails to Ensure Reliable Inference
Michael S. Balch, Ryan Martin, Scott Ferson
http://arxiv.org/abs/1706.08565v1

We present two examples, one pedagogical and one practical, in which confidence distributions support plainly misleading inferences. The root problem in both examples is that the coverage probability criterion allows certain false propositions to have a high probability of being assigned a high degree of confidence. In short, the problem is false confidence. This deficiency is almost universally present in confidence distributions. It is merely not obvious in most problems. Since Bayesian posterior distributions are sometimes rationalized as approximate confidence distributions, this finding has troubling implications for pragmatic users of Bayesian methods. The Martin--Liu validity criterion offers an alternative path for modern frequentist inference. Under this approach, confidence and plausibility assigned to propositions must satisfy a reliability-of-inference interpretation. Consonant confidence structures satisfy this interpretation, and confidence intervals are a special case within that class. So, while confidence distributions fail to provide reliable inference, confidence intervals themselves suffer no such failure. Approaches to inference based solely on coverage probability are the result of a long- and widely-held misunderstanding of confidence intervals and the rigorous inferences that they support. Future efforts to improve upon confidence intervals should be grounded in the stricter Martin--Liu validity criterion.

• [math.ST]Empirical priors and posterior concentration rates for a monotone density
Ryan Martin
http://arxiv.org/abs/1706.08567v1

In a Bayesian context, prior specification for inference on monotone densities is straightforward, but proving posterior convergence theorems is complicated by the fact that desirable prior concentration properties often are not satisfied. In this paper, I first develop a new prior designed specifically to satisfy an empirical version of the prior concentration property, and then I give sufficient conditions on the prior inputs such that the corresponding empirical Bayes posterior concentrates around the true monotone density at nearly the optimal minimax rate.

• [math.ST]Group Synchronization on Grids
Emmanuel Abbe, Laurent Massoulie, Andrea Montanari, Allan Sly, Nikhil Srivastava
http://arxiv.org/abs/1706.08561v1

Group synchronization requires to estimate unknown elements $({\theta}v){v\in V}$ of a compact group ${\mathfrak G}$ associated to the vertices of a graph $G=(V,E)$, using noisy observations of the group differences associated to the edges. This model is relevant to a variety of applications ranging from structure from motion in computer vision to graph localization and positioning, to certain families of community detection problems. We focus on the case in which the graph $G$ is the $d$-dimensional grid. Since the unknowns ${\boldsymbol \theta}_v$ are only determined up to a global action of the group, we consider the following weak recovery question. Can we determine the group difference ${\theta}_u^{-1}{\theta}_v$ between far apart vertices $u, v$ better than by random guessing? We prove that weak recovery is possible (provided the noise is small enough) for $d\ge 3$ and, for certain finite groups, for $d\ge 2$. Viceversa, for some continuous groups, we prove that weak recovery is impossible for $d=2$. Finally, for strong enough noise, weak recovery is always impossible.

• [math.ST]Laplace deconvolution in the presence of indirect long-memory data
Rida Benhaddou
http://arxiv.org/abs/1706.08648v1

We investigate the problem of estimating a function $f$ based on observations from its noisy convolution when the noise exhibits long-range dependence. We construct an adaptive estimator based on the kernel method, derive minimax lower bound for the $L^2$-risk when $f$ belongs to Sobolev space and show that such estimator attains optimal rates that deteriorate as the LRD worsens.

• [math.ST]New insights into non-central beta distributions
Carlo Orsi
http://arxiv.org/abs/1706.08557v1

The beta family owes its privileged status within unit interval distributions to several relevant features such as, for example, easyness of interpretation and versatility in modeling different types of data. However, its flexibility at the unit interval endpoints is poor enough to prevent from properly modeling the portions of data having values next to zero and one. Such a drawback can be overcome by resorting to the class of the non-central beta distributions. Indeed, the latter allows the density to take on arbitrary positive and finite limits which have a really simple form. That said, new insights into such class are provided in this paper. In particular, new representations and moments expressions are derived. Moreover, its potential with respect to alternative models is highlighted through applications to real data.

• [math.ST]Robust Sparse Covariance Estimation by Thresholding Tyler's M-Estimator
John Goes, Gilad Lerman, Boaz Nadler
http://arxiv.org/abs/1706.08020v2

Estimating a high-dimensional sparse covariance matrix from a limited number of samples is a fundamental problem in contemporary data analysis. Most proposals to date, however, are not robust to outliers or heavy tails. Towards bridging this gap, in this work we consider estimating a sparse shape matrix from $n$ samples following a possibly heavy tailed elliptical distribution. We propose estimators based on thresholding either Tyler's M-estimator or its regularized variant. We derive bounds on the difference in spectral norm between our estimators and the shape matrix in the joint limit as the dimension $p$ and sample size $n$ tend to infinity with $p/n\to\gamma>0$. These bounds are minimax rate-optimal. Results on simulated data support our theoretical analysis.

• [q-bio.PE]Well-supported phylogenies using largest subsets of core-genes by discrete particle swarm optimization
Reem Alsrraj, Bassam AlKindy, Christophe Guyeux, Laurent Philippe, Jean-François Couchot
http://arxiv.org/abs/1706.08514v1

The number of complete chloroplastic genomes increases day after day, making it possible to rethink plants phylogeny at the biomolecular era. Given a set of close plants sharing in the order of one hundred of core chloroplastic genes, this article focuses on how to extract the largest subset of sequences in order to obtain the most supported species tree. Due to computational complexity, a discrete and distributed Particle Swarm Optimization (DPSO) is proposed. It is finally applied to the core genes of Rosales order.

• [quant-ph]Preservation of quantum Fisher information and geometric phase of a single qubit system in a dissipative reservoir through the addition of qubits
Youneng Guo, Qinglong Tian, Yunfei Mo, K Zeng
http://arxiv.org/abs/1706.08634v1

In this paper, we have investigated the preservation of quantum Fisher information of a single-qubit system coupled to a common zero temperature reservoir through the addition of noninteracting qubits. The results show that, the QFI is completely protected in both Markovian and non-Markovian regimes by increasing the number of additional qubits. Besides, the phenomena of QFI display monotonic decay or non-monotonic with revival oscillations depending on the number of additional qubits in a common dissipative reservoir. Moreover, we extend this model to investigate the effect of additional qubits N-1 and the initial conditions of the system on the geometric phase. It is found that, the robustness of GP against the dissipative reservoir has been demonstrated by increasing gradually the number of additional qubits. Besides, the GP is sensitive to the initial parameter theta, and possesses symmetric in a range regime 0,2 pi.

• [stat.AP]Robust and Efficient Parametric Spectral Estimation in Atomic Force Microscopy
Bryan Yates, Aleksander Labuda, Martin Lysy
http://arxiv.org/abs/1706.08938v1

An atomic force microscope (AFM) is capable of producing ultra-high resolution measurements of nanoscopic objects and forces. It is an indispensable tool for various scientific disciplines such as molecular engineering, solid-state physics, and cell biology. Prior to a given experiment, the AFM must be calibrated by fitting a spectral density model to baseline recordings. However, since AFM experiments typically collect large amounts of data, parameter estimation by maximum likelihood can be prohibitively expensive. Thus, practitioners routinely employ a much faster least-squares estimation method, at the cost of substantially reduced statistical efficiency. Additionally, AFM data is often contaminated by periodic electronic noise, to which parameter estimates are highly sensitive. This article proposes a two-stage estimator to address these issues. Preliminary parameter estimates are first obtained by a variance-stabilizing procedure, by which the simplicity of least-squares combines with the efficiency of maximum likelihood. A test for spectral periodicities then eliminates high-impact outliers, considerably and robustly protecting the second-stage estimator from the effects of electronic noise. Simulation and experimental results indicate that a two- to ten-fold reduction in mean squared error can be expected by applying our methodology.

• [stat.CO]archivist: An R Package for Managing, Recording and Restoring Data Analysis Results
Przemyslaw Biecek, Marcin Kosinski
http://arxiv.org/abs/1706.08822v1

Everything that exists in R is an object [Chambers2016]. This article examines what would be possible if we kept copies of all R objects that have ever been created. Not only objects but also their properties, meta-data, relations with other objects and information about context in which they were created. We introduce archivist, an R package designed to improve the management of results of data analysis. Key functionalities of this package include: (i) management of local and remote repositories which contain R objects and their meta-data (objects' properties and relations between them); (ii) archiving R objects to repositories; (iii) sharing and retrieving objects (and it's pedigree) by their unique hooks; (iv) searching for objects with specific properties or relations to other objects; (v) verification of object's identity and context of it's creation. The presented archivist package extends, in a combination with packages such as knitr and Sweave, the reproducible research paradigm by creating new ways to retrieve and validate previously calculated objects. These new features give a variety of opportunities such as: sharing R objects within reports or articles; adding hooks to R objects in table or figure captions; interactive exploration of object repositories; caching function calls with their results; retrieving object's pedigree (information about how the object was created); automated tracking of the performance of considered models, restoring R libraries to the state in which object was archived.

• [stat.ME]Evaluating the hot hand phenomenon using predictive memory selection for multistep Markov Chains: LeBron James' error correcting free throws
Joshua C. Chang
http://arxiv.org/abs/1706.08881v1

Consider the problem of modeling memory for discrete-state random walks using higher-order Markov chains. This Letter introduces a general Bayesian framework under the principle of minimizing prediction error to select, from data, the number of prior states of recent history upon which a trajectory is statistically dependent. In this framework, I provide closed-form expressions for several alternative model selection criteria that approximate model prediction error for future data. Using simulations, I evaluate the statistical power of these criteria. These methods, when applied to data from the 2016--2017 NBA season, demonstrate evidence of statistical dependencies in LeBron James' free throw shooting. In particular, a model depending on the previous shot (single-step Markovian) is approximately as predictive as a model with independent outcomes. A hybrid jagged model of two parameters, where James shoots a higher percentage after a missed free throw than otherwise, is more predictive than either model.

• [stat.ME]Extrinsic Gaussian processes for regression and classification on manifolds
Lizhen Lin, Mu Niu, Pokman Cheung, David Dunson
http://arxiv.org/abs/1706.08757v1

Gaussian processes (GPs) are very widely used for modeling of unknown functions or surfaces in applications ranging from regression to classification to spatial processes. Although there is an increasingly vast literature on applications, methods, theory and algorithms related to GPs, the overwhelming majority of this literature focuses on the case in which the input domain corresponds to a Euclidean space. However, particularly in recent years with the increasing collection of complex data, it is commonly the case that the input domain does not have such a simple form. For example, it is common for the inputs to be restricted to a non-Euclidean manifold, a case which forms the motivation for this article. In particular, we propose a general extrinsic framework for GP modeling on manifolds, which relies on embedding of the manifold into a Euclidean space and then constructing extrinsic kernels for GPs on their images. These extrinsic Gaussian processes (eGPs) are used as prior distributions for unknown functions in Bayesian inferences. Our approach is simple and general, and we show that the eGPs inherit fine theoretical properties from GP models in Euclidean spaces. We consider applications of our models to regression and classification problems with predictors lying in a large class of manifolds, including spheres, planar shape spaces, a space of positive definite matrices, and Grassmannians. Our models can be readily used by practitioners in biological sciences for various regression and classification problems, such as disease diagnosis or detection. Our work is also likely to have impact in spatial statistics when spatial locations are on the sphere or other geometric spaces.

• [stat.ME]Invariant Causal Prediction for Nonlinear Models
Christina Heinze-Deml, Jonas Peters, Nicolai Meinshausen
http://arxiv.org/abs/1706.08576v1

An important problem in many domains is to predict how a system will respond to interventions. This task is inherently linked to estimating the system's underlying causal structure. To this end, 'invariant causal prediction' (ICP) (Peters et al., 2016) has been proposed which learns a causal model exploiting the invariance of causal relations using data from different environments. When considering linear models, the implementation of ICP is relatively straight-forward. However, the nonlinear case is more challenging due to the difficulty of performing nonparametric tests for conditional independence. In this work, we present and evaluate an array of methods for nonlinear and nonparametric versions of ICP for learning the causal parents of given target variables. We find that an approach which first fits a nonlinear model with data pooled over all environments and then tests for differences between the residual distributions across environments is quite robust across a large variety of simulation settings. We call this procedure "Invariant residual distribution test". In general, we observe that the performance of all approaches is critically dependent on the true (unknown) causal structure and it becomes challenging to achieve high power if the parental set includes more than two variables. As a real-world example, we consider fertility rate modelling which is central to world population projections. We explore predicting the effect of hypothetical interventions using the accepted models from nonlinear ICP. The results reaffirm the previously observed central causal role of child mortality rates.

• [stat.ME]Subspace Clustering with the Multivariate-t Distribution
Angelina Pesevski, Brian C. Franczak, Paul D. McNicholas
http://arxiv.org/abs/1706.08927v1

Clustering procedures suitable for the analysis of very high-dimensional data are needed for many modern data sets. In model-based clustering, a method called high-dimensional data clustering (HDDC) uses a family of Gaussian mixture models for clustering. HDDC is based on the idea that high-dimensional data usually exists in lower-dimensional subspaces; as such, an intrinsic dimension for each sub-population of the observed data can be estimated and cluster analysis can be performed in this lower-dimensional subspace. As a result, only a fraction of the total number of parameters need to be estimated and a computationally efficient parameter estimation scheme based on the EM algorithm was developed. This family of models has gained attention due to its superior classification performance compared to other families of mixture models; however, it still suffers from the usual limitations of Gaussian mixture model-based approaches. In this paper, a robust analogue of the HDDC approach is proposed. This approach, which extends the HDDC procedure to include the mulitvariate-t distribution, encompasses 28 models that rectify the aforementioned shortcomings of the HDDC procedure. Our tHDDC procedure is fitted to both simulated and real data sets and is compared to the HDDC procedure using an image reconstruction problem that arose from satellite imagery of Mars' surface.

• [stat.ML]Cognitive Psychology for Deep Neural Networks: A Shape Bias Case Study
Samuel Ritter, David G. T. Barrett, Adam Santoro, Matt M. Botvinick
http://arxiv.org/abs/1706.08606v1

Deep neural networks (DNNs) have advanced performance on a wide range of complex tasks, rapidly outpacing our understanding of the nature of their solutions. While past work sought to advance our understanding of these models, none has made use of the rich history of problem descriptions, theories, and experimental methods developed by cognitive psychologists to study the human mind. To explore the potential value of these tools, we chose a well-established analysis from developmental psychology that explains how children learn word labels for objects, and applied that analysis to DNNs. Using datasets of stimuli inspired by the original cognitive psychology experiments, we find that state-of-the-art one shot learning models trained on ImageNet exhibit a similar bias to that observed in humans: they prefer to categorize objects according to shape rather than color. The magnitude of this shape bias varies greatly among architecturally identical, but differently seeded models, and even fluctuates within seeds throughout training, despite nearly equivalent classification performance. These results demonstrate the capability of tools from cognitive psychology for exposing hidden computational properties of DNNs, while concurrently providing us with a computational model for human word learning.

• [stat.ML]Fast Algorithms for Learning Latent Variables in Graphical Models
Mohammadreza Soltani, Chinmay Hegde
http://arxiv.org/abs/1706.08936v1

We study the problem of learning latent variables in Gaussian graphical models. Existing methods for this problem assume that the precision matrix of the observed variables is the superposition of a sparse and a low-rank component. In this paper, we focus on the estimation of the low-rank component, which encodes the effect of marginalization over the latent variables. We introduce fast, proper learning algorithms for this problem. In contrast with existing approaches, our algorithms are manifestly non-convex. We support their efficacy via a rigorous theoretical analysis, and show that our algorithms match the best possible in terms of sample complexity, while achieving computational speed-ups over existing methods. We complement our theory with several numerical experiments.

• [stat.ML]Faster ICA by preconditioning with Hessian approximations
Pierre Ablin, Jean-François Cardoso, Alexandre Gramfort
http://arxiv.org/abs/1706.08171v2

Independent Component Analysis (ICA) is a powerful technique for unsupervised data exploration that is widely used across fields such as neuroscience, astronomy, chemistry or biology. Linear ICA is a linear latent factor model, similar to sparse dictionary learning, that aims at discovering statistically independent sources from multivariate observations. ICA is a probabilistic generative model for which inference is classically done by maximum likelihood estimation. Estimating sources by maximum likelihood leads to a smooth non-convex optimization problem where the unknown is a matrix called the separating or unmixing matrix. As the gradient of the likelihood is available in closed form, first order gradient methods, stochastic or non-stochastic, are often employed despite a slow convergence such as in the Infomax algorithm. While the Hessian is known analytically, the cost of its computation and inversion makes Newton method unpractical for a large number of sources. We show how sparse and positive approximations of the true Hessian can be obtained and used to precondition the L-BFGS algorithm. Results on simulations and two applied problems (EEG data and image patches) demonstrate that the proposed technique leads to convergence that can be orders of magnitude faster than algorithms commonly used today even when looking for hundred of sources.

• [stat.ML]MolecuLeNet: A continuous-filter convolutional neural network for modeling quantum interactions
Kristof T. Schütt, Pieter-Jan Kindermans, Huziel E. Sauceda, Stefan Chmiela, Alexandre Tkatchenko, Klaus-Robert Müller
http://arxiv.org/abs/1706.08566v1

Deep learning has the potential to revolutionize quantum chemistry as it is ideally suited to learn representations for structured data and speed up the exploration of chemical space. While convolutional neural networks have proven to be the first choice for images, audio and video data, the atoms in molecules are not restricted to a grid. Instead, their precise locations contain essential physical information, that would get lost if discretized. Thus, we propose to use continuous-filter convolutional layers to be able to model local correlations without requiring the data to lie on a grid. We apply those layers in MolecuLeNet: a novel deep learning architecture modeling quantum interactions in molecules. We obtain a joint model for the total energy and interatomic forces that follows fundamental quantum-chemical principles. This includes rotationally invariant energy predictions and a smooth, differentiable potential energy surface. Our architecture achieves state-of-the-art performance for benchmarks of equilibrium molecules and molecular dynamics trajectories. Finally, we introduce a more challenging benchmark with chemical and structural variations that suggests the path for further work.

• [stat.ML]On conditional parity as a notion of non-discrimination in machine learning
Ya'acov Ritov, Yuekai Sun, Ruofei Zhao
http://arxiv.org/abs/1706.08519v1

We identify conditional parity as a general notion of non-discrimination in machine learning. In fact, several recently proposed notions of non-discrimination, including a few counterfactual notions, are instances of conditional parity. We show that conditional parity is amenable to statistical analysis by studying randomization as a general mechanism for achieving conditional parity and a kernel-based test of conditional parity.

• [stat.ML]Two-Stage Hybrid Day-Ahead Solar Forecasting
Mohana Alanazi, Mohsen Mahoor, Amin Khodaei
http://arxiv.org/abs/1706.08699v1

Power supply from renewable resources is on a global rise where it is forecasted that renewable generation will surpass other types of generation in a foreseeable future. Increased generation from renewable resources, mainly solar and wind, exposes the power grid to more vulnerabilities, conceivably due to their variable generation, thus highlighting the importance of accurate forecasting methods. This paper proposes a two-stage day-ahead solar forecasting method that breaks down the forecasting into linear and nonlinear parts, determines subsequent forecasts, and accordingly, improves accuracy of the obtained results. To further reduce the error resulted from nonstationarity of the historical solar radiation data, a data processing approach, including pre-process and post-process levels, is integrated with the proposed method. Numerical simulations on three test days with different weather conditions exhibit the effectiveness of the proposed two-stage model.

• [stat.ML]Unsupervised Feature Selection Based on Space Filling Concept
Mohamed Laib, Mikhail Kanevski
http://arxiv.org/abs/1706.08894v1

The paper deals with the adaptation of a new measure for the unsupervised feature selection problems. The proposed measure is based on space filling concept and is called the coverage measure. This measure was used for judging the quality of an experimental space filling design. In the present work, the coverage measure is adapted for selecting the smallest informative subset of variables by reducing redundancy in data. This paper proposes a simple analogy to apply this measure. It is implemented in a filter algorithm for unsupervised feature selection problems. The proposed filter algorithm is robust with high dimensional data and can be implemented without extra parameters. Further, it is tested with simulated data and real world case studies including environmental data and hyperspectral image. Finally, the results are evaluated by using random forest algorithm.

• [stat.ML]When Neurons Fail
El Mahdi El Mhamdi, Rachid Guerraoui
http://arxiv.org/abs/1706.08884v1

We view a neural network as a distributed system of which neurons can fail independently, and we evaluate its robustness in the absence of any (recovery) learning phase. We give tight bounds on the number of neurons that can fail without harming the result of a computation. To determine our bounds, we leverage the fact that neural activation functions are Lipschitz-continuous. Our bound is on a quantity, we call the \textit{Forward Error Propagation}, capturing how much error is propagated by a neural network when a given number of components is failing, computing this quantity only requires looking at the topology of the network, while experimentally assessing the robustness of a network requires the costly experiment of looking at all the possible inputs and testing all the possible configurations of the network corresponding to different failure situations, facing a discouraging combinatorial explosion. We distinguish the case of neurons that can fail and stop their activity (crashed neurons) from the case of neurons that can fail by transmitting arbitrary values (Byzantine neurons). Interestingly, as we show in the paper, our bound can easily be extended to the case where synapses can fail. We show how our bound can be leveraged to quantify the effect of memory cost reduction on the accuracy of a neural network, to estimate the amount of information any neuron needs from its preceding layer, enabling thereby a boosting scheme that prevents neurons from waiting for unnecessary signals. We finally discuss the trade-off between neural networks robustness and learning cost.