This paper presents the scientific outcomes of the 2018 Data Fusion Contest organized by the Image Analysis and Data Fusion Technical Committee of the IEEE Geoscience and Remote Sensing Society. The 2018 Contest addressed the problem of urban observation and monitoring with advanced multi-source optical remote sensing (multispectral LiDAR, hyperspectral imaging, and very high-resolution imagery). The competition was based on urban land use and land cover classification, aiming to distinguish between very diverse and detailed classes of urban objects, materials, and vegetation. Besides data fusion, it also quantified the respective assets of the novel sensors used to collect the data. Participants proposed elaborate approaches rooted in remote-sensing, and also in machine learning and computer vision, to make the most of the available data. Winning approaches combine convolutional neural networks with subtle earth-observation data scientist expertise.

Automatic and accurate classification of apoptosis, or programmed cell death, will facilitate cell biology research. The state-of-the-art approaches in apoptosis classification use deep convolutional neural networks (CNNs). However, these networks are not efficient in encoding the part-whole relationships, thus requiring a large number of training samples to achieve robust generalization. This paper proposes an efficient variant of capsule networks (CapsNets) as an alternative to CNNs. Extensive experimental results demonstrate that the proposed CapsNets achieve competitive performances in target cell apoptosis classification, while significantly outperforming CNNs when the number of training samples is small. To utilize temporal information within microscopy videos, we propose a recurrent CapsNet constructed by stacking a CapsNet and a bi-directional long short-term recurrent structure. Our experiments show that when considering temporal constraints, the recurrent CapsNet achieves 93.8% accuracy and makes significantly more consistent prediction than NNs.

Understanding neural activity patterns in the developing brain remains one of the grand challenges in neuroscience. Developing neural networks are likely to be endowed with functionally important variability associated with the environmental context, age, gender, and other variables. Therefore, we conducted experiments with typically developing children in a stimulating museum setting and tested the feasibility of using deep learning techniques to help identify patterns of brain activity associated with different conditions. Approach. A four-channel dry EEG-based Mobile brain-body imaging data of children at rest and during videogame play (VGP) was acquired at the Children's Museum of Houston. A data-driven approach based on convolutional neural networks (CNN) was used to describe underlying feature representations in the EEG and their ability to discern task and gender. The variability of the spectral features of EEG during the rest condition as a function of age was also analyzed. Main results. Alpha power (7–13 Hz) was higher during rest whereas theta power (4–7 Hz) was higher during VGP. Beta (13–18 Hz) power was the most significant feature, higher in females, when differentiating between males and females. Using data from both temporoparietal channels to classify between VGP and rest condition, leave-one-subject-out cross-validation accuracy of 67% was obtained. Age-related changes in EEG spectral content during rest were consistent with previous developmental studies conducted in laboratory settings showing an inverse relationship between age and EEG power. Significance. These findings are the first to acquire, quantify and explain brain patterns observed during VGP and rest in freely behaving children in a museum setting using a deep learning framework. The study shows how deep learning can be used as a data driven approach to identify patterns in the data and explores the issues and the potential of conducting experiments involving children in a natural and engaging environment.

This interdisciplinary book is written for government and industry professionals who need a comprehensive, accessible guide to modern energy security. Introducing the ten predominant energy types, both renewable and non-renewable, the book illustrates the modern energy landscape from a geopolitical, commercial, economic and technological perspective. Energy is presented as the powerhouse of global economic activities.

With the revolutionary innovations emerging in the computer graphics domain, virtual reality (VR) has become increasingly popular and shown great potential for entertainment, medical simulation and education. In the highly interactive VR world, the motion-to-photon delay (MPD) which represents the delay from users' head motion to the responded image displayed on their head devices, is the most critical factor for a successful VR experience. Long MPD may cause users to experience significant motion anomalies: judder, lagging and sickness. In order to achieve the short MPD and alleviate the motion anomalies, asynchronous time warp (ATW) which is known as an image re-projection technique, has been proposed by VR vendors to map the previously rendered frame to the correct position using the latest headmotion information. However, after a careful investigation on the efficiency of the current GPU-accelerated ATW through executing real VR applications on modern VR hardware, we observe that the state-of-the-art ATW technique cannot deliver the ideal MPD and often misses the refresh deadline, resulting in reduced frame rate and motion anomalies. This is caused by two major challenges: inefficient VR execution model and intensive off-chip memory accesses. To tackle these, we propose a preemption-free Processing-In-Memory based ATW design which asynchronously executes ATW within a 3D-stacked memory, without interrupting the rendering tasks on the host GPU. We also identify a redundancy reduction mechanism to further simplify and accelerate the ATW operation. A comprehensive evaluation of our proposed design demonstrates that our PIM-based ATW can achieve the ideal MPD and provide superior user experience. Finally, we provide a design space exploration to showcase different design choices for the PIM-based ATW design, and the results show that our design scales well in future VR scenarios with higher frame resolution and even lower ideal MPD.

The glaciers flowing into the Amundsen Sea Embayment, West Antarctica, have undergone acceleration and grounding line retreat over the past few decades that may yield an irreversible mass loss. Using a constellation of satellites, we detect the evolution of ice velocity, ice thinning, and grounding line retreat of Thwaites Glacier from 1992 to 2017. The results reveal a complex pattern of retreat and ice melt, with sectors retreating at 0.8 km/year and floating ice melting at 200 m/year, while others retreat at 0.3 km/year with ice melting 10 times slower. We interpret the results in terms of buoyancy/slope-driven seawater intrusion along preferential channels at tidal frequencies leading to more efficient melt in newly formed cavities. Such complexities in ice-ocean interaction are not currently represented in coupled ice sheet/ocean models.

The increasing demand for underground infrastructure should be supported by innovation in monitoring and damage assessment solutions to minimise damage to surface structures caused by ground settlements. This paper evaluates the use of multitemporal synthetic aperture radar interferometry (MT-InSAR) to calculate tunnelling-induced deformations of buildings. The paper introduces a step-by-step procedure to use InSAR displacements as an input to the structural damage assessment. After a comparison between traditional and InSAR monitoring data for the London area during the Crossrail excavation, the high resolution, high density InSAR-based displacements were used to evaluate the building deformations for a number of case studies. Results demonstrate the quality of information provided by InSAR data on soil-structure interaction mechanisms. Such information, essential to evaluate current damage assessment procedures, is typically only collected for relatively few buildings due to the cost of traditional monitoring. A comparison between damage indicators derived from greenfield assumptions and building displacements quantifies the practical benefit of the proposed step-by-step procedure. This work aims at filling the gap between the most recent advances in remote sensing and the civil engineering practice, defining the first step of an automated damage assessment procedure which can impact large scale underground projects in urban areas.

Despite significant advances in the analysis and visualization of unsteady flow, the interpretation of it's behavior still remains a challenge. In this work, we focus on the linear correlation and non-linear dependency of different physical attributes of unsteady flows to aid their study from a new perspective. Specifically, we extend the existing spatial correlation quantification, i.e. the Local Correlation Coefficient (LCC), to the spatio-temporal domain to study the correlation of attribute-pairs from both the Eulerian and Lagrangian views. To study the dependency among attributes, which need not be linear, we extend and compute the mutual information (MI) among attributes over time. To help visualize and interpret the derived correlation and dependency among attributes associated with a particle, we encode the correlation and dependency values on individual pathlines. Finally, to utilize the correlation and MI computation results to identify regions with interesting flow behavior, we propose a segmentation strategy of the flow domain based on the ranking of the strength of the attributes relations. We have applied our correlation and dependency metrics to a number of 2D and 3D unsteady flows with varying spatio-temporal kernel sizes to demonstrate and assess their effectiveness.

Automated profiling of cell–cell interactions from high-throughput time-lapse imaging microscopy data of cells in nanowell grids (TIMING) has led to fundamental insights into cell–cell interactions in immunotherapy. This application note aims to enable widespread adoption of TIMING by (i) enabling the computations to occur on a desktop computer with a graphical processing unit instead of a server; (ii) enabling image acquisition and analysis to occur in the laboratory avoiding network data transfers to/from a server and (iii) providing a comprehensive graphical user interface. Results On a desktop computer, TIMING 2.0 takes 5 s/block/image frame, four times faster than our previous method on the same computer, and twice as fast as our previous method (TIMING) running on a Dell PowerEdge server. The cell segmentation accuracy (f-number = 0.993) is superior to our previous method (f-number = 0.821). A graphical user interface provides the ability to inspect the video analysis results, make corrective edits efficiently (one-click editing of an entire nanowell video sequence in 5–10 s) and display a summary of the cell killing efficacy measurements.

Lung cancer is the leading cause of cancer-related deaths in the past several years. A major challenge in lung cancer screening is the detection of lung nodules from computed tomography (CT) scans. State-of-the-art approaches in automated lung nodule classification use deep convolutional neural networks (CNNs). However, these networks require a large number of training samples to generalize well. This paper investigates the use of capsule networks (CapsNets) as an alternative to CNNs. We show that CapsNets significantly outperforms CNNs when the number of training samples is small. To increase the computational efficiency, our paper proposes a consistent dynamic routing mechanism that results in 3× speedup of CapsNet. Finally, we show that the original image reconstruction method of CapNets performs poorly on lung nodule data. We propose an efficient alternative, called convolutional decoder, that yields lower reconstruction error and higher classification accuracy.