Tutorials

Anima Anandkumar

California Institute of Technology

Furong Huang

University of Maryland College Park

The Role of Tensors in Deep Learning SLIDES

Abstract
Tensors are higher order extensions of matrices that can incorporate multiple modalities and encode higher order relationships in data. Tensors play a significant role in machine learning through (1) tensor contractions, (2) tensor sketches, and (3) tensor decompositions. Tensor contractions are extensions of matrix products to higher dimensions. Tensor sketches efficiently compress tensors while preserving information. Tensor decompositions compute low rank components that constitute a tensor. We show that tensor contractions are an effective replacement for fully connected layers in deep learning architectures. They result in significant space savings with negligible performance degradation. Tensor contractions present rich opportunities for hardware optimizations through extended BLAS kernels. I will end with many open challenges in the area.

Biography (Anima Anandkumar)
Anima Anandkumar's research interests are in the areas of large-scale machine learning, non-convex optimization and high-dimensional statistics. In particular, she has been spearheading the development and analysis of tensor algorithms for machine learning. Tensors are multi-dimensional extensions of matrices and can encode higher order relationships in data. At Amazon Web Services, she is researching the practical aspects of deploying machine learning at scale on the cloud infrastructure.

Biography (Furong Huang)
Furong Huang's research focuses on machine learning, high-dimensional statistics and distributed algorithms --- both the theoretical analysis and practical implementation of parallel spectral methods for latent variable graphical models. Some applications of her research include developing fast detection algorithms to discover hidden and overlapping user communities in social networks, learning convolutional sparse coding models for understanding semantic meanings of sentences and object recognition in images, healthcare analytics by learning a hierarchy on human diseases for guiding doctors to identify potential diseases afflicting patients, and more.

Aman Shaikh

AT&T Labs Research

Vijay Gopalakrishnan

AT&T Labs Research

Techniques for Monitoring and Measuring
Virtualized Networks

Joint work with: Minlan Yu (Harvard University)

Abstract
Network virtualization promises to revolutionize how networks are built and operated. While there has been a lot of focus on the flexibility and cost reduction that software-based virtualized networks promise, the move to software opens new doors for network measurement and monitoring of networks and network functions. In this tutorial, we discuss the opportunities, challenges, and advances in monitoring and measuring virtualized networks and network functions. We aim to demonstrate that using these advances, one can not only overcome the challenges that come with an unproven and maturing technology, but also have monitoring and measurement capabilities that have so far eluded large physical appliance-based networks, in general.

Biography (Aman Shaikh)
Aman Shaikh is a principal inventive scientist at AT&T Labs Research. He obtained his Ph.D. and M.S. in Computer Engineering from the University of California, Santa Cruz in 2003 and 2000, respectively. He also holds a B.E. (HONS) in Computer Science and an M.Sc. (HONS) in Mathematics from the Birla Institute of Technology and Science, Pilani, India. His current research interests include service quality management, SDN, and NFV. Several tools that have emerged from his research are being used extensively by AT&T operations teams.

Biography (Vijay Gopalakrishnan)
Vijay Gopalakrishnan is a Director in the Network and Service Quality Management Center in AT&T Labs-Research. His research interests fall broadly in the areas of networked systems and protocols. He leads a team of researchers focused on various systems aspects of networking including network management, network function virtualization, software defined networking, network protocols, and mechanisms for efficient content delivery. Before joining AT&T in 2006, Vijay got his MS and PhD in Computer Science from the University of Maryland, College Park in 2003 and 2006 respectively. He is a member of the ACM and the IEEE.

Ivo Adan

Eindhoven University of Technology

Johan van Leeuwaarden

Eindhoven University of Technology

Structured Markov Chains SLIDES

Abstract
Markov chains are popular stochastic models, because of their intriguing mathematical properties and flexible structure, but more importantly, they provide a powerful instrument for modeling, analyzing and understanding a large variety of systems and networks, including manufacturing systems, communication networks, traffic networks and service systems. This tutorial provides an introduction to Markov chain modeling and analysis, with an emphasis on analytical methods to determine the steady-state behavior of Markov chains. We will classify Markov chains based on their structural properties. As it appears, these structural properties determine the analytical methods required for solving them. Various analytical methods will be discussed, including generating functions, spectral expansion, matrix-geometric and matrix-analytic methods. Markov chain modeling and analysis will be demonstrated through illustrative specific problems.

Biography (Ivo Adan)
Ivo Adan is a Full Professor in the sections Operations, Planning, Accounting and Control (department of Industrial Engineering & Innovation Sciences) and Dynamics and Control (department of Mechanical Engineering) at Eindhoven University of Technology (TU/e) and holds the Manufacturing Networks chair. His expertise and tuition areas include probability theory / statistics, operations research, manufacturing networks, stochastic operations research and queueing models. His current research interests are in the area of modeling, design and control of manufacturing systems, warehousing systems and transportation systems, and more specifically, in the mathematical analysis of multi-dimensional structured Markov processes and queueing models, providing an indispensable tool in the above mentioned area.

Biography (Johan van Leeuwaarden)
Johan van Leeuwaarden is professor of mathematics at Eindhoven University of Technology, focussing on stochastic operations research and applied probability. Markov processes are a central theme in this research (exact solutions, scaling limits and asymptotic behavior), with applications to random graphs, interacting particles and queueing networks. Key current interests include clustering in scale-free graphs, load balancing in cloud systems with many servers and ultra-dense wireless systems.

Li Zhang

IBM Research

Performance Modeling and Analysis of
Deep Learning Systems

Abstract
The tutorial will introduce the basic concepts of convolution neural networks, the training process, and various systems that support the distributed training using multiple GPUs on multiple hosts. It will cover the workload characterization and modeling for identifying performance bottlenecks. The performance models are also used for capacity planning, performance optimization, as well as system design.

Biography
Dr. Li Zhang is the manager of the System Analysis and Optimization group at IBM T.J. Watson Research Center. His research interests include design and optimization of high performance big data systems; performance analysis, control, scheduling, and resource allocation in parallel and distributed systems; traffic modeling and prediction for large scale computer systems. He has also been working on measurement based clock synchronization algorithms. He has co-authored over 100 technical articles and over 50 patents. A math major at Beijing University, he received his M.S. in Mathematics from Purdue University and his Ph.D. in Operations Research from Columbia University.