Computing Platforms Seminar Series (COMPASS)

The Computing Platforms Seminar Series (COMPASS) is focused on talks by industry and academia around the general topic of computing platforms.

COMPASS is held on most Thursdays during the semester 10:00-11:00 (with some exceptions) in CAB E 72.

Upcoming Talks:


Thursday, 25. April 2019, 10:00-11:00 in CAB E 72

Speaker: Peter Pietzuch (Imperial College London)

Title: Scaling Deep Learning on Multi-GPU Servers

 

 

 

Abstract

With the widespread availability of GPU servers, scalability in terms of the number of GPUs when training deep learning models becomes a paramount concern. For many deep learning models, there is a scalability challenge: to keep multiple GPUs fully utilised, the batch size must be sufficiently large, but a large batch size slows down model convergence due to the less frequent model updates.

In this talk, I describe CrossBow, a new single-server multi-GPU deep learning system that avoids the above trade-off. CrossBow trains multiple model replicas concurrently on each GPU, thereby avoiding under-utilisation of GPUs even when the preferred batch size is small. For this, CrossBow (i) decides on an appropriate number of model replicas per GPU and (ii) employs an efficient and scalable synchronisation scheme within and across GPUs.

Short Bio:

Peter Pietzuch is a Professor at Imperial College London, where he leads the Large-scale Data & Systems (LSDS) group (http://lsds.doc.ic.ac.uk) in the Department of Computing. His research focuses on the design and engineering of scalable, reliable and secure large-scale software systems, with a particular interest in performance, data management and security issues. He has published papers in premier international venues, including SIGMOD, VLDB, OSDI, USENIX ATC, EuroSys, SoCC, ICDCS, CCS, CoNEXT, NSDI, and Middleware. Before joining Imperial College London, he was a post-doctoral fellow at Harvard University. He holds PhD and MA degrees from the University of Cambridge.


 

Past COMPASS Talks:  

Date Speaker Affiliation Talk
28.03.2019 Theo Rekatsinas
University of Wisconsin A Machine Learning Perspective on Managing Noisy Data
21.03.2019 Marko Vukolic IBM Research Hyperledger Fabric: a Distributed Operating System for Permissioned Blockchains
28.02.2019 Alberto Lerner University of Fribourg
The Case for Network-Accelerated Query Processing
21.02.2019 Thomas Würthinger Oracle Labs Bringing the Code to the Data with GraalVM
31.01.2019 Irene Zhang Microsoft Research, Redmond Demikernel: An Operating System Architecture for Hardware-Accelerated Datacenter Servers
25.10.2018 Mihnea Andrei SAP HANA Snapshot isolation in HANA - the evolution towards production-grade HTAP
04.10.2018 Philippe Bonnet IT University, Copenhagen, Denmark Near-Data Processing with Open-Channel SSDs
25.09.2018 Nandita Vijaykumar   Carnegie Mellon University Expressive Memory: Rethinking the Hardware-Software Contract with Rich Cross-Layer Abstractions
20.09.2018 Patrick Stüdi IBM Research Data processing at the speed of 100 Gbps using Apache Crail (Incubating)
15.08.2018 Leonid Yavits
Technion Resistive CAM based architectures: Resistive Associative In-Storage Processor and Resistive Address Decoder
06.07.2018 Martin Burtscher Texas State University Automatic Hierarchical Parallelization of Linear Recurrences
15.06.2018 Nitin Agrawal Samsung Research Low-Latency Analytics on Colossal Data Streams with SummaryStore
24.05.2018 Cagri Balkesen Oracle Labs RAPID: In-Memory Analytical Query Processing Engine with Extreme Performance per Watt
16.05.2018 Carsten Binnig TU Darmstadt Towards Interactive Data Exploration
09.05.2018 Bastian Hossbach Oracle Labs Modern programming languages and code generation in the Oracle Database
26.04.2018 Spyros Blanas Ohio State University Scaling database systems to high-performance computers
19.04.2018 Jane Hung MIT The Challenges and Promises of Large-Scale Biological Imaging
12.04.2018 Christoph Hagleitner IBM Research Heterogeneous Computing Systems for Datacenter and HPC Applications
14.03.2018  Eric Sedlar
 Oracle Labs
Why Systems Research Needs Social Science Added to the Computer Science
01.03.2018 Saughata Ghose Carnegie Mellon University How Safe Is Your Storage? A Look at the Reliability and Vulnerability of Modern Solid-State Drives
22.02.2018  Ioannis Koltsidas IBM Research Zurich System software for commodity solid-state storage