Gennady Pekhimenko


Associate Professor

Computer Science Department

Electrical and Computer Engineering

University of Toronto

Department of Computer and Mathematical Sciences

University of Toronto Scarborough


CEO and Co-Founder

CentML


Faculty Member, CIFAR AI Chair

Vector Institute

Contact

pekhimenko@cs.toronto.edu

Bahen 5232
    40 St. George Street
    Toronto ON, M5S 2E4

Resume/CV

Links:


Gennady Pekhimenko

About Me


In summer 2017, I joined University of Toronto, CS Department as an Assistant Professor where I lead the EcoSystem research group. I'am also a Faculty Member at Vector Institute. My work is funded by Amazon, Facebook, Huawei, Xilinx, NVIDIA, IBM, NSERC, and CIFAR. From July 2016 I was a Researcher in Systems Research Group at Microsoft Research in Redmond.

I got my PhD from Computer Science Department at Carnegie Mellon University, working with Professor Todd Mowry and Professor Onur Mutlu. My work was funded by NVIDIA Graduate, Microsoft Research, Qualcomm Innovation, and NSERC CGS-D Fellowships.

Research

I am generally interested in the areas of computer architecture, systems, and applied machine learning. My major research focus is on efficient memory systems, systems for machine learning, stream processing, machine learning for systems, compilers, and hardware acceleration.

Current Students and PostDocs

  • Christina Giannoula (PostDoc)

  • Anand Jayarajan (PhD)
  • Shang (Sam) Wang (PhD)
  • Jiacheng Yang (PhD)
  • Qidong Su (PhD)
  • Yu Bo Gao (PhD)
  • Renbo Tu (PhD)
  • Pavel Golikov (PhD)
  • Yaoyao Ding (PhD)
  • Daniel Snider (PhD)
  • Kevin Song (PhD)
  • Peiming Yang (PhD)
  • Baorun Mu (PhD)
  • Chenhao Jiang (MASc)
  • Zhanda Zhu (MASc)
  • Xiao Zhang (MASc)

  • Wei Zhao (BSc)
  • Yudi Sun (BASc)
  • Qiwen Hua (BSc)

Teaching

Recent News

  • November 14, 2022: Two papers accepted at ASPLOS 2023.

  • September 14, 2022: One paper accepted at NeurIPS 2022.

  • July 31, 2022: Two papers accepted at PACT 2022.

  • June 14, 2022: I received the VMware Early Career Faculty Grant. Thank you VMware!

  • April 1, 2022: I received the Google Research Scholar award. Thank you Google!

  • March 19, 2022: One paper accepted at OSDI 2022.

  • January 14, 2022: One paper accepted at MLSys 2022.

  • November 6, 2021: One paper accepted at CGO 2022.

  • September 29, 2021: Two papers accepted at NeurIPS 2021.

  • July 14, 2021: Paper accepted at MICRO 2021.

  • June 14, 2021: Inducted into the ISCA Hall of Fame.

  • April 26, 2021: Paper accepted at USENIX ATC 2021.

  • March 4, 2021: One paper accepted at ISCA 2021.

  • January 18, 2021: Four (!) papers accepted at MLSys 2021.

  • January 9, 2021: Our MLPerf Inference ISCA 2020 paper received the MICRO Top Picks Award.

  • November 18, 2020: Paper accepted at ASPLOS 2021.

  • October 6, 2020: Giving a talk at Facebook Faculty Summit.

  • August 28, 2020: Serving as a Program Committee member at OSDI 2021.

  • July 27, 2020: Serving as a Program Committee member at MLSys 2021.

  • July 14, 2020: I received the AWS Machine Learning Research Award. Thank you Amazon!

  • July 9, 2020: I received the Facebook Faculty Research Award (AI Systems HW SW Co-Design). Thank you Facebook!

  • July 7, 2020: Paper accepted at MICRO 2020.

  • July 6, 2020: Facebook Research Gift. Thank you Facebook!

  • June 24, 2020: Paper accepted at UIST 2020.

Publications (outdated)

Skyline: Interactive In-Editor Computational Performance Profiling for Deep Neural Network Training
Geoffrey Yu, Tovi Grossman, Gennady Pekhimenko
UIST, October 2020

TensorDash: Exploiting Sparsity to Accelerate Deep Neural Network Training
Mostafa Mahmoud, Isak Edo Vivancos, Ali Hadi Zadeh, Omar Mohamed Awad, Gennady Pekhimenko, Jorge Albericio, Andreas Moshovos
MICRO, October 2020

Daydream: Accurately Estimating the Efficacy of Optimizations for DNN Training
Hongyu Zhu, Amar Phanishayee, Gennady Pekhimenko
USENIX ATC, July 2020

Echo: Compiler-based GPU Memory Footprint Reduction for LSTM RNN Training
Bojian Zheng, Nandita Vijaykumar, Gennady Pekhimenko
ISCA, June 2020

MLPerf Inference Benchmark
Vijay Janapa Reddi, Christine Cheng, David Kanter, Peter Mattson, Guenther Schmuelling, Carole-Jean Wu, Brian Anderson, Maximilien Breughe, Mark Charlebois, William Chou, Ramesh Chukka, Cody Coleman, Sam Davis, Pan Deng, Greg Diamos, Jared Duke, Dave Fick, J. Scott Gardner, Itay Hubara, Sachin Idgunji, Thomas B. Jablin, Jeff Jiao, Tom St. John, Pankaj Kanwar, David Lee, Jeffery Liao, Anton Lokhmotov, Francisco Massa, Peng Meng, Paulius Micikevicius, Colin Osborne, Gennady Pekhimenko, Arun Tejusve Raghunath Rajan, Dilip Sequeira, Ashish Sirasao, Fei Sun, Hanlin Tang, Michael Thomson, Frank Wei, Ephrem Wu, Lingjie Xu, Koichi Yamada, Bing Yu, George Yuan, Aaron Zhong, Peizhao Zhang, Yuchen Zhou
ISCA, June 2020

BPPSA: Scaling Back-propagation by Parallel Scan Algorithm
Shang Wang, Yifan Bai, Gennady Pekhimenko
MLSys, March 2020

MLPerf Training Benchmark
Peter Mattson, Christine Cheng, Cody Coleman, Greg Diamos, Paulius Micikevicius, David Patterson, Hanlin Tang, Gu-Yeon Wei, Peter Bailis, Victor Bittorf, David Brooks, Dehao Chen, Debojyoti Dutta, Udit Gupta, Kim Hazelwood, Andrew Hock, Xinyuan Huang, Bill Jia, Daniel Kang, David Kanter, Naveen Kumar, Jeffery Liao, Deepak Narayanan, Tayo Oguntebi, Gennady Pekhimenko, Lillian Pentecost, Vijay Janapa Reddi, Taylor Robie, Tom St. John, Carole-Jean Wu, Lingjie Xu, Cliff Young, Matei Zaharia
MLSys, March 2020

Janus: Optimizing Memory and Storage Support for Non-Volatile Memory Systems
Sihang Liu, Korakit Seemakhupt, Gennady Pekhimenko, Aasheesh Kolli, and Samira Khan
ISCA-46, June 2019
MICRO Top Picks Honorable Mention.

Priority-based Parameter Propagation for Distributed DNN Training
Anand Jayarajan, Jinliang Wei, Garth Gibson, Alexandra Fedorova, and Gennady Pekhimenko
SysML, April 2019

HyStream: Stream Analytics on High Bandwidth Hybrid Memory
Hongyu Miao, Myeongjae Jeon, Gennady Pekhimenko, Kathryn S. McKinley, and Felix Xiaozhu Lin
ASPLOS, April 2019

Benchmarking and Analyzing Deep Neural Network Training
Hongyu Zhu, Mohamed Akrout, Bojian Zheng, Andrew Pelegris, Amar Phanishayee, Bianca Schroeder, and Gennady Pekhimenko
IISWC, October 2018

TerseCades: Efficient Data Compression in Stream Processing
Gennady Pekhimenko, Chuanxiong Guo, Myeongjae Jeon, Ryan Huang, and Lidong Zhou
USENIX Annual Technical Conference, July 2018

Gist: Efficient Data Encoding for Deep Neural Network Training
Animesh Jain, Amar Phanishayee, Jason Mars, Lingjia Tang, and Gennady Pekhimenko
ISCA-45, June 2018

A Case for Richer Cross-layer Abstractions: Bridging the Semantic Gap to Enhance Memory Optimization
Nandita Vijaykumar, Abhilasha Jain, Diptesh Majumdar, Kevin Hsieh, Gennady Pekhimenko, Eiman Ebrahimi, Nastaran Hajinazaran, Phillip B. Gibbons, Onur Mutlu
ISCA-45, June 2018

TBD: Benchmarking and Analyzing Deep Neural Network Training
Hongyu Zhu, Mohamed Akrout, Bojian Zheng, Andrew Pelegris, Amar Phanishayee, Bianca Schroeder, and Gennady Pekhimenko
arXiv, March 2018

DNN-Train: Benchmarking and Analyzing DNN Training
Hongyu Zhu, Bojian Zheng, Amar Phanishayee, Bianca Schroeder, and Gennady Pekhimenko
SysML, February 2018

StreamBox: Modern Stream Processing on a Multicore Machine
Hongyu Miao, Heejin Park, Myeongjae Jeon, Gennady Pekhimenko, Kathryn S. McKinley, Felix Xiaozhu Lin
USENIX Annual Technical Conference, July 2017

Design-Induced Latency Variation in Modern DRAM Chips: Characterization, Analysis, and Latency Reduction Mechanisms
Donghyuk Lee, Samira Khan, Lavanya Subramanian, Saugata Ghose, Rachata Ausavarungnirun, Gennady Pekhimenko, Vivek Seshadri, Onur Mutlu
SIGMETRICS, June 2017

SoftMC: A Flexible and Practical Open-Source Infrastructure for Enabling Experimental DRAM Studies
Hasan Hassan, Nandita Vijaykumar, Samira Khan, Saugata Ghose, Kevin Chang, Gennady Pekhimenko, Donghyuk Lee, Oguz Ergin, Onur Mutlu
HPCA-23, February 2017

Zorua: A Holistic Approach to Resource Virtualization in GPUs
Nandita Vijaykumar, Kevin Hsieh, Gennady Pekhimenko, Samira Khan, Ashish Shrestha, Saugata Ghose, Adwait Jog, Phillip B. Gibbons, Onur Mutlu
MICRO-49, October 2016

Understanding Latency Variation in Modern DRAM Chips: Experimental Characterization, Analysis, and Optimization
Kevin Chang, Abhijith Kashyap, Hasan Hassan, Samira Khan, Kevin Hsieh, Donghyuk Lee, Saugata Ghose, Gennady Pekhimenko, Tianshi Li, and Onur Mutlu
SIGMETRICS, June 2016

A Case for Toggle-Aware Compression for GPU Systems
Gennady Pekhimenko, Evgeny Bolotin, Nandita Vijaykumar, Onur Mutlu, Todd C. Mowry, and Stephen W. Keckler
HPCA-22, March 2016

ChargeCache: Reducing DRAM Latency by Exploiting Row Access Locality
Hasan Hassan, Gennady Pekhimenko, Nandita Vijaykumar, Vivek Seshadri, Donghyuk Lee, Oguz Ergin, and Onur Mutlu
HPCA-22, March 2016

Optimal Seed Solver: Optimizing Seed Selection in Read Mapping
Hongyi Xin, Richard Zhu, Sunny Nahar, John Emmons, Gennady Pekhimenko, Carl Kingsford, Can Alkan, and Onur Mutlu
Oxford Bioinformatics, 2016

RFVP: Rollback-Free Value Prediction with Safe-to-Approximate Loads
Amir Yazdanbakhsh, Gennady Pekhimenko, Bradley Thwaites, Hadi Esmaeilzadeh, Onur Mutlu, and Todd C. Mowry
ACM TACO, January 2016

Simultaneous Multi Layer Access: A High Bandwidth and Low Cost 3D-Stacked Memory Interface
Donghyuk Lee, Saugata Ghose, Gennady Pekhimenko, Samira Khan, Onur Mutlu
ACM TACO, January 2016

Mitigating the Memory Bottleneck with Approximate Load Value Prediction
Amir Yazdanbakhsh, Gennady Pekhimenko, Bradley Thwaites, Hadi Esmaeilzadeh, Onur Mutlu, and Todd C. Mowry
IEEE Design and Test, January 2016

Optimal Seed Solver: Optimizing Seed Selection in Read Mapping
Hongyi Xin, Richard Zhu, Sunny Nahar, John Emmons, Gennady Pekhimenko, Carl Kingsford, Can Alkan, and Onur Mutlu
HITSEQ (Poster Session), July 2015

Toggle-Aware Compression for GPUs
Gennady Pekhimenko, Evgeny Bolotin, Mike O'Connor, Onur Mutlu, Todd C. Mowry, Stephen W. Keckler
IEEE CAL, June 2015

A Case for Core-Assisted Bottleneck Acceleration in GPUs: Enabling Efficient Data Compression
Nandita Vijaykumar, Gennady Pekhimenko, Adwait Jog, Abhishek Bhowmick, Rachata Ausavarungnirun, Chita Das, Mahmut Kandemir, Todd C. Mowry, and Onur Mutlu
ISCA-42, June 2015

Page Overlays: An Enhanced Virtual Memory Framework to Enable Fine-grained Memory Management
Vivek Seshadri, Gennady Pekhimenko, Olatunji Ruwase, Onur Mutlu, Phillip B. Gibbons, Michael A. Kozuch, Todd C. Mowry, and Trishul Chilimbi
ISCA-42, June 2015

PocketTrend: Timely Identification and Delivery of Trending Search Content to Mobile Users
Gennady Pekhimenko, Dimitrios Lymberopoulos, Oriana Riva, Karin Strauss, and Doug Burger
WWW-15, May 2015

Energy-Efficient Data Compression for GPU Memory Systems
Gennady Pekhimenko, Evgeny Bolotin, Mike O'Connor, Onur Mutlu, Todd C. Mowry, and Stephen W. Keckler
SRC@ASPLOS, Short Paper, March 2015
First place in ACM Student Research Competition.

Exploiting Compressed Block Size as an Indicator of Future Reuse
Gennady Pekhimenko, Tyler Huberty, Rui Cai, Onur Mutlu, Phillip P. Gibbons, Michael A. Kozuch, and Todd C. Mowry
HPCA-21, February 2015

Adaptive-Latency DRAM: Optimizing DRAM Timing for the Common-Case
Donghyuk Lee, Yoongu Kim, Gennady Pekhimenko, Samira Khan, Vivek Seshadri, Kevin Chang, and Onur Mutlu
HPCA-21, February 2015

RFVP: Rollback-Free Value Prediction with Safe-to-Approximate Loads
Amir Yazdanbakhsh, Gennady Pekhimenko, Bradley Thwaites, Hadi Esmaeilzadeh, Taesoo Kim, Onur Mutlu, and Todd C. Mowry
SAFARI Technical Report, TR-SAFARI-2015-002, February 2015

Shifted Hamming Distance: A Fast and Accurate SIMD-Friendly Filter for Local Alignment in Read Mapping
Hongyi Xin, John Greth, John Emmons, Gennady Pekhimenko, Carl Kingsford, Can Alkan, Onur Mutlu
Oxford Bioinformatics, January 2015

Rollback-Free Value Prediction with Approximate Memory Loads
Bradley Thwaites, Gennady Pekhimenko, Amir Yazdanbakhsh, Girish Mururu, Jongse Park, Hadi Esmaeilzadeh, Onur Mutlu, Todd C. Mowry
PACT-23, Short Paper, August 2014

Linearly Compressed Pages: A Low-Complexity, Low-Latency Main Memory Compression Framework
Gennady Pekhimenko, Vivek Seshadri, Yoongu Kim, Hongyi Xin, Onur Mutlu, Philip B. Gibbons, Michael A. Kozuch, and Todd C. Mowry
MICRO-46, December 2013

RowClone: Fast and Energy-Efficient In-DRAM Bulk Data Copy and Initialization
Vivek Seshadri, Yoongu Kim, Chris Fallin, Donghyuk Lee, Rachata Ausavarungnirun, Gennady Pekhimenko, Yixin Luo, Onur Mutlu, Michael A. Kozuch, Phillip B. Gibbons, and Todd C. Mowry
MICRO-46, December 2013

Base-Delta-Immediate Compression: Practical Data Compression for On-Chip Caches
Gennady Pekhimenko, Vivek Seshadri, Onur Mutlu, Philip B. Gibbons, Michael A. Kozuch, and Todd C. Mowry
PACT-21, Septermber 2012

Linearly Compressed Pages: A Main Memory Compression Framework with Low Complexity and Low Latency
Gennady Pekhimenko, Todd C. Mowry, and Onur Mutlu
PACT-21, Short Paper, Septermber 2012
Second place in ACM Student Research Competition.

Software Automatic Tuning: From Concepts to State-of-the-Art Results
Gennady Pekhimenko and Angela Demke Brown
Chapter 19, Springer, September 2010

Efficient Program Compilation through Machine Learning Techniques
Gennady Pekhimenko and Angela Demke Brown
iWAPT, October 2009

Selected Talks (Videos)

Service

    Publicity Co-Chair for ASPLOS 2018
    Reviewer for:
    ISCA 2011-2016, MICRO 2011-2015, ASPLOS 2012, 2016, HPCA 2012-2017, PACT 2013-2014, DAC 2014-2015, DATE 2016, IISWC 2014, ICCD 2014, NOCS 2012, MICRO Top Picks 2012-2013,2015, IEEE Transactions on Parallel and Distributed Systems 2014-2015, TACO 2016, Transactions on Multi-Scale Computing Systems 2016, Information 2016, Transactions on Computers 2013,2015, Transactions on Very Large Scale Integration Systems 2011-2012, ICAC 2013.