| Bloom: A 176b-parameter open-access multilingual language model T Le Scao, A Fan, C Akiki, E Pavlick, S Iliæ, D Hesslow, R Castagné, ... | 1482 | 2023 |
| Zero: Memory optimizations toward training trillion parameter models S Rajbhandari, J Rasley, O Ruwase, Y He SC20: International Conference for High Performance Computing, Networking …, 2020 | 1042 | 2020 |
| Deepspeed: System optimizations enable training deep learning models with over 100 billion parameters J Rasley, S Rajbhandari, O Ruwase, Y He Proceedings of the 26th ACM SIGKDD International Conference on Knowledge …, 2020 | 960 | 2020 |
| Zero-infinity: Breaking the gpu memory wall for extreme scale deep learning S Rajbhandari, O Ruwase, J Rasley, S Smith, Y He Proceedings of the international conference for high performance computing …, 2021 | 282 | 2021 |
| Planck: millisecond-scale monitoring and control for commodity networks J Rasley, B Stephens, C Dixon, E Rozner, W Felter, K Agarwal, J Carter, ... Proceedings of the 2014 ACM conference on SIGCOMM, 407-418, 2014 | 276 | 2014 |
| Deepspeed-inference: enabling efficient inference of transformer models at unprecedented scale RY Aminabadi, S Rajbhandari, AA Awan, C Li, D Li, E Zheng, O Ruwase, ... SC22: International Conference for High Performance Computing, Networking …, 2022 | 220 | 2022 |
| Deepspeed-moe: Advancing mixture-of-experts inference and training to power next-generation ai scale S Rajbhandari, C Li, Z Yao, M Zhang, RY Aminabadi, AA Awan, J Rasley, ... International conference on machine learning, 18332-18346, 2022 | 196 | 2022 |
| Efficient queue management for cluster scheduling J Rasley, K Karanasos, S Kandula, R Fonseca, M Vojnovic, S Rao Proceedings of the Eleventh European Conference on Computer Systems, 1-15, 2016 | 143 | 2016 |
| Retaining sandbox containment despite bugs in privileged memory-safe code J Cappos, A Dadgar, J Rasley, J Samuel, I Beschastnikh, C Barsan, ... Proceedings of the 17th ACM conference on Computer and communications …, 2010 | 64 | 2010 |
| Hyperdrive: Exploring hyperparameters with pop scheduling J Rasley, Y He, F Yan, O Ruwase, R Fonseca Proceedings of the 18th ACM/IFIP/USENIX Middleware Conference, 1-13, 2017 | 63 | 2017 |
| Deepspeed-chat: Easy, fast and affordable rlhf training of chatgpt-like models at all scales Z Yao, RY Aminabadi, O Ruwase, S Rajbhandari, X Wu, AA Awan, ... arXiv preprint arXiv:2308.01320, 2023 | 43 | 2023 |
| Crowdsourcing from scratch: A pragmatic experiment in data collection by novice requesters A Papoutsaki, H Guo, D Metaxa-Kakavouli, C Gramazio, J Rasley, W Xie, ... Proceedings of the AAAI Conference on Human Computation and Crowdsourcing 3 …, 2015 | 28 | 2015 |
| Deepspeed-fastgen: High-throughput text generation for llms via mii and deepspeed-inference C Holmes, M Tanaka, M Wyatt, AA Awan, J Rasley, S Rajbhandari, ... arXiv preprint arXiv:2401.08671, 2024 | 16 | 2024 |
| Accelerating large scale deep learning inference through {DeepCPU} at microsoft M Zhang, S Rajbandari, W Wang, E Zheng, O Ruwase, J Rasley, J Li, ... 2019 USENIX Conference on Operational Machine Learning (OpML 19), 5-7, 2019 | 13 | 2019 |
| Mcr-dl: Mix-and-match communication runtime for deep learning Q Anthony, AA Awan, J Rasley, Y He, A Shafi, M Abduljabbar, ... 2023 IEEE International Parallel and Distributed Processing Symposium (IPDPS …, 2023 | 7 | 2023 |
| Detecting latent cross-platform api violations J Rasley, E Gessiou, T Ohmann, Y Brun, S Krishnamurthi, J Cappos 2015 IEEE 26th International Symposium on Software Reliability Engineering …, 2015 | 6 | 2015 |
| DeepSpeed4Science Initiative: Enabling Large-Scale Scientific Discovery through Sophisticated AI System Technologies SL Song, B Kruft, M Zhang, C Li, S Chen, C Zhang, M Tanaka, X Wu, ... arXiv preprint arXiv:2310.04610, 2023 | 4 | 2023 |
| Deepspeed inference: Enabling efficient inference of transformer models at unprecedented scale R Yazdani Aminabadi, S Rajbhandari, M Zhang, AA Awan, C Li, D Li, ... arXiv e-prints, arXiv: 2207.00032, 2022 | 2 | 2022 |
| Seattle: The Internet as a Testbed J Rasley, M Muhammad, A Hanson, S Morgan, A Loh, J Cappos | | 2011 |
He YuxiongMicrosoft ResearchVerified email at microsoft.com
