Stas Bekman
|
30d3f5df7a
fix a mispelled attribute (#2750)
|
1 year ago |
Ma, Guokai
|
98cc35b6a8
Abstract accelerator (step 3) (#2677)
|
1 year ago |
Stas Bekman
|
ddd48b36ac
[GatheredParameters] fix memory leak (#2665)
|
1 year ago |
Stas Bekman
|
217cc07bb5
[GatheredParameters] add support for any iterator (#2664)
|
1 year ago |
iLeGend
|
06e00f61ce
Fix typos: deepseed -> deepspeed (#2499)
|
1 year ago |
Olatunji Ruwase
|
2210ebe70f
Release swap buffers for persisted params (#2089)
|
2 years ago |
Jeff Rasley
|
46401b3884
[zero-3] shutdown zero.Init from within ds.init (#2150)
|
2 years ago |
Michael Wyatt
|
5997589683
Refactor ZeRO configs to use Pydantic (#2004)
|
2 years ago |
Alex Hedges
|
316c4a43e0
Add flake8 to pre-commit checks (#2051)
|
2 years ago |
Quentin Anthony
|
5349347bb6
DeepSpeed Communication Profiling and Logging (#2012)
|
2 years ago |
Karim Foda
|
735406e536
fix import errors (#2026)
|
2 years ago |
Olatunji Ruwase
|
d86a2de993
Use partition size (#2011)
|
2 years ago |
Ammar Ahmad Awan
|
36ad3119d5
DeepSpeed comm backend v1 (#1985)
|
2 years ago |
kisseternity
|
5053217e5d
trivial fix (#1954)
|
2 years ago |
Jeff Rasley
|
50893458d6
Fairseq support (#1915)
|
2 years ago |
Manuel R. Ciosici
|
ae43ba1270
Update partition_parameters.py (#1943)
|
2 years ago |
Stas Bekman
|
73c0798bd7
GatheredParameters - accept a tuple of params (#1941)
|
2 years ago |
Olatunji Ruwase
|
673cb60808
Improve z3 trace management (#1916)
|
2 years ago |
Jeff Rasley
|
b4fcd98ff0
Inference PP changes for neox (#1899)
|
2 years ago |
Olatunji Ruwase
|
32d97976ce
Fix OOM and type mismatch (#1884)
|
2 years ago |
Stas Bekman
|
fb00e6a1db
[partition_parameters.py] better diagnostics (#1887)
|
2 years ago |
Alex Hedges
|
4cf970e6bb
Add codespell to pre-commit checks (#1717)
|
2 years ago |
Justin Chiu
|
4912e0ad7e
Various ZeRO Stage3 Optimizations + Improvements (including bfloat16 support) (#1453)
|
2 years ago |
Jeff Rasley
|
4b854a37cb
[zero-3] set default device during zero.Init (#1605)
|
2 years ago |
Alex Hedges
|
fc2f378ece
Improve pre-commit hooks (#1602)
|
2 years ago |
Jeff Rasley
|
2332cb31a7
Enables ZeRO-3 inference (#1514)
|
2 years ago |
Cheng Li
|
9caa74e577
Autotuning (#1554)
|
2 years ago |
Olatunji Ruwase
|
bd3ebddf36
Use cuda tensors for allgather (#1548)
|
2 years ago |
Zhen Zhang
|
c0eeb69dfb
ZeRO3, improved parameter all-gather operation (#1188)
|
3 years ago |
Olatunji Ruwase
|
58a8e13ccd
Ensure single zero3 context (#1462)
|
3 years ago |