André Storhaug
|
2518cc429d
Support `exclude_frozen_parameters` for `zero_to_fp32.py` script (#4979)
|
8 months ago |
Nadav Elyahu
|
691458f8b6
zero_to_fp32.py: Handle a case where shape doesn't have numel attr (#4842)
|
9 months ago |
Earlee
|
241ae39a29
zero_to_fp32 script adds support for tag argument (#4089)
|
1 year ago |
Stas Bekman
|
1cc9caa9c6
[zero_to_fp32] 3x less cpu memory requirements (#4025)
|
1 year ago |
Eugene Cheah
|
103884aeee
Update zero_to_fp32.py (#3936)
|
1 year ago |
Stas Bekman
|
77ebf760f3
[zero_to_fp32] fix shared param recovery (#3407)
|
1 year ago |
ShijieZZZZ
|
39825a9092
Fix redundant shared_params in zero_to_fp32.py (#3149)
|
1 year ago |
Olatunji Ruwase
|
dd8df20fe0
zero3 checkpoint frozen params (#3205)
|
1 year ago |
ShijieZZZZ
|
30d9770549
Recover shared parameters (#3033)
|
1 year ago |
Olatunji Ruwase
|
fcb868e27c
Fix launch issue (#3137)
|
1 year ago |
Michael Wyatt
|
b361c72761
Update DeepSpeed copyright license to Apache 2.0 (#3111)
|
1 year ago |
Jeff Rasley
|
91d63e0228
update formatter version and style settings (#3098)
|
1 year ago |
Olatunji Ruwase
|
541e423ae6
Enable tensor fragments for zero 2 & 3 (#2727)
|
1 year ago |
Jeff Rasley
|
da84e60d98
add missing license info to top of all source code (#2889)
|
1 year ago |
Alex Hedges
|
316c4a43e0
Add flake8 to pre-commit checks (#2051)
|
2 years ago |
Shuai Zheng
|
801c172345
fix file ordering (#1822)
|
2 years ago |
Olatunji Ruwase
|
135a625619
Move param_shapes to model files (#1732)
|
2 years ago |
eelxpeng
|
dd0c8fa73b
Revise param_shapes to be a list of ordered dict (#1424)
|
3 years ago |
Stas Bekman
|
dd6bf4d0f1
[zero_to_fp32] fix to handle world_size (#1422)
|
3 years ago |
Alex Hedges
|
be789b1665
Fix many typos (#1423)
|
3 years ago |
Jeff Rasley
|
e2fdd254ed
Big science related changes (#1407)
|
3 years ago |
Stas Bekman
|
364994ad34
[zero_to_fp32] fix padding removal (#1380)
|
3 years ago |
Stas Bekman
|
30537e719c
[zero_to_fp32] adapt to 4-bytes alignment in z2 (#1372)
|
3 years ago |
Ammar Ahmad Awan
|
f28432441b
DeepSpeed MoE (#1310)
|
3 years ago |
Stas Bekman
|
2a921069d7
[model weights] zero_to_fp32 multiple improvements (#1181)
|
3 years ago |
Stas Bekman
|
df8b1f884f
zero_to_fp32: restore persistent buffers (#1146)
|
3 years ago |
Stas Bekman
|
a8cf887d65
[zero_to_fp32.py] support param groups (#1017)
|
3 years ago |
Stas Bekman
|
7531c6bf53
full fp32 weights reconstruction for zero 2+3 (#892)
|
3 years ago |