NameError: name 'DeepSpeedCPUAdam' is not defined

[lipiji]

I have installed the cpu-adam, but still met the following issue:

Successfully installed deepspeed-0.3.0+d720fdb
Removed build tracker: '/tmp/pip-req-tracker-iqs3vipd'
[SUCCESS] deepspeed successfully imported.
[INFO] torch install path: ['/dockerdata/xx/anaconda3/lib/python3.7/site-packages/torch']
[INFO] torch version: 1.5.1+cu101, torch.cuda: 10.1
[INFO] deepspeed install path: ['/dockerdata/xx/anaconda3/lib/python3.7/site-packages/deepspeed']
[INFO] deepspeed info: 0.3.0+d720fdb, d720fdb, master
[SUCCESS] apex extensions successfully installed
[INFO] using new-style apex
[SUCCESS] fused lamb successfully installed.
[SUCCESS] transformer kernels successfully installed.
[WARNING] sparse attention is NOT installed.
[SUCCESS] cpu-adam (used by ZeRO-offload) successfully installed.
Installation is successful

=======

And when I run this script:
sh ./scripts/ds_zero-offload_10B_pretrain_gpt2_model_parallel.sh

...python3.7/site-packages/deepspeed/runtime/zero/stage2.py", line 161, in init
and type(init_optimizer) == DeepSpeedCPUAdam)
NameError: name 'DeepSpeedCPUAdam' is not defined
Adam Optimizer #0 is created with AVX512 arithmetic capability.
Optimizer = DeepSpeedCPUAdam
Checking ZeRO support for optimizer=DeepSpeedCPUAdam type=<class 'deepspeed.ops.adam.cpu_adam.DeepSpeedCPUAdam'>
[2020-10-28 00:06:32,670] [INFO] [engine.py:613:_configure_zero_optimizer] Creating fp16 ZeRO stage 2 optimizer

[lipiji]

It seems a bug that an unimported Class is used:
https://github.com/microsoft/DeepSpeed/blob/master/deepspeed/runtime/zero/stage2.py#L161

[jeffra]

Hi @lipiji, so sorry you're running into this issue. We have since fixed this issue but the PR (#476) isn't merged yet (it should be merged by tomorrow). Feel free to try it out before it's merged if you'd like. Check out the notes at the top of the PR there are a lot of changes coming in the install process.

[linehammer]

Python knows the purposes of certain names (ex. built-in functions ). Other names are defined within the program (ex. variables). If Python encounters a name that it doesn't recognize, you'll probably get NameError: global name 'xx' is not defined error. In most cases, this error is triggered when Python sees a variable name (Global or Local) and doesn't know what it's for. These errors can happen if you forget to initialize a variable , if you misspell a variable, or if you misspell a reserved word such as "True". Before you use the global variable in your function for reading, it must be first initialized somewhere: either outside of the function or inside it.