Skip to content

lack of context initialization in case of host memory #18

Open

Description

@e-ago

In case of GDS_ALLOC_CQ_ON_GPU=1 or GDS_ALLOC_DBREC_ON_GPU=1 the gds_peer_malloc() function is called and the cuInit(0) function initialize the context.

On the contrary, if both QP and DBREC reside on host memory, libgdsync doesn't initialize properly the context before calling the gds_register_va() function and the following error can occur:

[19078] GDS ERR gds_register_mem_internal() Error 709 (context is destroyed) while register address=0x7f4570008000 size=65536 flags=00000003
[19078] GDS ERR register_range() got 22 while registering range [0x7f4570008000,0x7f4570017fff]
[19078] GDS ERR gds_register_va() error while registering range, returning 0 as error value
[19078] GDS ERR gds_create_cq() error 2 in ibv_exp_create_cq, old errno 2
[19078] GDS ERR gds_create_qp() error 2 while creating TX CQ, old_errno=2
[19078] [1] MP ERR mp_init() qp creation failed

To overcome this issue the application using gds needs to call, for example, a cudaFree(0) before creating QPs and CQs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions