Hip refactor for loss, dropout, flat, and gather

[Bob-Chen222]

Description of changes:

Related Issues:
Hip refactor for loss, dropout, flat, and gather

Linked Issues:

• Issue Hip kernel refactor #1296

Issues closed by this PR:

• Closes #

---

This change is 

[reyna-abhyankar]

comments

[Bob-Chen222]

Reviewable status: 0 of 28 files reviewed, 2 unresolved discussions (waiting on @reyna-abhyankar)

---

lib/kernels/src/hip/conv_2d_kernels.h line 0 at r1 (raw file):

Previously, reyna-abhyankar (Reyna Abhyankar) wrote…

why deleted?

Previously, I deleted it because I thought it was not in the cuda folder. Now recovered!

---

lib/kernels/src/hip/loss_function_kernels.cpp line 162 at r1 (raw file):

Previously, reyna-abhyankar (Reyna Abhyankar) wrote…

change cudaStream_t to hipStream_t

Done.

[reyna-abhyankar]

Reviewable status: 0 of 28 files reviewed, 3 unresolved discussions (waiting on @Bob-Chen222)

---

lib/kernels/src/hip/conv_2d_kernels.h line 0 at r1 (raw file):

Previously, Bob-Chen222 (Bob Chen) wrote…

Previously, I deleted it because I thought it was not in the cuda folder. Now recovered!

Why is the new path lib/kernel/src/conv_2d_kernels.h instead of lib/kernels/src/hip/ops/conv_2d_kernels.h?

---

lib/kernels/src/hip/loss_function_kernels.cpp line 171 at r2 (raw file):

  checkCUDA(get_legion_stream(&stream));
  hipLaunchKernelGGL(identity_loss_backward,
                     dim3(GET_BLOCKS(loss_volume), 1, 1),

We don't need dim3() here right? based on your comment on the last PR

[Bob-Chen222]

Reviewable status: 0 of 28 files reviewed, 2 unresolved discussions (waiting on @reyna-abhyankar)

---

lib/kernels/src/hip/loss_function_kernels.cpp line 171 at r2 (raw file):

Previously, reyna-abhyankar (Reyna Abhyankar) wrote…

We don't need dim3() here right? based on your comment on the last PR

Done.

[Bob-Chen222]

Reviewable status: 0 of 29 files reviewed, 2 unresolved discussions (waiting on @reyna-abhyankar)

---

lib/kernels/src/hip/conv_2d_kernels.h line 0 at r1 (raw file):

Previously, reyna-abhyankar (Reyna Abhyankar) wrote…

Why is the new path lib/kernel/src/conv_2d_kernels.h instead of lib/kernels/src/hip/ops/conv_2d_kernels.h?

Done.

[reyna-abhyankar]

Reviewable status: 0 of 29 files reviewed, 1 unresolved discussion (waiting on @Bob-Chen222)

---

lib/kernels/src/hip/conv_2d_kernels.h line 0 at r1 (raw file):

Previously, Bob-Chen222 (Bob Chen) wrote…

Done.

I still see two
https://github.com/Bob-Chen222/FlexFlow/blob/bob-hip-refactor-dfgl/lib/kernels/src/hip/ops/conv_2d_kernels.h
https://github.com/Bob-Chen222/FlexFlow/blob/bob-hip-refactor-dfgl/lib/kernels/src/conv_2d_kernels.h <--- get rid of the second one

---

lib/kernels/src/hip/loss_function_kernels.cpp line 80 at r4 (raw file):

  // launch kernel in hip
  hipLaunchKernelGGL(sparse_categorical_crossentropy_loss_backward,
                     GET_BLOCKS(num_samples * k),

Why?

[Bob-Chen222]

Reviewable status: 0 of 29 files reviewed, 1 unresolved discussion (waiting on @reyna-abhyankar)

---

lib/kernels/src/hip/conv_2d_kernels.h line 0 at r1 (raw file):

Previously, reyna-abhyankar (Reyna Abhyankar) wrote…

I still see two
https://github.com/Bob-Chen222/FlexFlow/blob/bob-hip-refactor-dfgl/lib/kernels/src/hip/ops/conv_2d_kernels.h
https://github.com/Bob-Chen222/FlexFlow/blob/bob-hip-refactor-dfgl/lib/kernels/src/conv_2d_kernels.h <--- get rid of the second one

removed.

---

lib/kernels/src/hip/loss_function_kernels.cpp line 80 at r4 (raw file):

Previously, reyna-abhyankar (Reyna Abhyankar) wrote…

Why?

removed. It shouldn't be there

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 38.10%. Comparing base (af1caf5) to head (23cacf7).

Additional details and impacted files

@@              Coverage Diff               @@
##           repo-refactor    #1373   +/-   ##
==============================================
  Coverage          38.10%   38.10%           
==============================================
  Files                167      167           
  Lines               5026     5026           
  Branches             246      246           
==============================================
  Hits                1915     1915           
  Misses              3111     3111           

Flag	Coverage Δ
unittests	38.10% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.