pyg::subgraph CUDA implementation

[rusty1s]

No description provided.

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 99.48%. Comparing base (c1656ed) to head (9113818).
Report is 359 commits behind head on master.

Additional details and impacted files

@@           Coverage Diff           @@
##           master      #42   +/-   ##
=======================================
  Coverage   99.48%   99.48%           
=======================================
  Files           9        9           
  Lines         195      195           
=======================================
  Hits          194      194           
  Misses          1        1           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[yaoyaowd]

I'm actually not sure if it is necessary to parallelize with 32 threads per nodes. Most of the time we are dealing with sparse data and a lot of threads will not go into for loop.

If you are looking for extreme performance, you can bundle to_local_node_data and col_data into one iterator structure and use this function. I haven't seen any better performance than it in the past.
https://nvlabs.github.io/cub/structcub_1_1_device_segmented_reduce.html#a4854a13561cb66d46aa617aab16b8825

[rusty1s]

Do you have an example of bundling to_local_node_data and col_data into one iterator structure? This looks really interesting.

I am okay with dropping the warp-level parallelism for now, but we will lose the contiguous access to col_data, and probably under-utilize the number of threads available on modern GPUs.

[rusty1s]

On a second look, this doesn't seem possible since col_data refers to edges, while to_local_node_data refers to nodes, while we actually want do the compute across the number of nodes in the induced subgraph.

[ZenoTan]

Does N means the number of nodes in the graph?
What if we could filter each node in nodes_data since it should be much smaller than rowptr_data.
Otherwise we may consider caching this tensor to reduce memory allocation for each time.

[rusty1s]

Good points! We use this vector as the mapping from global node indices to new local ones. In C++, we use a map for this but can't do the same in CUDA. I don't know of a more elegant solution for this.

Caching is an option as well, but requires a (non-intuitive and backend-specific) change in input arguments. I added it as a TODO for now.

[ZenoTan]

There's GPU hash table/set which may require some atomic operations when you build it, but lookup is fast.
I found that caching is not a good option since you have to reset the array every time.

Since you can sample on GPU, then the graph is not that big, a node array is not that bad and can make the code less complicated

[yaoyaowd]

Let's see if we can improve it later.