Hello, I am a beginner in GAT , and I've been studying your GATv2 code lately. I have a question while going through the code in
labml_nn/graphs/gatv2/init.py
When calculating g_sum
g_sum = g_l_repeat + g_r_repeat_interleave
You mentioned in the comments: Now we add the two tensors to get
$$
\lbrace\overrightarrow{g_{l1}} + \overrightarrow{g_{r1}}, \overrightarrow{g_{l1}} + \overrightarrow{g_{r2}}, \dots, \overrightarrow{g_{l1}} + \overrightarrow{g_{rN}}, \overrightarrow{g_{l2}} + \overrightarrow{g_{r1}}, \overrightarrow{g_{l2}} + \overrightarrow{g_{r2}}, \dots, \overrightarrow{g_{l2}} + \overrightarrow{g_{rN}}, \dots\rbrace
$$
But in the previous code, g_l_repeat gets
$$
\lbrace\overrightarrow{g_{l1}}, \overrightarrow{g_{l2}}, \dots, \overrightarrow{g_{lN}}, \overrightarrow{g_{l1}}, \overrightarrow{g_{l2}}, \dots, \overrightarrow{g_{lN}}, \dots\rbrace
$$
and g_r_repeat_interleave gets
$$
\lbrace\overrightarrow{g_{r1}}, \overrightarrow{g_{r1}}, \dots, \overrightarrow{g_{r1}}, \overrightarrow{g_{r2}}, \overrightarrow{g_{r2}}, \dots, \overrightarrow{g_{r2}}, \dots\rbrace
$$
So I think the result of adding the two tensors should be
$$
\lbrace\overrightarrow{g_{l1}} + \overrightarrow{g_{r1}}, \overrightarrow{g_{l2}} + \overrightarrow{g_{r2}}, \dots, \overrightarrow{g_{lN}} + \overrightarrow{g_{r1}}, \overrightarrow{g_{l1}} + \overrightarrow{g_{r2}}, \overrightarrow{g_{l2}} + \overrightarrow{g_{r2}}, \dots, \overrightarrow{g_{lN}} + \overrightarrow{g_{r2}}, \dots\rbrace
$$
I'm not sure whether I may have overlooked some crucial information or if there's a mismatch between your comments and the code. I would greatly appreciate it if you could help clarify my confusion. Thank you.
