Week 6. Apr. 26: Auto-encoders, Network & Table Learning - Orienting

[JunsolKim]

Post your questions here about: “Network Learning” & “Knowledge and Table Learning”, Thinking with Deep Learning, chapters 11 & 12

[maddiehealy]

In Chapter 12's HinSAGE example, ReLU and softmax are the chosen activation functions. Are there any combinations of activation functions to be wary about in this context?

[Marugannwg]

• Regarding analyzing networks through graphs, I think one essential question (before applying techniques) is -- how should we define each node and edge? (Any interesting examples beyond considering node as person and edge as frequency/relatedness?)

• Regarding heterogeneous graphs, does that mean we can have different types of nodes and types of edges on the same graph and process it with one algorithm? (e.g. If I have a graph with nodes representing people and multiple different types of edges (frequency of conversation, number of collaboration on project, student/teacher/job-title relationship...), what might I get out of it when I process it with HinSAGE?

[kceeyang]

The problem of the GNN transductive learning method was mentioned while discussing Deepwalk in Chapter 12 — that is, when a new node is added or a node is removed, the model needs to be retrained from scratch. As opposed to this, the GNN inductive learning approach, like GraphSAGE, would be able to test the trained model on new, unseen nodes. Although both methods have certain limitations, would this retraining effort and high memory and computational cost make the GNN inductive learning approach generally better than the GNN transductive learning method?

[XueweiLi1027]

Network Learning
Unlike running models on tabular or text data, network data often has clustering. Some clusterings might have more influence on the predicted outcome than others. During the network learning process, is it possible to manually assign weights to different clusterings?

[Pei0504]

Given the detailed exploration of Graph Neural Networks (GNNs) and their application in the document, how does the performance of Graph Attention Networks (GATs) compare with that of traditional Graph Convolutional Networks (GCNs) when applied to dynamic graphs, particularly in scenarios involving frequently updated network data such as social media interactions or traffic networks?

[guanhongliu2000]

How do hyperbolic spaces, as a specific type of non-Euclidean geometry, provide advantages over traditional Euclidean embeddings when modeling hierarchical data structures in networks? Could you discuss the intrinsic properties of hyperbolic spaces that make them particularly suited for capturing the complexities of hierarchical relationships, such as in social networks or biological systems? Additionally, what are the computational trade-offs and challenges associated with implementing hyperbolic embeddings, particularly in terms of scalability, computational complexity, and integration with existing machine learning frameworks? How do these factors affect the practical deployment of hyperbolic embeddings in large-scale applications?

[risakogit]

What are some examples of tasks that bridge the gap between network clustering and classification?

[CYL24]

In chapter 12, it seems that GraphSAGE outperforms shallow node embeddings like DeepWalk and Node2Vec by considering both graph structure and node attributes, which leads to more informative node embeddings that capture both local and global patterns in the graph. In that case, how sensitive is GraphSAGE's performance to hyperparameters such as layer sizes, sampling techniques, and optimization algorithms? Are there best practices for tuning these hyperparameters to achieve optimal performance?

In addition, one crucial step in GraphSAGE is sampling positive and negative node pairs based on their co-occurrence within a context window in the graph. How does the size of the context window impact the sampling procedure and the performance of GraphSAGE? Are there any trade-offs between a smaller or larger context window size?

[Xtzj2333]

Last week's papers on social simulacra also investigated social networks in communities such as Reddit. I wonder how social simulacra could be complement network learning in order to predict and interpret social network phenomena?

[uc-diamon]

How to apply graph modeling to other datasets that do not come in the form of nodes and edges? Moreover, how do we determine what is a node and what is an edge and their connections?

[HongzhangXie]

In this chapter, I learned how to use methods like DeepWalk to detect communities and plot graphs between them. I am curious about how to evaluate metrics for graph neural networks. What are some of the key metrics used to evaluate the performance of graph neural networks, and how do they differ when assessing tasks such as node classification versus community detection?

[anzhichen1999]

How can inductive graph embedding models, particularly GraphSAGE, be optimized to maintain high accuracy and efficiency in evolving social networks without losing the characteristcs mentioned in the DeepWalk paper?

[Brian-W00]

This chapter mentions various types of graph neural networks (GNN), including graph convolutional networks (GCN), graph attention networks (GAT), and other unsupervised deep graph learning methods. In practical applications, which tasks do these networks usually perform best?

[mingxuan-he]

Recently there are some applications that use knowledge graphs as the knowledge base for LLM agents and they got better retrieval accuracy compared to using simply chunked documents. But currently most knowledge documents exist in the unstructured format. Is it possible to convert unstructured data into knowledge graphs? If so what tools/algorithms would best at this task?

[erikaz1]

In class, we briefly touched upon the use of networks for causal analysis. Do networks need to achieve a particular structure/size prior to applying causal assumptions?