Skip to content

Files

Latest commit

author
twitter-team
Mar 31, 2023
ef4c5eb · Mar 31, 2023

History

History
This branch is 27 commits behind twitter/the-algorithm:main.

user_tweet_entity_graph

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
Mar 31, 2023
Mar 31, 2023
Mar 31, 2023
Mar 31, 2023
Mar 31, 2023
Mar 31, 2023
Mar 31, 2023
Mar 31, 2023
Mar 31, 2023
Mar 31, 2023
Mar 31, 2023
Mar 31, 2023
Mar 31, 2023
Mar 31, 2023
Mar 31, 2023

UserTweetEntityGraph (UTEG)

What is it

User Tweet Entity Graph (UTEG) is a Finalge thrift service built on the GraphJet framework. In maintains a graph of user-tweet relationships and serves user recommendations based on traversals in this graph.

How is it used on Twitter

UTEG generates the "XXX Liked" out-of-network tweets seen on Twitter's Home Timeline. The core idea behind UTEG is collaborative filtering. UTEG takes a user's weighted follow graph (i.e a list of weighted userIds) as input, performs efficient traversal & aggregation, and returns the top weighted tweets engaged basd on # of users that engaged the tweet, as well as the engaged users' weights.

UTEG is a stateful service and relies on a Kafka stream to ingest & persist states. It maintains an in-memory user engagements over the past 24-48 hours. Older events are dropped and GC'ed.

For full details on storage & processing, please check out our open-sourced project GraphJet, a general-purpose high performance in-memory storage engine.