Week 5. Apr. 19: Transformers and Social Simulation - Orienting

[JunsolKim]

Post your questions here about: “Language Learning with Large Language Models” and “Digital Doubles for Simulation, Prediction and Insight” in Thinking with Deep Learning, chapters 11 and 18; “Social Simulacra: Creating Populated Prototypes for Social Computing Systems.” UIST. Park, Joon Sung, Lindsay Popowski, Carrie Cai, Meredith Ringel Morris, Percy Liang, Michael S. Bernstein. 2022.

[Marugannwg]

I'm deeply resonating with the discussion of LLM limitations. Short summary from my perspective here:

• Positivity Bias: LLMs often omit negative content due to safety or reputation concerns --> unrealistic (and boring!) when simulating certain interactions;
• Uniformity Bias: LLM responses tend to be more uniform and less varied than human responses (which is predictable and boring! due to their design, which averages language usage across vast datasets)
• Atemporality Bias & Monolingual Bias: Limited ability to analyze historical and cultural changes and inference about language differences.
• Senseless Bias: LLMs, based purely on text, lack direct sensory perceptions, leading to discrepancies between language associations and real-world perceptions.
  Inference @ Scale: Can be very, very costly dependent on your task!

I wonder if those limitations can be resolved (naturally) as larger models, following currents mainstream transformer architecture, come out.
And if we want to utilize existing model and API, what are some strategies (besides extensive retraining and tuning) that can handle those issues?

[maddiehealy]

After reading Social Simulacra, my (really broad!) question veers more on the philosophical side of DL, but to what extent is human behavior predictable? Reading about social simulacra prototyping reminded me of a quote from Adolphe Quetelet’s study of social phenomena that notes how “every suicide may be a product of individual choice, but all suicides in a year fall into knowable patterns.” I think this quote illustrates some ideas of social simulacra, especially getting into trackability of free will and more philosophical thinking in the computational social sciences.

I was also intrigued by the naming of this prototype as “social simulacra.” Is it meant to invoke Jean Baudrillard’s ideas simulacra? Maybe my introduction to what simulacra means is limited and/or skewed through an academic lens, but I have mainly encountered the concept of simulacra through Jean Baudrillard’s work on Simulacra and Simulation so far.

With Baudrillard’s work, I got a warning tone about simulacrum, specifically with quotes like “we live in a world where there is more and more information, and less and less meaning." But also, the stages of simulacra development end with the simulacrum having no relationship to any reality whatsoever. I've attached a screenshot summarizing the stages of simulacra development for reference. Did the authors of this paper intend to invoke this connotation when naming this social simulation prototype? Will social simulacra prototyping eventually follow the steps listed below?

[Pei0504]

The chapter 11 effectively outlines the advancements and applications of large language models like GPT, emphasizing their role in diverse fields from policy to business. It touches upon their development from basic models to complex systems capable of detailed simulations and interactions. However, it lacks a thorough discussion on ethical concerns and the potential biases in training data. How can developers ensure that the training of large language models like GPT incorporates ethical considerations and effectively mitigates biases found in the training data? Future explorations could benefit from a deeper focus on these ethical implications and the technological limitations to better understand and enhance the reliability of LLMs.

[HongzhangXie]

The GPT model employs the Transformer architecture, relying on self-attention mechanisms to process sequential data. By learning to predict the next word in a sentence, the model grasps the general patterns and structures of language, allowing it to consider the entire text sequence when generating each word, thereby enhancing its understanding of context.

GPT generates text based on probabilities. However, probability may not be the sole criterion for evaluating the quality of a model. For instance, the content produced by the model might be incorrect or unethical. How should we ensure that the model's generated content conforms to factual accuracy or ethical standards? For example, how can we make the machine recognize and refrain from generating a statement like "Benjamin Franklin was a President of the United States" that contradicts historical fact?

I am also particularly interested in the method of Reinforcement Learning with Human Feedback (RLHF). I would like to understand more about how it works. I have a concern that different groups may provide feedback with varying frequencies, and as feedback accumulates, the model might increasingly favor the preferences of those groups that provide feedback more actively, potentially neglecting the perspectives of less vocal groups.

[Xtzj2333]

As a language model, why is GPT 4 so good at coding? Why is it so bad at maths? Is this due to its transformer architecture, the data that it trains on, or something else?

[guanhongliu2000]

In Social Simulacra, the term "Social Simulacra" resonates with Jean Baudrillard’s concept of simulacra and simulation. How might the use of AI to simulate human social interactions relate to Baudrillard's stages of simulacra, especially concerning the blurring of boundaries between simulation and reality?

[kceeyang]

I am very interested in the Reinforcement Learning with Human Feedback (RLHF), which was introduced in Chapter 11. A multiple-model training process seems to be involved while optimizing a language model. A language model needs to be pretrained first, and a reward model trained with human feedback using methods from reinforcement learning would then be used to retune the language models. I am wondering, how people would ensure the quality and impartiality of the collected human feedback so that they would have a less biased model?

[mingxuan-he]

In the "simulating political actors" subsection, I think there's interesting caveat with using propriatary LLMs for political simulation. Recently many big news media like New York Times, CNN, Disney, etc. has banned openai's data scraper, therefore excluding themselves from the training data. How would that impact the quality of political simulation from openai's models?
None of the popular LLM benchmarks seem to measure the quality of political simulation, so is it possible that some older models like GPT-3 actually perform better/less biased at these tasks?

[beilrz]

One question I have for the design of LLM is that what incentive LLM have to model unlikely but logically true statement. Considering the following objective function, LLM should always return the mostly likely next token to reap the max gain in the objective function (for example, next token for "I love" would mostly likely be "you" rather than "myself"). This could be an issue if we want to construct a representative dataset from LLM output, would this issue mostly be addressed by using a higher temperature?

[uc-diamon]

In regards to chapter 12 Programming Foundation Models with Language, how does a zero-shot model "learn" without back propagation and minimizing a loss function?

[anzhichen1999]

What are the potential long-term effects on less-represented languages? (I think Xuechunzi Bai also touched on this topic)

[HamsterradYC]

What are the ethical considerations and potential societal impacts of using LLMs to simulate public opinion or forecast political behaviors? How might these simulations influence public policy or societal trust if made public?

[CYL24]

Chapter 11 mentions that sequential training of LLMs on ordered time periods could enhance their ability to simulate historical socio-cultural changes. In that case, do researchers take scalability and generalizability of these simulated social interactions across different historical contexts and cultural settings into account? If yes, how might researchers balance the need for accuracy from the perspective of history with the scalability required for large-scale simulations?

[Brian-W00]

Can LLMs have logical reasoning ability on the data not trained before?

[MarkValadez]

It seems to me there is a limitation to the self-attention scheme which still refrains from identifying a notion of meaning separate from the interpretation of language. I wonder what are the limitations if there is such a thing as the essential representation of Forms from Plato or something as the collective representations by Durkheim. Moreover, there is a notion of a personal image or idilic projection of object and meanings in our concouss self, separate from Forms or collective representations. While machines can emulate behavior, how does this difference inform our behavior as separate and distinguishable from an LLM?