Sociotechnical Scale| Rethinking Sociotechnical Harms of Large Foundation Models: A Critical Reflection Through Scale Hacking

Abstract

With the rise of large general-purpose models, understanding the societal impact of AI systems becomes more difficult yet critical. Many AI harms are uniquely tied to scale, where certain sociotechnical risks only emerge or become intensified when widely adopted by users. Assessing these risks before large-scale model deployment is desirable; in practice, it is extremely challenging to study such harms without involving a large number of users. This paper introduces scale hacking—a set of methodological strategies from the human-computer interaction (HCI) literature designed to extend research insights without requiring massive sample sizes—as a lens to rethinking such harms. I reflect on how we applied these scale hacking techniques to examine the impact of foundation models in an applied setting (i.e., freelance economy), highlighting three effective approaches: (1) triangulating across multiple methods and data streams, (2) including participants with varying degrees of experience over both short and long time frames, and (3) focusing not on specific AI applications but on user practices such as disclosure of AI use.