The Personality Panorama: Conceptualizing Personality Through Big Behavioural Data
Corresponding Author
Ryan L. Boyd
Department of Psychology, Lancaster University, UK
The first author is also currently affiliated with Security Lancaster, Lancaster University, UK and Data Science Institute, Lancaster University, UK.
Correspondence to: Ryan L. Boyd, Department of Psychology, Lancaster University, Lancaster, UK LA1 4YF.
E-mail: r.boyd@lancaster.ac.uk
Search for more papers by this authorPaola Pasca
Department of History, Society and Human Studies, University of Salento, Italy
Search for more papers by this authorKevin Lanning
Wilkes Honors College, Florida Atlantic University, USA
Search for more papers by this authorCorresponding Author
Ryan L. Boyd
Department of Psychology, Lancaster University, UK
The first author is also currently affiliated with Security Lancaster, Lancaster University, UK and Data Science Institute, Lancaster University, UK.
Correspondence to: Ryan L. Boyd, Department of Psychology, Lancaster University, Lancaster, UK LA1 4YF.
E-mail: r.boyd@lancaster.ac.uk
Search for more papers by this authorPaola Pasca
Department of History, Society and Human Studies, University of Salento, Italy
Search for more papers by this authorKevin Lanning
Wilkes Honors College, Florida Atlantic University, USA
Search for more papers by this authorAbstract
Personality psychology has long been grounded in data typologies, particularly in the delineation of behavioural, life outcome, informant-report, and self-report sources of data from one another. Such data typologies are becoming obsolete in the face of new methods, technologies, and data philosophies. In this article, we discuss personality psychology's historical thinking about data, modern data theory's place in personality psychology, and several qualities of big data that urge a rethinking of personality itself. We call for a move away from self-report questionnaires and a reprioritization of the study of behaviour within personality science. With big data and behavioural assessment, we have the potential to witness the confluence of situated, seamlessly interacting psychological processes, forming an inclusive, dynamic, multiangle view of personality. However, big behavioural data come hand in hand with important ethical considerations, and our emerging ability to create a ‘personality panopticon’ requires careful and thoughtful navigation. For our research to improve and thrive in partnership with new technologies, we must not only wield our new tools thoughtfully, but humanely. Through discourse and collaboration with other disciplines and the general public, we can foster mutual growth and ensure that humanity's burgeoning technological capabilities serve, rather than control, the public interest. © 2020 European Association of Personality Psychology
References
- Adali, S., & Golbeck, J. (2012). Predicting personality with social behavior. 2012 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, 302–309. https://doi.org/10.1109/ASONAM.2012.58
- Anastasi, A. (1967). Psychology, psychologists, and psychological testing. American Psychologist, 22, 297–306. https://doi.org/10.1037/h0024521.
- Anastasi, A., & Urbina, S. (1997). Psychological testing. Prentice Hall.
- Anderson, C. (2008). The end of theory: The data deluge makes the scientific method obsolete. Wired. https://www.wired.com/2008/06/pb-theory/
- Apicella, C. L., Marlowe, F. W., Fowler, J. H., & Christakis, N. A. (2012). Social networks and cooperation in hunter-gatherers. Nature, 481, 497–501. https://doi.org/10.1038/nature10736.
- Azucar, D., Marengo, D., & Settanni, M. (2018). Predicting the Big 5 personality traits from digital footprints on social media: A meta-analysis. Personality and Individual Differences, 124, 150–159. https://doi.org/10.1016/j.paid.2017.12.018.
- Bachrach, Y., Kosinski, M., Graepel, T., Kohli, P., & Stillwell, D. (2012). Personality and patterns of Facebook usage. Proceedings of the 4th Annual ACM Web Science Conference, 24–32. https://doi.org/10.1145/2380718.2380722
- Bakker, M., Hartgerink, C. H. J., Wicherts, J. M., & van der Maas, H. L. J. (2016). Researchers' intuitions about power in psychological research. Psychological Science, 27, 1069–1077. https://doi.org/10.1177/0956797616647519.
- Balsis, S., Cooper, L. D., & Oltmanns, T. F. (2015). Are informant reports of personality more internally consistent than self reports of personality? Assessment, 22, 399–404. https://doi.org/10.1177/1073191114556100.
- Barbaro, M., & Zeller, T. (2006). A face is exposed for AOL searcher no. 4417749. The New York Times. https://www.nytimes.com/2006/08/09/technology/09aol.html
- Baumeister, R. F., Vohs, K. D., & Funder, D. C. (2007). Psychology as the science of self-reports and finger movements: Whatever happened to actual behavior? Perspectives on Psychological Science, 2, 396–403. https://doi.org/10.1111/j.1745-6916.2007.00051.x.
- Bentham, J. (1791). Panopticon; or, the inspection-house (2008 reprinting). Dodo Press.
- Bier, W. C. (1980). Privacy, a vanishing value? Fordham Univ Press.
- Binet, A., & Simon, Th. (1916). The development of intelligence in children (The Binet-Simon Scale). Williams & Wilkins Co. https://doi.org/10.1037/11069-000
- Birnbaum, L. M., Ernala, K. S., Rizvi, F. A., De Choudhury, M., & Kane, M. J. (2017). A collaborative approach to identifying social media markers of schizophrenia by employing machine learning and clinical appraisals. Journal of Medical Internet Research, 19, e289. https://doi.org/10.2196/jmir.7956.
- Bleidorn, W., & Hopwood, C. J. (2019). Using machine learning to advance personality assessment and theory. Personality and Social Psychology Review, 23, 190–203. https://doi.org/10.1177/1088868318772990.
- Block, J. (1977). Advancing the psychology of personality: Paradigmatic shift or improving the quality of research? In D. Magnusson & N. Endler (Eds.), Personality at the crossroads: Current issues in interactional psychology (pp. 37–63). Hillsdale, NJ: Erlbaum.
- Blok, A., & Pedersen, M. A. (2014). Complementary social science? Quali-quantitative experiments in a big data world. Big Data & Society, 1(), 2053951714543908. https://doi.org/10.1177/2053951714543908.
- Boase, J., & Ling, R. (2013). Measuring mobile phone use: Self-report versus log data. Journal of Computer-Mediated Communication, 18, 508–519. https://doi.org/10.1111/jcc4.12021.
- Bond, R. M., Fariss, C. J., Jones, J. J., Kramer, A. D. I., Marlow, C., Settle, J. E., & Fowler, J. H. (2012). A 61-million-person experiment in social influence and political mobilization. Nature, 489. https://doi.org/10.1038/nature11421.
- Borelli, J. L., Ramsook, K. A., Smiley, P., Bond, D. K., West, J. L., & Buttitta, K. H. (2017). Language matching among mother-child dyads: Associations with child attachment and emotion reactivity. Social Development, 26, 610–629. https://doi.org/10.1111/sode.12200.
- Bosson, J. K., Swann, W. B., & Pennebaker, J. W. (2000). Stalking the perfect measure of implicit self-esteem: The blind men and the elephant revisited? Journal of Personality and Social Psychology, 79, 631–643. https://doi.org/10.1037/0022-3514.79.4.631.
- Boyd, R. L., Pasca, P., & Conroy-Beam, D. (2019). You're only Jung once: Building generalized motivational systems theories using contemporary research on language. Psychological Inquiry, 30, 93–98. https://doi.org/10.1080/1047840X.2019.1633122.
- Boyd, R. L., & Pennebaker, J. W. (2017). Language-based personality: A new approach to personality in a digital world. Current Opinion in Behavioral Sciences, 18, 63–68. https://doi.org/10.1016/j.cobeha.2017.07.017.
- Boyd, R. L., Wilson, S. R., Pennebaker, J. W., Kosinski, M., Stillwell, D. J., & Mihalcea, R. (2015). Values in words: Using language to evaluate and understand personal values. Proceedings of the Ninth International AAAI Conference on Web and Social Media, 31–40. http://www.aaai.org/ocs/index.php/ICWSM/ICWSM15/paper/view/10482
- Brayfield, A. H. (1965). Testimony before the Senate Subcommittee on Constitutional Rights of the Committee on the Judiciary. June 7-10, 1965, Washington, D. C. The American Psychologist, 20, 888–954.
- Breiman, L. (1996). Bagging predictors. Machine Learning, 24, 123–140. https://doi.org/10.1023/A:1018054314350.
- Bremner-Harrison, S., Cypher, B. L., Van Horn Job, C., & Harrison, S. W. R. (2018). Assessing personality in San Joaquin kit fox in situ: Efficacy of field-based experimental methods and implications for conservation management. Journal of Ethology, 36, 23–33. https://doi.org/10.1007/s10164-017-0525-9.
- Brierley, S. S. (1920). The present attitude of employees to industrial psychology. British Journal of Psychology, 10, 210–227. https://doi.org/10.1111/j.2044-8295.1920.tb00020.x.
10.1111/j.2044-8295.1920.tb00020.x Google Scholar
- Briffa, M., & Greenaway, J. (2011). High in situ repeatability of behaviour indicates animal personality in the beadlet anemone actinia equina (cnidaria). PLoS ONE, 6, e21963. https://doi.org/10.1371/journal.pone.0021963.
- Brown, N. A., Blake, A. B., & Sherman, R. A. (2017). A snapshot of the life as lived: Wearable cameras in social and personality psychological science. Social Psychological and Personality Science, 8, 592–600. https://doi.org/10.1177/1948550617703170.
- Burdick, L., Mihalcea, R., Boyd, R. L., & Pennebaker, J. W. (2020). Analyzing connections between user attributes, images, and text. Cognitive Computation. https://doi.org/10.1007/s12559-019-09695-3.
- Canossa, A., Badler, J. B., El-Nasr, M. S., Tignor, S., & Colvin, R. C. (2015). In your face(t): Impact of personality and context on gameplay behavior. Proceedings of the 10th International Conference on the Foundations of Digital Games, 1–10.
- Carpita, M., Ciavolino, E., & Pasca, P. (2019). Exploring and modelling team performances of the Kaggle European Soccer database. Statistical Modelling, 19, 74–101. https://doi.org/10.1177/1471082X18810971.
- Celie, K.-B., & Paris, J. J. (2019). Respect for personhood: Concrete implications of a philosophical misunderstanding. Clinical Ethics, 14, 146–150. https://doi.org/10.1177/1477750919851057.
10.1177/1477750919851057 Google Scholar
- Chang, E. (2019). What women know about the internet. The New York Times. https://www.nytimes.com/2019/04/10/opinion/privacy-feminism.html
- Chung, C. K., & Pennebaker, J. W. (2008). Revealing dimensions of thinking in open-ended self-descriptions: An automated meaning extraction method for natural language. Journal of Research in Personality, 42, 96–132. https://doi.org/10.1016/j.jrp.2007.04.006.
- Cloninger, C. R., Sigvardsson, S., & Bohman, M. (1988). Childhood personality predicts alcohol abuse in young adults. Alcoholism: Clinical and Experimental Research, 12, 494–505. https://doi.org/10.1111/j.1530-0277.1988.tb00232.x.
- Confessore, N., & Hakim, D. (2017). Data firm says ‘secret sauce’ aided Trump; many scoff. The New York Times. https://www.nytimes.com/2017/03/06/us/politics/cambridge-analytica.html
- Cooper, C. (2019). Pitfalls of personality theory. Personality and Individual Differences, 151, 109551. https://doi.org/10.1016/j.paid.2019.109551.
- Corbyn, Z. (2012). Facebook experiment boosts US voter turnout. Nature News. https://doi.org/10.1038/nature.2012.11401
- Coxe, W. W. (1940). Tentative report of the Committee on Professional Ethics, American Association for Applied Psychology, September, 1939. Journal of Consulting Psychology, 4, 24–26. https://doi.org/10.1037/h0050914.
10.1037/h0050914 Google Scholar
- Cummings, J. A., & Day, T. E. (2019). But what do participants want? Comment on the “data sharing in psychology” special section (2018). American Psychologist, 74, 245–247. https://doi.org/10.1037/amp0000408.
- Custers, B., La Fors, K., Jozwiak, M., Esther, K., Bachlechner, D., Friedewald, M., & Aguzzi, S. (2017). Lists of ethical, legal, societal and economic issues of big data technologies (SSRN Scholarly Paper ID 3091018). Social Science Research Network. https://papers.ssrn.com/abstract=3091018
- Cyders, M. A., & Coskunpinar, A. (2011). Measurement of constructs using self-report and behavioral lab tasks: Is there overlap in nomothetic span and construct representation for impulsivity? Clinical Psychology Review, 31, 965–982. https://doi.org/10.1016/j.cpr.2011.06.001.
- Danescu-Niculescu-Mizil, C., West, R., Jurafsky, D., Leskovec, J., & Potts, C. (2013). No country for old members: User lifecycle and linguistic change in online communities. Proceedings of the 22Nd international conference on world wide web, 307–318. https://doi.org/10.1145/2488388.2488416
- Davidson, B. I., Jones, S. L., Joinson, A. N., & Hinds, J. (2019). The evolution of online ideological communities. PLoS ONE, 14, e0216932. https://doi.org/10.1371/journal.pone.0216932.
- Drew, C. (2016). Data science ethics in government. Philosophical Transactions. Series A, Mathematical, Physical, and Engineering Sciences, 374. https://doi.org/10.1098/rsta.2016.0119.
- Dunlop, W. L., Karan, A., Wilkinson, D., & Harake, N. (2019). Love in the first degree: Individual differences in first-person pronoun use and adult romantic attachment styles. Social Psychological and Personality Science, 1948550619847455. https://doi.org/10.1177/1948550619847455.
- Ellis, A. (1946). The validity of personality questionnaires. Psychological Bulletin, 43, 385–440. https://doi.org/10.1037/h0055483.
- Ellis, D. A., Davidson, B. I., Shaw, H., & Geyer, K. (2019). Do smartphone usage scales predict behavior? International Journal of Human-Computer Studies, 130, 86–92. https://doi.org/10.1016/j.ijhcs.2019.05.004.
- Endler, N. S. (1982). Interactionism: A personality model, but not yet a theory. Nebraska Symposium on Motivation, 155–200.
- Endler, N. S., & Parker, J. D. A. (1992). Interactionism revisited: Reflections on the continuing crisis in the personality area. European Journal of Personality, 6, 177–198. https://doi.org/10.1002/per.2410060302.
- Fan, S., Lau, R. Y. K., & Zhao, J. L. (2015). Demystifying big data analytics for business intelligence through the lens of marketing mix. Big Data Research, 2, 28–32. https://doi.org/10.1016/j.bdr.2015.02.006.
- Fang, R., Landis, B., Zhang, Z., Anderson, M. H., Shaw, J. D., & Kilduff, M. (2015). Integrating personality and social networks: A meta-analysis of personality, network position, and work outcomes in organizations. Organization Science, 26, 1243–1260. https://doi.org/10.1287/orsc.2015.0972.
- Fetterman, A. K., Boyd, R. L., & Robinson, M. D. (2015). Power versus affiliation in political ideology. Personality and Social Psychology Bulletin, 41, 1195–1206. https://doi.org/10.1177/0146167215591960.
- Foucault, M. (1977). Discipline and punish: The birth of the prison. Pantheon Books.
- Funder, D. C. (2015). The personality puzzle ( 7th ed.). W. W. Norton & Company, Inc.
- Furr, R. M. (2009a). Personality psychology as a truly behavioural science. European Journal of Personality, 23, 369–401. https://doi.org/10.1002/per.724.
- Furr, R. M. (2009b). The study of behaviour in personality psychology: Meaning, importance and measurement. European Journal of Personality, 23, 437–453. https://doi.org/10.1002/per.726.
- Garcia, D. (2017). Leaking privacy and shadow profiles in online social networks. Science Advances, 3. https://doi.org/10.1126/sciadv.1701172.
- General Data Protection Regulation. (2016). Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation) (Text with EEA relevance), Pub. L. No. 32016R0679, 119 OJ L. http://data.europa.eu/eli/reg/2016/679/oj/eng
- Geukes, K., Nestler, S., Hutteman, R., Küfner, A. C. P., & Back, M. D. (2017). Trait personality and state variability: Predicting individual differences in within- and cross-context fluctuations in affect, self-evaluations, and behavior in everyday life. Journal of Research in Personality, 69, 124–138. https://doi.org/10.1016/j.jrp.2016.06.003.
- Gleibs, I. H. (2014). Turning virtual public spaces into laboratories: Thoughts on conducting online field studies using social network sites. Analyses of Social Issues and Public Policy, 14, 352–370. https://doi.org/10.1111/asap.12036.
- Gluck, J. P. (1997). Harry F. Harlow and animal research: Reflection on the ethical paradox. Ethics & Behavior, 7, 149–161. https://doi.org/10.1207/s15327019eb0702_6.
- Golbeck, J. A. (2016). Predicting personality with social media. AIS Transactions on Replication Research, 2, 1–10.
10.17705/1atrr.00009 Google Scholar
- Goldberg, A., Manian, G., Monroe, W., Potts, C., & Srivastava, S. B. (2015). Fitting in or standing out? The tradeoffs of structural and cultural embeddedness. Academy of Management Proceedings, 2015, 12263. https://doi.org/10.5465/AMBPP.2015.12263abstract.
10.5465/ambpp.2015.12263abstract Google Scholar
- Goldstone, R. L., & Lupyan, G. (2016). Discovering psychological principles by mining naturally occurring data sets. Topics in Cognitive Science, 8, 548–568. https://doi.org/10.1111/tops.12212.
- Gosling, S. D., Augustine, A. A., Vazire, S., Holtzman, N., & Gaddis, S. (2011). Manifestations of personality in online social networks: Self-reported Facebook-related behaviors and observable profile information. Cyberpsychology, Behavior and Social Networking, 14, 483–488. https://doi.org/10.1089/cyber.2010.0087.
- Gosling, S. D., Ko, S. J., Mannarelli, T., & Morris, M. E. (2002). A room with a cue: Personality judgments based on offices and bedrooms. Journal of Personality and Social Psychology, 82, 379–398. https://doi.org/10.1037/0022-3514.82.3.379.
- Gough, H. G. (1956). California Psychological Inventory. Consulting Psychologists Press.
- Gough, H. G., & Bradley, P. (1996). CPI manual. Consulting Psychologists Press.
- Groom, C. J., & Pennebaker, J. W. (2002). Words. Journal of Research in Personality, 36, 615–621.
- Guntuku, S. C., Yaden, D. B., Kern, M. L., Ungar, L. H., & Eichstaedt, J. C. (2017). Detecting depression and mental illness on social media: An integrative review. Current Opinion in Behavioral Sciences, 18, 43–49. https://doi.org/10.1016/j.cobeha.2017.07.005.
- Hair, J., Hult, G. T. M., Ringle, C. M., & Sarstedt, M. (2016). A primer on partial least squares structural equation modeling ( Second ed.). SAGE Publications, Inc.
- Hand, D. J. (2018). Aspects of data ethics in a changing world: Where are we now? Big Data, 6, 176–190. https://doi.org/10.1089/big.2018.0083.
- Hänsel, K., Alomainy, A., & Haddadi, H. (2016). Large scale mood and stress self-assessments on a smartwatch. Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing Adjunct – UbiComp'16, 1180–1184. https://doi.org/10.1010/ggcgcp
- Harari, G. M., Müller, S. R., Aung, M. S., & Rentfrow, P. J. (2017). Smartphone sensing methods for studying behavior in everyday life. Current Opinion in Behavioral Sciences, 18, 83–90. https://doi.org/10.1016/j.cobeha.2017.07.018.
- Harlow, L. L., & Oswald, F. L. (2016). Big data in psychology: Introduction to the special issue. Psychological Methods, 21, 447–457. https://doi.org/10.1037/met0000120.
- Hassanpour, S., Tomita, N., DeLise, T., Crosier, B., & Marsch, L. A. (2019). Identifying substance use risk based on deep neural networks and Instagram social media data. Neuropsychopharmacology, 44, 487–494. https://doi.org/10.1038/s41386-018-0247-x.
- Healey, J. (2011). Recording affect in the field: Towards methods and metrics for improving ground truth labels. In S. D'Mello, A. Graesser, B. Schuller, & J.-C. Martin (Eds.), Affective computing and intelligent interaction (pp. 107–116). Springer https://doi.org/10.1007/978-3-642-24600-5_14.
10.1007/978-3-642-24600-5_14 Google Scholar
- Henrich, J., Heine, S. J., & Norenzayan, A. (2010). The weirdest people in the world? The Behavioral and Brain Sciences, 33, 61–83; discussion 83-135. https://doi.org/10.1017/S0140525X0999152X.
- Hilbig, B. E., Thielmann, I., Hepp, J., Klein, S. A., & Zettler, I. (2015). From personality to altruistic behavior (and back): Evidence from a double-blind dictator game. Journal of Research in Personality, 55, 46–50. https://doi.org/10.1016/j.jrp.2014.12.004.
- Hofmans, J., De Clercq, B., Kuppens, P., Verbeke, L., & Widiger, T. A. (2019). Testing the structure and process of personality using ambulatory assessment data: An overview of within-person and person-specific techniques. Psychological Assessment, 31, 432–443. https://doi.org/10.1037/pas0000562.
- Hogan, R., Hogan, J., & Roberts, B. W. (1996). Personality measurement and employment decisions: Questions and answers. American Psychologist, 51, 469–477. https://doi.org/10.1037/0003-066X.51.5.469.
- Hoover, J., Dehghani, M., Johnson, K., Iliev, R., & Graham, J. (2018). Into the wild: Big data analytics in moral psychology. In Atlas of moral psychology (pp. 525–536). The Guilford Press.
- Hough, L. M. (2001). I/Owes its advances to personality. In Personality psychology in the workplace (pp. 19–44). American Psychological Association https://doi.org/10.1037/10434-001.
10.1037/10434-001 Google Scholar
- Hudson, N. W., & Fraley, R. C. (2015). Volitional personality trait change: Can people choose to change their personality traits? Journal of Personality and Social Psychology, 109, 490–507. https://doi.org/10.1037/pspp0000021.
- Iliev, R., Hoover, J., Dehghani, M., & Axelrod, R. (2016). Linguistic positivity in historical texts reflects dynamic environmental and psychological factors. Proceedings of the National Academy of Sciences, 113, E7871–E7879. https://doi.org/10.1073/pnas.1612058113.
- IntHout, J., Ioannidis, J. P. A., Borm, G. F., & Goeman, J. J. (2015). Small studies are more heterogeneous than large ones: A meta-meta-analysis. Journal of Clinical Epidemiology, 68, 860–869. https://doi.org/10.1016/j.jclinepi.2015.03.017.
- Jordan, K. N., Sterling, J., Pennebaker, J. W., & Boyd, R. L. (2019). Examining long-term trends in politics and culture through language of political leaders and cultural institutions. Proceedings of the National Academy of Sciences, 201811987, https://doi.org/10.1073/pnas.1811987116.
- Kahler, C. W., Read, J. P., Wood, M. D., & Palfai, T. P. (2003). Social environmental selection as a mediator of gender, ethnic, and personality effects on college student drinking. Psychology of Addictive Behaviors, 17, 226–234. https://doi.org/10.1037/0893-164X.17.3.226.
- Keren, G., & Schul, Y. (2009). Two is not always better than one: A critical evaluation of two-system theories. Perspectives on Psychological Science, 4, 533–550. https://doi.org/10.1111/j.1745-6924.2009.01164.x.
- Kern, M. L., Eichstaedt, J. C., Schwartz, H. A., Dziurzynski, L., Ungar, L. H., Stillwell, D. J., Kosinski, M., … Seligman, M. E. P. (2014). The online social self: An open vocabulary approach to personality. Assessment, 21, 158–169. https://doi.org/10.1177/1073191113514104.
- Kindon, S., Pain, R., & Kesby, M. (2007). Participatory action research: Origins, approaches, and methods. In S. Kindon, R. Pain, & M. Kesby (Eds.), Participatory action research approaches and methods. Routledge https://doi.org/10.4324/9780203933671.
- Kitchin, R. (2014). Big data, new epistemologies and paradigm shifts. Big Data & Society, 1(), 2053951714528481. https://doi.org/10.1177/2053951714528481.
- Kramer, A. D. I., Guillory, J. E., & Hancock, J. T. (2014). Experimental evidence of massive-scale emotional contagion through social networks. Proceedings of the National Academy of Sciences, 111, 8788–8790. https://doi.org/10.1073/pnas.1320040111.
- Krasner, L. (1962). Behavior control and social responsibility. American Psychologist, 17, 199–204.
- Krause, J., James, R., & Croft, D. P. (2010). Personality in the context of social networks. Philosophical Transactions of the Royal Society, B: Biological Sciences, 365, 4099–4106. https://doi.org/10.1098/rstb.2010.0216.
- Krueger, R. F. (2005). Continuity of Axes I and II: Toward a unified model of personality, personality disorders, and clinical disorders. Journal of Personality Disorders, 19, 233–261.
- Lambiotte, R., & Kosinski, M. (2014). Tracking the digital footprints of personality. Proceedings of the IEEE, 102, 1934–1939. https://doi.org/10.1109/JPROC.2014.2359054.
- Landers, R. N., Brusso, R. C., Cavanaugh, K. J., & Collmus, A. B. (2016). A primer on theory-driven web scraping: Automatic extraction of big data from the Internet for use in psychological research. Psychological Methods, 21, 475–492. https://doi.org/10.1037/met0000081.
- Lanning, K., Pauletti, R. E., King, L. A., & McAdams, D. P. (2018). Personality development through natural language. Nature Human Behaviour, 1. https://doi.org/10.1038/s41562-018-0329-0.
- Lazer, D., & Kennedy, R. (2015). What we can learn from the epic failure of Google Flu Trends. Wired. https://www.wired.com/2015/10/can-learn-epic-failure-google-flu-trends/
- Lazer, D., Kennedy, R., King, G., & Vespignani, A. (2014). The parable of Google Flu: Traps in big data analysis. Science (New York, N.Y.), 343, 1203–1205. https://doi.org/10.1126/science.1248506.
- Li, Y., Ishi, C. T., Ward, N., Inoue, K., Nakamura, S., Takanashi, K., & Kawahara, T. (2017). Emotion recognition by combining prosody and sentiment analysis for expressing reactive emotion by humanoid robot. 2017 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), 1356–1359. https://doi.org/10.1109/APSIPA.2017.8282243
- Lien, C.-Y., Bai, G.-J., & Chen, H.-H. (2019). Visited websites may reveal users' demographic information and personality. 2019 IEEE/WIC/ACM International Conference on Web Intelligence (WI), 248–252.
- Lin, H.-T., Balcan, M.-F., Hadsell, R., & Ranzato, M. (2020). Getting started with NeurIPS 2020. Medium. https://medium.com/@NeurIPSConf/getting-started-with-neurips-2020-e350f9b39c28
- Liu, L., Preotiuc-Pietro, D., Samani, Z. R., Moghaddam, M. E., & Ungar, L. (2016). Analyzing personality through social media profile picture choice. Tenth International AAAI Conference on Web and Social Media. Tenth International AAAI Conference on Web and Social Media. https://www.aaai.org/ocs/index.php/ICWSM/ICWSM16/paper/view/13102
- Maltby, J., Wood, A. M., Day, L., & Pinto, D. (2012). The position of authenticity within extant models of personality. Personality and Individual Differences, 52, 269–273. https://doi.org/10.1016/j.paid.2011.10.014.
- Manzo, L. C., & Brightbill, N. (2007). Toward a participatory ethics. In S. Kindon, R. Pain, & M. Kesby (Eds.), Participatory action research approaches and methods. Routledge https://doi.org/10.4324/9780203933671.
- Marks, D. F. (2018). American psychologists, the Central Intelligence Agency, and enhanced interrogation. Health Psychology Open, 5. https://doi.org/10.1177/2055102918796610.
- Martin, J. (2019). What the bible says about secrets. The New York Times. https://www.nytimes.com/2019/04/10/opinion/sunday/privacy-religion.html
- Matz, S. C., Gladstone, J. J., & Stillwell, D. J. (2016). Money buys happiness when spending fits our personality. Psychological Science, 27, 715–725. https://doi.org/10.1177/0956797616635200.
- Matz, S. C., Kosinski, M., Nave, G., & Stillwell, D. J. (2017). Psychological targeting as an effective approach to digital mass persuasion. Proceedings of the National Academy of Sciences, 114, 12714–12719. https://doi.org/10.1073/pnas.1710966114.
- Mauss, I. B., & Robinson, M. D. (2009). Measures of emotion: A review. Cognition & Emotion, 23, 209–237. https://doi.org/10.1080/02699930802204677.
- Maxwell, S. E., Lau, M. Y., & Howard, G. S. (2015). Is psychology suffering from a replication crisis? What does “failure to replicate” really mean? American Psychologist, 70, 487–498. https://doi.org/10.1037/a0039400.
- Mayer, J. D. (2005). A tale of two visions: Can a new view of personality help integrate psychology? American Psychologist, 60, 294–307. https://doi.org/10.1037/0003-066X.60.4.294.
- Mayer, J. D. (2007). The big questions of personality psychology: Defining common pursuits of the discipline. Imagination, Cognition and Personality, 27, 3–26. https://doi.org/10.2190/IC.27.1.b.
10.2190/IC.27.1.b Google Scholar
- Mayer, J. D., & Allen, J. L. (2013). A personality framework for the unification of psychology. Review of General Psychology, 17, 196–202. https://doi.org/10.1037/a0032934.
- McAbee, S. T., & Connelly, B. S. (2016). A multi-rater framework for studying personality: The trait-reputation-identity model. Psychological Review, 123, 569–591. https://doi.org/10.1037/rev0000035.
- McMullan, T. (2015). What does the panopticon mean in the age of digital surveillance? The Guardian. https://www.theguardian.com/technology/2015/jul/23/panopticon-digital-surveillance-jeremy-bentham
- Meehl, P. E. (1945). The dynamics of “structured” personality tests. Journal of Clinical Psychology, 1, 296–303. https://doi.org/10.1002/1097-4679(194510)1:4<296::AID-JCLP2270010410>3.0.CO;2-#.
- Meerloo, J. A. M. (1956). The rape of the mind: The psychology of thought control, menticide, and brainwashing. World Pub. Co.
10.1037/13187-000 Google Scholar
- Mehl, M. R. (2017). The electronically activated recorder (EAR): A method for the naturalistic observation of daily social behavior. Current Directions in Psychological Science, 26, 184–190. https://doi.org/10.1177/0963721416680611.
- Miller, G. (2012). The smartphone psychology manifesto. Perspectives on Psychological Science, 7, 221–237. https://doi.org/10.1177/1745691612441215.
- Mitchell, L., Frank, M. R., Harris, K. D., Dodds, P. S., & Danforth, C. M. (2013). The geography of happiness: Connecting Twitter sentiment and expression, demographics, and objective characteristics of place. PLoS ONE, 8, e64417. https://doi.org/10.1371/journal.pone.0064417.
- Mitra, T., Counts, S., & Pennebaker, J. W. (2016). Understanding anti-vaccination attitudes in social media. Proceedings of the Tenth International AAAI Conference on Web and Social Media. International AAAI Conference on Web and Social Media. https://www.aaai.org/ocs/index.php/ICWSM/ICWSM16/paper/view/13073
- Mozur, P. (2018). Inside China's dystopian dreams: A.I., shame and lots of cameras. The New York Times. https://www.nytimes.com/2018/07/08/business/china-surveillance-technology.html
- Murray, H. A. (1938). Explorations in personality. Oxford Univ. Press.
- Narayanan, A., & Shmatikov, V. (2008). Robust de-anonymization of large sparse datasets. 2008 IEEE Symposium on Security and Privacy (Sp 2008), 111–125. https://doi.org/10.1109/SP.2008.33
- Negley, G. (1966). Philosophical views on the value of privacy. Law and Contemporary Problems, 31, 319–325.
10.2307/1190674 Google Scholar
- Nye, C. D., Allemand, M., Gosling, S. D., Potter, J., & Roberts, B. W. (2016). Personality trait differences between young and middle-aged adults: Measurement artifacts or actual trends? Journal of Personality, 84, 473–492. https://doi.org/10.1111/jopy.12173.
- Oboler, A., Welsh, K., & Cruz, L. (2012). The danger of big data: Social media as computational social science. First Monday, 17. https://doi.org/10.5210/fm.v17i7.3993.
- Obschonka, M., Stuetzer, M., Rentfrow, P. J., Shaw-Taylor, L., Satchell, M., Silbereisen, R. K., Potter, J., et al. (2018). In the shadow of coal: How large-scale industries contributed to present-day regional differences in personality and well-being. Journal of Personality and Social Psychology, 115, 903–927. https://doi.org/10.1037/pspp0000175.
- O'Neil, C. (2016). Weapons of math destruction: How big data increases inequality and threatens democracy. Penguin Random House USA Ex.
- O'Neil, C., & Schutt, R. (2013). Doing data science: Straight talk from the frontline ( 1st ed.). O'Reilly Media.
- Orwell, G. (1949). Nineteen eighty-four: A novel. Secker and Warburg.
- Pan, W., Altshuler, Y., & Pentland, A. (Sandy). (2012). Decoding social influence and the wisdom of the crowd in financial trading network. MIT Web Domain. https://dspace.mit.edu/handle/1721.1/80764
- Pardo, Y., Aguilar, R., Molinuevo, B., & Torrubia, R. (2007). Alcohol use as a behavioural sign of disinhibition: Evidence from J.A. Gray's model of personality. Addictive Behaviors, 32, 2398–2403. https://doi.org/10.1016/j.addbeh.2007.02.010.
- Park, G., Schwartz, H. A., Eichstaedt, J. C., Kern, M. L., Kosinski, M., Stillwell, D. J., Ungar, L. H., et al. (2015). Automatic personality assessment through social media language. Journal of Personality and Social Psychology, 108, 934–952. https://doi.org/10.1037/pspp0000020.
- Paulhus, D. L., & Vazire, S. (2007). The self-report method. In Handbook of research methods in personality psychology (pp. 224–239). Guilford.
- Paxton, A., & Griffiths, T. L. (2017). Finding the traces of behavioral and cognitive processes in big data and naturally occurring datasets. Behavior Research Methods, 49, 1630–1638. https://doi.org/10.3758/s13428-017-0874-x.
- Pennebaker, J. W., Chung, C. K., Frazee, J., Lavergne, G. M., & Beaver, D. I. (2014). When small words foretell academic success: The case of college admissions essays. PLoS ONE, 9, e115844.
- Powell, R. A., Digdon, N., Harris, B., & Smithson, C. (2014). Correcting the record on Watson, Rayner, and Little Albert: Albert Barger as “psychology's lost boy”. The American Psychologist, 69, 600–611. https://doi.org/10.1037/a0036854.
- Power, C., & Elliott, J. (2006). Cohort profile: 1958 British birth cohort (National Child Development Study). International Journal of Epidemiology, 35, 34–41. https://doi.org/10.1093/ije/dyi183.
- Prilleltensky, I. (1989). Psychology and the status quo. American Psychologist, 44, 795–802. https://doi.org/10.1037/0003-066X.44.5.795.
- Prince, S. A., Adamo, K. B., Hamel, M. E., Hardt, J., Gorber, S. C., & Tremblay, M. (2008). A comparison of direct versus self-report measures for assessing physical activity in adults: A systematic review. International Journal of Behavioral Nutrition and Physical Activity, 5, 56. https://doi.org/10.1186/1479-5868-5-56.
- Pryor, J. B., Gibbons, F. X., Wicklund, R. A., Fazio, R. H., & Hood, R. (1977). Self-focused attention and self-report validity. Journal of Personality, 45, 513–527. https://doi.org/10.1111/j.1467-6494.1977.tb00168.x.
- Quodling, A. (2018). Shadow profiles—Facebook knows about you, even if you're not on Facebook. The Conversation. https://theconversation.com/shadow-profiles-facebook-knows-about-you-even-if-youre-not-on-facebook-94804
- Rauthmann, J. F. (2017). Development of implicit personality. In J. Specht (Ed.), Personality development across the lifespan (pp. 537–554). Academic Press https://doi.org/10.1016/B978-0-12-804674-6.00032-6.
10.1016/B978-0-12-804674-6.00032-6 Google Scholar
- Rauthmann, J. F., Gallardo-Pujol, D., Guillaume, E. M., Todd, E., Nave, C. S., Sherman, R. A., Ziegler, M., … Funder, D. C. (2014). The Situational Eight DIAMONDS: A taxonomy of major dimensions of situation characteristics. Journal of Personality and Social Psychology, 107, 677–718. https://doi.org/10.1037/a0037250.
- Rauthmann, J. F., & Sherman, R. A. (2018). The description of situations: Towards replicable domains of psychological situation characteristics. Journal of Personality and Social Psychology, 114, 482–488. https://doi.org/10.1037/pspp0000162.
- Reece, A. G., & Danforth, C. M. (2017). Instagram photos reveal predictive markers of depression. EPJ Data Science, 6, 1–12. https://doi.org/10.1140/epjds/s13688-017-0110-z.
- Reiss, J., & Sprenger, J. (2017). Scientific objectivity. In E. N. Zalta (Ed.), The Stanford encyclopedia of philosophy (Winter 2017). Metaphysics Research Lab, Stanford University https://plato.stanford.edu/archives/win2017/entries/scientific-objectivity/.
- Rexroth, K. (1959). Bird in the bush: Obvious essays. New Directions.
- Reynolds, B., Ortengren, A., Richards, J. B., & de Wit, H. (2006). Dimensions of impulsive behavior: Personality and behavioral measures. Personality and Individual Differences, 40, 305–315. https://doi.org/10.1016/j.paid.2005.03.024.
- Reynolds, K. J., Turner, J. C., Branscombe, N. R., Mavor, K. I., Bizumic, B., & Subašić, E. (2010). Interactionism in personality and social psychology: An integrated approach to understanding the mind and behaviour. European Journal of Personality, 24, 458–482. https://doi.org/10.1002/per.782.
- Roberts, J. J. (2018). The GDPR is in effect: A plain English guide for US companies. Fortune https://fortune.com/2018/05/24/the-gdpr-is-in-effect-should-u-s-companies-be-afraid/
- Rosenthal, S. A., & Hooley, J. M. (2010). Narcissism assessment in social–personality research: Does the association between narcissism and psychological health result from a confound with self-esteem? Journal of Research in Personality, 44, 453–465. https://doi.org/10.1016/j.jrp.2010.05.008.
- Roski, J., Bo-Linn, G. W., & Andrews, T. A. (2014). Creating value in health care through big data: Opportunities and policy implications. Health Affairs, 33, 1115–1122. https://doi.org/10.1377/hlthaff.2014.0147.
- Sariyska, R., & Montag, C. (2019). An overview on doing psychodiagnostics in personality psychology and tracking physical activity via smartphones. In H. Baumeister, & C. Montag (Eds.), Digital phenotyping and mobile sensing: New developments in psychoinformatics (pp. 45–63). Springer International Publishing https://doi.org/10.1007/978-3-030-31620-4_4.
10.1007/978-3-030-31620-4_4 Google Scholar
- Saxbe, D. E., Yang, X.-F., Borofsky, L. A., & Immordino-Yang, M. H. (2013). The embodiment of emotion: Language use during the feeling of social emotions predicts cortical somatosensory activity. Social Cognitive and Affective Neuroscience, 8, 806–812. https://doi.org/10.1093/scan/nss075.
- Schiele, B. C., Baker, A. B., & Hathaway, S. R. (1943). The Minnesota Multiphasic Personality Inventory. The Journal-Lancet, 63, 292–297.
- Schmit, M. J., Ryan, A. M., Stierwalt, S. L., & Powell, A. B. (1995). Frame-of-reference effects on personality scale scores and criterion-related validity. Journal of Applied Psychology, 80, 607–620. https://doi.org/10.1037/0021-9010.80.5.607.
- Segalin, C., Celli, F., Polonio, L., Kosinski, M., Stillwell, D., Sebe, N., Cristani, M., & Lepri, B. (2017). What your Facebook profile picture reveals about your personality. Proceedings of the 25th ACM international conference on multimedia, 460–468. https://doi.org/10.1145/3123266.3123331
- Seligman, M. (2018). The Hoffman report, the Central Intelligence Agency, and the defense of the nation: A personal view. Health Psychology Open, 5. https://doi.org/10.1177/2055102918796192.
- Settanni, M., Azucar, D., & Marengo, D. (2018). Predicting individual characteristics from digital traces on social media: A meta-analysis. Cyberpsychology, Behavior and Social Networking, 21, 217–228. https://doi.org/10.1089/cyber.2017.0384.
- Sharma, L., Markon, K. E., & Clark, L. A. (2014). Toward a theory of distinct types of “impulsive” behaviors: A meta-analysis of self-report and behavioral measures. Psychological Bulletin, 140, 374–408. https://doi.org/10.1037/a0034418.
- Sherman, R. A., Nave, C. S., & Funder, D. C. (2010). Situational similarity and personality predict behavioral consistency. Journal of Personality and Social Psychology, 99, 330–343. https://doi.org/10.1037/a0019796.
- Shiramizu, V. K. M., Kozma, L., DeBruine, L. M., & Jones, B. C. (2019). Are dark triad cues really visible in faces? Personality and Individual Differences, 139, 214–216. https://doi.org/10.1016/j.paid.2018.11.011.
- Shobeiri, S., Laroche, M., & Mazaheri, E. (2013). Shaping e-retailer's website personality: The importance of experiential marketing. Journal of Retailing and Consumer Services, 20, 102–110. https://doi.org/10.1016/j.jretconser.2012.10.011.
10.1016/j.jretconser.2012.10.011 Google Scholar
- Shokri, R., Theodorakopoulos, G., Le Boudec, J.-Y., & Hubaux, J.-P. (2011). Quantifying location privacy. IEEE Symposium on Security and Privacy, 2011, 247–262. https://doi.org/10.1109/SP.2011.18.
- Soto, C. J. (2019). How replicable are links between personality traits and consequential life outcomes? The life outcomes of personality replication project. Psychological Science, 30, 711–727. https://doi.org/10.1177/0956797619831612.
- Staiano, J., Lepri, B., Aharony, N., Pianesi, F., Sebe, N., & Pentland, A. (2012). Friends don't lie: Inferring personality traits from social network structure. Proceedings of the 2012 ACM Conference on Ubiquitous Computing, 321–330. https://doi.org/10.1145/2370216.2370266
- Stanton, A. M., Boyd, R. L., Pulverman, C. S., & Meston, C. M. (2015). Determining women's sexual self-schemas through advanced computerized text analysis. Child Abuse and Neglect, 46. https://doi.org/10.1016/j.chiabu.2015.06.003.
- Stanton, A. M., Meston, C. M., & Boyd, R. L. (2017). Sexual self-schemas in the real world: Investigating the ecological validity of language-based markers of childhood sexual abuse. Cyberpsychology, Behavior and Social Networking, 20. https://doi.org/10.1089/cyber.2016.0657.
- Stella, M., Ferrara, E., & Domenico, M. D. (2018). Bots increase exposure to negative and inflammatory content in online social systems. Proceedings of the National Academy of Sciences, 201803470. https://doi.org/10.1073/pnas.1803470115.
- Strong, E. K. (1934). The vocational interest test. The Vocational Guidance Magazine, 12, 49–56. https://doi.org/10.1002/j.2164-5884.1934.tb00594.x.
10.1002/j.2164-5884.1934.tb00594.x Google Scholar
- Strong, E. K. (1935). Predictive value of the vocational interest test. Journal of Educational Psychology, 26, 331–349. https://doi.org/10.1037/h0062498.
10.1037/h0062498 Google Scholar
- Sugiura, L., Wiles, R., & Pope, C. (2017). Ethical challenges in online research: Public/private perceptions. Research Ethics, 13, 184–199. https://doi.org/10.1177/1747016116650720.
10.1177/1747016116650720 Google Scholar
- Sun, J., Schwartz, H. A., Son, Y., Kern, M. L., & Vazire, S. (2019). The language of well-being: Tracking fluctuations in emotion experience through everyday speech. Journal of Personality and Social Psychology, 118, 364–387. https://doi.org/10.1037/pspp0000244.
- Sweeney, L. (2005). Privacy technologies for homeland security. Statement before the Privacy and Integrity Advisory Committee of the Department of Homeland Security.
- Tellakat, M., Boyd, R. L., & Pennebaker, J. W. (2019). How do online learners study? The psychometrics of students' clicking patterns in online courses. PLoS ONE, 14, 1–17. https://doi.org/10.1371/journal.pone.0213863.
- Thompson, S. A., & Warzel, C. (2019). Twelve million phones, one dataset, zero privacy. The New York Times. https://www.nytimes.com/interactive/2019/12/19/opinion/location-tracking-cell-phone.html
- Thorstad, R., & Wolff, P. (2019). Predicting future mental illness from social media: A big-data approach. Behavior Research Methods, 51, 1586–1600. https://doi.org/10.3758/s13428-019-01235-z.
- Tkaczynski, P. J., Ross, C., MacLarnon, A., Mouna, M., Majolo, B., & Lehmann, J. (2018). Measuring personality in the field: An in situ comparison of personality quantification methods in wild Barbary macaques (Macaca sylvanus). Journal of Comparative Psychology, No Pagination Specified-No Pagination Specified.. https://doi.org/10.1037/com0000163.
- Todorov, A., Mandisodza, A. N., Goren, A., & Hall, C. C. (2005). Inferences of competence from faces predict election outcomes. Science (New York, N.Y.), 308, 1623–1626. https://doi.org/10.1126/science.1110589.
- Torra, V. (2017). Data privacy: Foundations, new developments and the big data challenge. Springer.
10.1007/978-3-319-57358-8 Google Scholar
- Tufekci, Z. (2019). Think you're discreet online? Think again. The New York Times. https://www.nytimes.com/2019/04/21/opinion/computational-inference.html
- Tunick, M. (2014). Balancing privacy and free speech: Unwanted attention in the age of social media ( 1st ed.). Routledge.
10.4324/9781315763132 Google Scholar
- Turkyilmaz, C. A., Erdem, S., & Uslu, A. (2015). The effects of personality traits and website quality on online impulse buying. Procedia - Social and Behavioral Sciences, 175, 98–105. https://doi.org/10.1016/j.sbspro.2015.01.1179.
10.1016/j.sbspro.2015.01.1179 Google Scholar
- R. R. Vallacher, S. J. Read, & A. Nowak (Eds) (2017). Computational social psychology. Routledge.
10.4324/9781315173726 Google Scholar
- Walker, M., Schönborn, S., Greifeneder, R., & Vetter, T. (2018). The Basel Face Database: A validated set of photographs reflecting systematic differences in big two and big five personality dimensions. PLoS ONE, 13.
- Warren, S. D., & Brandeis, L. D. (1890). The right to privacy. Harvard Law Review, 4, 193–220. JSTOR. https://doi.org/10.2307/1321160.
- Watson, J. B. (1913). Psychology as the behaviorist views it. Psychological Review, 20, 158–177. https://doi.org/10.1037/h0074428.
- Wilson, G. D. (1990). Personality, time of day and arousal. Personality and Individual Differences, 11, 153–168. https://doi.org/10.1016/0191-8869(90)90008-F.
- Wilt, J., Noftle, E. E., Fleeson, W., & Spain, J. S. (2011). The dynamic role of personality states in mediating the relationship between extraversion and positive affect. Journal of Personality, 80, 1205–1236. https://doi.org/10.1111/j.1467-6494.2011.00756.x.
- Wojcik, S. P., Hovasapian, A., Graham, J., Motyl, M., & Ditto, P. H. (2015). Conservatives report, but liberals display, greater happiness. Science, 347, 1243–1246. https://doi.org/10.1126/science.1260817.
- Wrzus, C., & Mehl, M. R. (2015). Lab and/or field? Measuring personality processes and their social consequences. European Journal of Personality, 29, 250–271. https://doi.org/10.1002/per.1986.
- Wu, T. (2019 w capitalism betrayed privacy. The New York Times. https://www.nytimes.com/2019/04/10/opinion/sunday/privacy-capitalism.html
- Yin, S., & Kaynak, O. (2015). Big data for modern industry: Challenges and trends. Proceedings of the IEEE, 103, 143–146. https://doi.org/10.1109/JPROC.2015.2388958.
- Youyou, W., Kosinski, M., & Stillwell, D. (2015). Computer-based personality judgments are more accurate than those made by humans. Proceedings of the National Academy of Sciences, 112, 1036–1040. https://doi.org/10.1073/pnas.1418680112.
- Yuan, Y. (2020). YOLO creator Joseph Redmon stopped CV research due to ethical concerns. Medium. https://medium.com/syncedreview/yolo-creator-says-he-stopped-cv-research-due-to-ethical-concerns-b55a291ebb29
- Zimbardo, P. G. (1973). On the ethics of intervention in human psychological research: With special reference to the Stanford prison experiment. Cognition, 2, 243–256. https://doi.org/10.1016/0010-0277(72)90014-5.
- Zimmer, M. (2018). Addressing conceptual gaps in big data research ethics: An application of contextual integrity. Social Media + Society, 4(), 2056305118768300. https://doi.org/10.1177/2056305118768300.
Citing Literature
Special Issue:Behavioral personality science in the age of big data
September/October 2020
Pages 599-612