Searching for Prosociality in Qualitative Data: Comparing Manual, Closed-Vocabulary, and Open-Vocabulary Methods
Corresponding Author
William H.B. McAuliffe
Department of Psychology, University of Miami, Miami, FL, USA
Department of Health Care Policy, Harvard Medical School, Harvard University, Boston, MA, USA
Correspondence to: William H. B. McAuliffe, Department of Health Care Policy, Harvard Medical School, 180 Longwood Avenue, Boston, MA, USA 02115-5899.
E-mail: williamhbmcauliffe@gmail.com
Search for more papers by this authorHannah Moshontz
Department of Psychology and Neuroscience, Duke University, Durham, NC, USA
Search for more papers by this authorThomas G. McCauley
Department of Psychology, University of Miami, Miami, FL, USA
Department of Psychology, University of California, San Diego, La Jolla, CA, USA
Search for more papers by this authorMichael E. McCullough
Department of Psychology, University of Miami, Miami, FL, USA
Department of Psychology, University of California, San Diego, La Jolla, CA, USA
Search for more papers by this authorCorresponding Author
William H.B. McAuliffe
Department of Psychology, University of Miami, Miami, FL, USA
Department of Health Care Policy, Harvard Medical School, Harvard University, Boston, MA, USA
Correspondence to: William H. B. McAuliffe, Department of Health Care Policy, Harvard Medical School, 180 Longwood Avenue, Boston, MA, USA 02115-5899.
E-mail: williamhbmcauliffe@gmail.com
Search for more papers by this authorHannah Moshontz
Department of Psychology and Neuroscience, Duke University, Durham, NC, USA
Search for more papers by this authorThomas G. McCauley
Department of Psychology, University of Miami, Miami, FL, USA
Department of Psychology, University of California, San Diego, La Jolla, CA, USA
Search for more papers by this authorMichael E. McCullough
Department of Psychology, University of Miami, Miami, FL, USA
Department of Psychology, University of California, San Diego, La Jolla, CA, USA
Search for more papers by this authorAbstract
Although most people present themselves as possessing prosocial traits, people differ in the extent to which they actually act prosocially in everyday life. Qualitative data that were not ostensibly collected to measure prosociality might contain information about prosocial dispositions that is not distorted by self-presentation concerns. This paper seeks to characterise charitable donors from qualitative data. We compared a manual approach of extracting predictors from participants' self-described personal strivings to two automated approaches: A summation of words predefined as prosocial and a support vector machine classifier. Although variables extracted by the support vector machine predicted donation behaviour well in the training sample (N = 984), virtually, no variables from any method significantly predicted donations in a holdout sample (N = 496). Raters' attempts to predict donations to charity based on reading participants' personal strivings were also unsuccessful. However, raters' predictions were associated with past charitable involvement. In sum, predictors derived from personal strivings did not robustly explain variation in charitable behaviour, but personal strivings may nevertheless contain some information about trait prosociality. The sparseness of personal strivings data, rather than the irrelevance of open-ended text or individual differences in goal pursuit, likely explains their limited value in predicting prosocial behaviour. © 2020 European Association of Personality Psychology
Open Research
Open Research Badges
This article earned Open Date, Open Materials and Preregistered + Analysis Plan badges through Open Practices Disclosure from the Center for Open Science: https://osf.io/tvyxz/wiki. The data and materials are permanently and openly accessible at https://osf.io/yhxgf/, https://osf.io/f3ckm/ and https://osf.io/b254j. Author's disclosure form may also be found at the Supporting Information in the online version.
Supporting Information
| Filename | Description |
|---|---|
| per2240-sup-0001-Open_Practices_Disclosure_Form.pdfPDF document, 294.5 KB |
Supporting info item |
Please note: The publisher is not responsible for the content or functionality of any supporting information supplied by the authors. Any queries (other than missing content) should be directed to the corresponding author for the article.
REFERENCES
- Andreoni, J., Rao, J. M., & Trachtman, H. (2017). Avoiding the ask: A field experiment on altruism, empathy, and charitable giving. Journal of Political Economy, 125, 625–653.
- Azen, R., & Budescu, D. V. (2003). The dominance analysis approach for comparing predictors in multiple regression. Psychological Methods, 8, 129–148. https://doi.org/10.1037/1082-989x.8.2.129.
- Azen, R., & Traxel, N. (2009). Using dominance analysis to determine predictor importance in logistic regression. Journal of Educational and Behavioral Statistics, 34, 319–347.
- Bainter, S., McCauley, T. G., Wager, T., & Losin, E. A. R.. (in press). Improving practices for selecting a subset of important predictors in psychology: An application to predicting pain. Advances in Methods and Practices in Psychological Science.. https://doi.org/10.31234/osf.io/j8t7s.
- Banks, G. C., Woznyj, H. M., Wesslen, R. S., & Ross, R. L. (2018). A review of best practice recommendations for text analysis in R (and a user-friendly app). Journal of Business and Psychology, 33, 445–459.
- Barclay, P., & Willer, R. (2006). Partner choice creates competitive altruism in humans. Proceedings of the Royal Society B: Biological Sciences, 274, 749–753.
- Batson, C. D., & Shaw, L. L. (1991). Evidence for altruism: Toward a pluralism of prosocial motives. Psychological Inquiry, 2, 107–122.
10.1207/s15327965pli0202_1 Google Scholar
- Baumard, N., André, J. B., & Sperber, D. (2013). A mutualistic approach to morality: The evolution of fairness by partner choice. Behavioral and Brain Sciences, 36, 59–78.
- Bekkers, R., & Wiepking, P. (2011). Accuracy of self-reports on donations to charitable organizations. Quality & Quantity, 45, 1369–1383.
- Bleidorn, W., & Hopwood, C. J. (2019). Using machine learning to advance personality assessment and theory. Personality and Social Psychology Review, 23, 190–203. https://doi.org/10.1177/1088868318772990.
- Bustos, C. & Soares, F.C. (2019) The dominance analysis package. https://CRAN.R-project.org/package=dominanceanalysis
- Colby, A., & Damon, W. (1992). Some do care. Free press.
- Damon, W., & Colby, A. (2015). The power of ideals: The real story of moral choice. USA: Oxford University Press.
- Deci, E. L., & Ryan, R. M. (2000). The "what" and" why" of goal pursuits: Human needs and the self-determination of behavior. Psychological Inquiry, 11, 227–268.
- Dehghani, M., Johnson, K. M., Garten, J., Boghrati, R., Hoover, J., Balasubramanian, V., … Parmar, N. J. (2017). TACIT: An open-source text analysis, crawling, and interpretation tool. Behavior Research Methods, 49, 538–547. https://doi.org/10.3758/s13428-016-0722-4.
- Dunlop, W. L. (2015). Contextualized personality, beyond traits. European Journal of Personality, 29, 310–325.
- Emmons, R. A. (1999). The psychology of ultimate concerns: Motivation and spirituality in personality. New York, NY: Guilford Press.
- Epstein, S., & O'Brien, E. J. (1985). The person–situation debate in historical and current perspective. Psychological Bulletin, 98, 513–537.
- Faul, F., Erdfelder, E., Buchner, A., & Lang, A. G. (2009). Statistical power analyses using G* Power 3.1: Tests for correlation and regression analyses. Behavior Research Methods, 41, 1149–1160.
- Firmin, R. L., Bonfils, K. A., Luther, L., Minor, K. S., & Salyers, M. P. (2017). Using text-analysis computer software and thematic analysis on the same qualitative data: A case example. Qualitative Psychology, 4, 201.
10.1037/qup0000050 Google Scholar
- Frimer, J. A., Aquino, K., Gebauer, J. E., Zhu, L. L., & Oakes, H. (2015). A decline in prosocial language helps explain public disapproval of the US Congress. Proceedings of the National Academy of Sciences, 112, 6591–6594.
- Frimer, J. A., Schaefer, N. K., & Oakes, H. (2014). Moral actor, selfish agent. Journal of Personality and Social Psychology, 106, 790–802.
- Frimer, J. A., Walker, L. J., & Dunlop, W. L. (2009). Values embedded in Narrative (VEiN) coding manual. Department of Psychology, University of British Columbia, Vancouver, Canada: Unpublished manuscript.
- Frimer, J. A., Walker, L. J., Dunlop, W. L., Lee, B. H., & Riches, A. (2011). The integration of agency and communion in moral personality: Evidence of enlightened self-interest. Journal of Personality and Social Psychology, 101, 149–163. https://doi.org/10.1037/a0023780.
- George, E. I., & McCulloch, R. E. (1993). Variable selection via Gibbs sampling. Journal of the American Statistical Association, 88, 881–889.
- Goodman, F. R., Kashdan, T. B., Stiksma, M. C., & Blalock, D. V. (2019). Personal strivings to understand anxiety disorders: Social anxiety as an exemplar. Clinical Psychological Science, 7, 283–301.
- Goodwin, G. P., Piazza, J., & Rozin, P. (2014). Moral character predominates in person perception and evaluation. Journal of Personality and Social Psychology, 106, 148–168. https://doi.org/10.1037/a0034726.
- Graziano, W. G., & Eisenberg, N. (1997). Agreeableness: A dimension of personality. In Handbook of personality psychology (pp. 795–824). Academic Press.
10.1016/B978-012134645-4/50031-7 Google Scholar
- Hart, H. M., McAdams, D. P., Hirsch, B. J., & Bauer, J. J. (2001). Generativity and social involvement among African Americans and White adults. Journal of Research in Personality, 35, 208–230.
- Koo, T. K., & Li, M. Y. (2016). A guideline of selecting and reporting intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine, 15, 155–163. https://doi.org/10.1016/j.jcm.2016.02.012.
- Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 159–174.
- List, J. A. (2011). The market for charitable giving. Journal of Economic Perspectives, 25, 157–180.
- Magee, J. C., & Langner, C. A. (2008). How personalized and socialized power motivation facilitate antisocial and prosocial decision-making. Journal of Research in Personality, 42, 1547–1559.
- Mansfield, E. D., & McAdams, D. P. (1996). Generativity and themes of agency and communion in adult autobiography. Personality and Social Psychology Bulletin, 22, 721–731.
- Marsh, A. A., Stoycos, S. A., Brethel-Haurwitz, K. M., Robinson, P., VanMeter, J. W., & Cardinale, E. M. (2014). Neural and cognitive characteristics of extraordinary altruists. Proceedings of the National Academy of Sciences, 111, 15036–15041.
- McAdams, D. P. (1995). What do we know when we know a person? Journal of Personality, 63, 365–396.
- McAuliffe, W. H. B. (2019). Can studies of trait altruism be trusted? Open Access Dissertations., 2252. https://scholarlyrepository.miami.edu/oa_dissertations/2252.
- McCauley, T.G., and McCullough, M.E. (2019). Determined donors: An experimental investigation of moral excellence. Working paper.
- Oliner, S. P., & Oliner, P. M. (1988). The altruistic personality. New York: Free Press.
10.1177/002071528802900317 Google Scholar
- Park, G., Schwartz, H. A., Eichstaedt, J. C., Kern, M. L., Kosinski, M., Stillwell, D. J., … Seligman, M. E. (2015). Automatic personality assessment through social media language. Journal of Personality and Social Psychology, 108, 934–952.
- Paulhus, D. L., & John, O. P. (1998). Egoistic and moralistic biases in self-perception: The interplay of self-deceptive styles with basic traits and motives. Journal of Personality, 66, 1025–1060.
- Pennebaker, J. W., Booth, R. J., & Francis, M. E. (2007). Linguistic inquiry and word count: LIWC [computer software]. Austin, TX: LIWC.net.
- Preacher, K. J., Rucker, D. D., MacCallum, R. C., & Nicewander, W. A. (2005). Use of the extreme groups approach: a critical reexa mination and new recommendations. Psychological Methods, 10(2), 178-192.
- Prospect Research and Wealth Screening Statistics (2015, January 7). Retrieved from https://www.donorsearch.net/prospect-research-statistics/
- R Core Team (2019). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. Retrieved from https://www.r-project.org.
- Rand, D. G., & Epstein, Z. G. (2014). Risking your life without a second thought: Intuitive decision-making and extreme altruism. PLoS ONE, 9, e109687. https://doi.org/10.1371/journal.pone.0109687.
- Rushton, J. P., Chrisjohn, R. D., & Fekken, G. C. (1981). The altruistic personality and the self-report altruism scale. Personality and Individual Differences, 2(4), 293-302.
- Schönbrodt, F. D., & Perugini, M. (2013). At what sample size do correlations stabilize? Journal of Research in Personality, 47, 609–612.
- Schwartz, S. H. (1994). Are there universal aspects in the structure and contents of human values? Journal of Social Issues, 50, 19–45.
- Schwartz, S. H. (2010). Basic values: How they motivate and inhibit prosocial behavior. Prosocial motives, emotions, and behavior: The better angels of our nature, 14, 221–241.
10.1037/12061-012 Google Scholar
- Seeboth, A., & Mõttus, R. (2018). Successful explanations start with accurate descriptions: Questionnaire items as personality markers for more accurate predictions. European Journal of Personality, 32(3), 186-201.
- Sikkel, D., & Schoenmakers, E. (2012). Bequests to health-related charitable organisations: A structural model. International Journal of Nonprofit and Voluntary Sector Marketing, 17, 183–197.
10.1002/nvsm.1421 Google Scholar
- Sparks, A., Burleigh, T., & Barclay, P. (2016). We can see inside: Accurate prediction of Prisoner's Dilemma decisions in announced games following a face-to-face interaction. Evolution and Human Behavior, 37, 210–216.
- Sun, J., & Vazire, S. (2019). Do people know what they're like in the moment? Psychological Science, 30, 405–414. https://doi.org/10.1177/0956797618818476.
- Thielmann, I., Spadaro, G., & Balliet, D. (2020). Personality and prosocial behavior: A theoretical framework and meta-analysis. Psychological Bulletin, 146, 30–90.
- Vazire, S. (2010). Who knows what about a person? The self–other knowledge asymmetry (SOKA) model. Journal of Personality and Social Psychology, 98, 281–300. https://doi.org/10.1037/a0017908.
- Walker, L. J., & Frimer, J. A. (2007). Moral personality of brave and caring exemplars. Journal of Personality and Social Psychology, 93, 845–860. https://doi.org/10.1037/0022-3514.93.5.845.
- Yarkoni, T., & Westfall, J. (2017). Choosing prediction over explanation in psychology: Lessons from machine learning. Perspectives on Psychological Science, 12, 1100–1122.
- Zhao, K., & Smillie, L. D. (2015). The role of interpersonal traits in social decision making: Exploring sources of behavioral heterogeneity in economic games. Personality and Social Psychology Review, 19, 277–302.
Citing Literature
Special Issue:Behavioral personality science in the age of big data
September/October 2020
Pages 903-916
