Email this article An experimental Critical Multimodal Discourse Study to the AI-driven sentiment analysis of online crisis communication
Abstract
In response to the challenges of crisis management and communication in business digital scenarios (Umar et al. 2022; Catenaccio 2021), this case study presents an example of the evolution and implications of online crisis communication and discursive practices that combine human input and AI for the management of crisis events and trust repair.
Reliability of communication in web-based business scenarios cannot be easily achieved because of the huge amount of data. In addition, reputation management and trust are constantly put under threat by misinformation and misunderstanding (Garzone, Giordano 2018). With a view to countering these risks, companies are increasingly outsourcing digital marketing services (Palttala et al. 2012) based on AI methods to analyse consumers’ online needs and behaviour (Schwaiger et al. 2021), even though research has recently raised some concerns on the over-reliance on AI-based tools (Tam, Kim 2019).
We intend to show how multimodal critical discourse studies (Djonov, Zhao 2014) can help identify and understand the potentials and limits of social media listening tools that are designed for crisis prediction and management (van Zoonen, van der Meer 2015). To do so, we present a case study that we engaged with during our research traineeship at Digital Trails, a B2B company dealing with online visibility, digital marketing and reputation analysis.
In reporting the outcomes of online reputation analysis to gauge possible damages after a crisis event, we present the comparison between the AI-driven sentiment analysis (conducted via Meltwater) and the human-based revision and fine-tuning of AI-driven sentiment analysis. Our aim is to discuss the potentials and the criticalities of AI-driven sentiment detection. Among the latter, we highlight the unrecognizability of languages other than English, and the flaw in interpreting pragmatic aspects, as well as multimodal digital artefacts. Consistent with our findings, we argue that AI models, based on a unimodal and decontextualized architecture still require human validation. We conclude by indicating research directions for the detection of sentiment polarity, which include higher collaboration between IT developers and multimodal discourse analysts so that multimodally-informed models can assist crisis communication and management more efficiently.
References
Al-Ghamdi L. M. 2021, Towards adopting AI techniques for monitoring social media activities, in “Sustainable Engineering and Innovation” 3 [1], pp. 15-22.
Androutsopoulos J. 2013, Participatory culture and metalinguistic discourse: Performing and negotiating German dialects on YouTube, in Tanner D. and Trester A. M. (eds.), Discourse 2.0: Language and New Media, Georgetown University Press, Washington, pp. 47-71.
Bateman J. A. et al. 2017, Multimodality: Foundations, research and analysis: A problem-oriented introduction, De Gruyter Mouton, Berlin/ Boston.
Bukar U. A. et al. 2022, How Advanced Technological Approaches Are Reshaping Sustainable Social Media Crisis Management and Communication: A Systematic Review, in “Sustainability” 14 [10], pp. 5854.
Buzoianu C. and Bîră M. 2021, Using Social Media Listening in Crisis Communication and Management: New Methods and Practices for Looking into Crises, in “Sustainability” 13, pp. 13015. https://doi.org/10.3390/su132313015
Chan S. and Fyshe V. 2018, Social and Emotional Correlates of Capitalization on Twitter, in Proceedings of the Second Workshop on Computational Modeling of People’s Opinions, Personality, and Emotions in Social Media, Association for Computational Linguistics, pp. 10-15. Doi:10.18653/v1/W18-1102
Collins Dictionary, https://www.collinsdictionary.com/dictionary/english/too-hot-to-handle (10.06.2023).
Coombs T. 2014, State of Crisis Communication: Evidence and the Bleeding Edge, in Institute for public relations. https://instituteforpr.org/state-crisis-communication-evidence-bleeding-edge/#:~:text=in%20crisis%20communication.-,Social%20media%20is%20the%20driving%20force%20in%20the%20bleeding%20edge,is%20amplified%20by%20social%20media (10.06.2023).
Coombs T. and Holladay J. S. 2012, The paracrisis: The challenges created by publicly managing crisis prevention, in “Public Relations Review” 38, pp. 408-415.
Das R. and Singh T. D. 2023, Multimodal Sentiment Analysis: A survey of methods, trends, and challenges, in “ACM Computer Survey” 55 [13], pp. 1-38. https://doi.org/10.1145/3586075
Davidson T. et al. 2017, Automated hate speech detection and the problem of offensive language, in Proceedings of the 11th international AAAI conference on web and social media, https://arxiv.org/abs/1703.04009
Di Cristofaro M. 2023, Corpus Approaches to Language in Social Media, Routledge, London/ New York.
Dias Oliva T. et al. 2021, Fighting Hate Speech, Silencing Drag Queens? Artificial Intelligence in Content Moderation and Risks to LGBTQ Voices Online, in “Sexuality & Culture” 25, pp. 700-732.
Diers-Lawson A. 2019, Crisis Communication: Managing Stakeholder Relationships, Routledge, Oxford.
Djonov E. and van Leeuwen T. 2012, Normativity and software: A multimodal social semiotic approach, in Norris S. (ed.), Multimodality and practice: Investigating theory in practice through method, Routledge, New York, pp. 119-137.
Djonov E. and Zhao S. 2018, Social Semiotics. A Theorist and a Theory in Retrospect and Prospect, in Zhao S. et al. (eds.), Advancing Multimodal and Critical Discourse Studies. Interdisciplinary Research inspired by Theo van Leeuwen’s Social Semiotics, Routledge, New York, pp. 1-18.
Djonov E. and Zhao S. 2014, From Multimodal to Critical Multimodal Studies through Popular Discourse, in Djonov E. and Zhao S. (eds.), Critical multimodal studies of popular discourse, Routledge, London/ New York, pp. 1-16.
Duarte N. et al. 2017, Mixed messages? The limits of automated social media content analysis, in Center for Democracy and Technology, https://cdt.org/insight/mixed-messagesthelimits-of-automated-social-media-content-analysis. (14.06.2023).
Fairclough N. 1992, Discourse and social change, Polity Press, Cambridge.
Foucault M. 1965, Madness and civilization, Pantheon, New York.
Gandhi A., et al. 2023, Multimodal Sentiment Analysis: A systematic review of history, datasets, multimodal fusion methods, applications, challenges and future directions, in “Information Fusion” 91, pp. 424-444. https://doi.org/10.1016/j.inffus.2022.09.025.
Garbade M. J. 2018, A simple introduction to natural language processing, in “Becominghuman. Artifical Intelligence magazine”, https://becominghuman.ai/a-simple-introduction-to-natural-language-processing-ea66a1747b32 (20.06.2023).
Garzone G. E. and Catenaccio P. 2021, Ethics in professional and corporate discourse. Linguistic perspectives, Frank & Timme, Berlin.
Garzone G. E. and Giordano W. 2018, Discourse, Communication and the Enterprise: Developments and Issues. An Introduction, in Garzone G. E. and Giordano W. (eds.), Discourse, Communication and the Enterprise: where business meets discourse, Cambridge Scholars, Newcastle upon Tyne, pp. 1-17.
Hayes J. L. et al. 2021, Can Social Media Listening Platforms' Artificial Intelligence Be Trusted? Examining the Accuracy of Crimson Hexagon's (Now Brandwatch Consumer Research's) AI-Driven Analyses, in “Journal of advertising” 50 [1], pp. 81-91.
Jewitt C. (ed.) 2014, The Routledge Handbook of Multimodal Analysis, Routledge, London/ New York.
Joshi et al. 2017, Sentiment Resources: Lexicons and Datasets, in Cambria E. et al. (eds.), A Practical Guide to Sentiment Analysis, Springer, Cham, pp. 85-106.
Kiritchenko S. et al. 2016, Semeval-2016 task 7: Determining sentiment intensity of English and Arabic phrases, in Proceedings of the International Workshop on Semantic Evaluation, San Diego. Doi:10.18653/v1/S16-1004
Kotras B. 2020, Opinions that matter: The hybridization of opinion and reputation measurement in social media listening software, in “Media Culture and Society” 42 [7-8], pp. 1495-1511.
Kress G. 2010, Multimodality. A social semiotic approach to contemporary communication, Routledge, London/ New York.
Kress G. and van Leeuwen T. 2020, Reading images: The grammar of visual design, Routledge, London/ New York.
Lewandowsky S. et al. 2017, Beyond misinformation: understanding and coping with the ‘Post-Truth’ era, in “Journal of Applied Research in Memory and Cognition” 6 [4], pp. 353-369. https://doi.org/10.1016/j.jarmac.2017.07.008
Liu B. 2017, Many Facets of Sentiment Analysis, in Cambria E. et al. (eds.), A Practical Guide to Sentiment Analysis, Springer, Cham, pp. 11-40.
Liu B. 2012, Sentiment Analysis and Opinion Mining, in Synthesis Lectures on Human Language Technologies, Morgan & Claypool, Toronto. https://doi.org/10.2200/S00416ED1V01Y201204HLT016.
Liu B. et al. 2011, How publics respond to crisis communication strategies: The interplay of information form and source, in “Public Relations Review” 37, pp. 345-353.
Majumder N. et al. 2018, Multimodal sentiment analysis using hierarchical fusion with context modelling, in “Knowledge-Based Systems” 161, pp. 124-133.
Morgan A. and Wilk V. 2021, Social media users’ crisis response: A lexical exploration of social media content in an international sport crisis, in “Public Relations Review” 47 [4] https://doi.org/10.1016/j.pubrev.2021.102057.
Moschini I. and Sindoni M. G. (eds.) 2022, Mediation and Multimodal Meaning Making in Digital Environments, Routledge, London/ New York. https://doi.org/10.4324/9781003225423.
O'Halloran K. L. et al. 2014, Multimodal pragmatics, in Schneider K. and Barron A. (eds.), Pragmatics of Discourse, De Gruyter Mouton, Berlin, pp. 239-268.
Odin R. 2022, Spaces of Communication: Elements of Semio-Pragmatics, Amsterdam University Press, Amsterdam.
Palttala P. et al. 2012, Communication gaps in disaster management: Perceptions by experts from governmental and non-governmental organizations, in “Journal of Contingencies and Crisis Management” 20, pp. 2-12.
Perakakis E. et al. 2019, Social Media Monitoring: An Innovative Intelligent Approach, in “Designs” 3 [2], pp. 1-12.
Schwaiger J. et al. 2021, UR: SMART–A tool for analyzing social media content, in “Information Systems and e-Business Management” 19, pp. 1275-1320.
Sindoni M. G. 2013, Spoken and written discourse in online interactions: A multimodal approach, Routledge, London/ New York.
Tam L. and J.-N. Kim 2019, Social media analytics: How they support company public relations, in “Journal of Business Strategy” 40 [1], pp. 28-34.
Umar A. B. et al. 2022, How Advanced Technological Approaches Are Reshaping Sustainable Social Media Crisis Management and Communication: A Systematic Review, in “Sustainability” 14 [10], pp. 1-26.
van Noort G. and Willemsen L. 2012, Online damage control: the effects of proactive versus reactive webcare interventions in consumer-generated and brand generated platforms, in “Journal of Interactive Marketing” 26 [3], pp. 131-140.
van Zoonen W. and van der Meer T.V 2015, The importance of source and credibility perception in times of crisis: Crisis communication in a socially mediated era, in “Journal of Public Relations Research” 27 [5], pp. 371-388.
Ziora L. 2016, The sentiment analysis as a tool of business analytics in contemporary organizations, in “Studia Ekonomiczne” 281, pp. 234-241.
Full Text: PDF
Refbacks
- There are currently no refbacks.
