Eric Geelhoed (2020) discusses the potential use of the online community for qualitative and quantitive methods of research. I have participated as a play tester in the following forms of user and audience research:
- OpenDev call for HumanKind – Played through three different setup examples of gameplay including combat, settling a city and resource and research activity and then answering a survey based on my experience
- Early Access games and feedback for Thea: The Awakening and Starbound
- General feedback like Steam reviews
“It is difficult to make something simple and it takes time to make something short” (Geelhoed 2020).
Ethics and integrity, informed consent and carrying out play testing and user feedback
Alcwyn Parker (2020) references the Milgram Experiment (Milgram 1961) and Stanford Prison Experiment (Zimbardo 1973) to start the conversation of ‘ethically flawed’ and ‘good research’. While both infamous experiments are unsettling they have both heavily contributed scientifically to research into human behaviour. I am familiar with both experiments prior to the course, and on first hearing about these experiments I was disturbed. The Milgram experiment is about obedience in the face of authority, and I am still abhorred that anyone would continue to electrocute someone because someone in a white coat asked them to. Likewise, the Stanford Prison Experiment looked into the abuse of power, and the old adage ‘Absolute Power Corrupts Absolutely’. The Milgram Experiment and Stanford Prison Experiment left lasting psychological damage on participants and in some cases the creators of the studies themselves (Falmouth University 2020). The Milgram Experiment (Milgram 1961) and Stanford Prison Experiment (Zimbardo 1973) almost serve as a ‘what not to do’ in ethical research.
‘The Wave’ (1967) is another experiment that comes to mind, on the topic of what is good research and what is ethically abhorrent. ‘The Wave’ was conducted by a high school teacher, not a scientist and what followed was not part of an official experiment but a scenario that got out of hand. The results just like The Milgram Experiment and Stanford Prison Experiment tells us a lot about human beings, our nature and how the rise of fascist regimes like the Nazi Party arise. History Teacher Ron Jones issued slogans, uniforms and recruited secret police, convincing pupils of the existence of a new national party – the experiment took on a life of its own, and it’s hard to separate fact from fiction in this famous event.
Play testing and player feedback
I would like to follow the model of early access games, where you iterate based on feedback until your game is ready for official release. I have played many early access games, as mentioned before and I admire the creators of those games. Now that I have complete the materials on Week 8 Research, I will hopefully leave no playtesting participants forever traumatised by my model of research and feedback.
Playtesting, ‘simple’ feedback and research methods have strict ethics and integrity rules that aren’t as simple as you might think. In all honesty, I had not considered the complexities involved in playtesting. Getting consent, considering your content, and assessing the risk involved is paramount to avoid any potential harm.
Before carrying out any play testing/user feedback, consider the following:
Qualitative methods:
- Individual interview
- structured, semi-structured
- Cognitive walkthrough
- Group interview
- Focus group
- Asking participants to keep a diary
- Observations
- Describing what (and why) participants do something
- Self-reflections by (you the) researcher (Falmouth University 2020)
Quantitative Methods:
- Questionnaires
- Physiological measurements
- Technology logs / Automated logging of the use of an application
- Observations
- Describing not just ‘what’ but also:
- When, how long and how often participants do something
- Psycho-physical testing (Falmouth University 2020)
Consent
- Informed consent – explanations of. the procedures to be followed, description of risks and benefits, considering alternative procedures for participants, offer your time to answer any questions, explain they can leave at any time and stop at any time.
- Ethical dilemma’s, considering the well-known ideas of ethical wrong-doing’s like coercion, deception and the invasion of privacy.
- Vulnerable Participants: Children and young people, Participants who can’t give informed consent, Significant cultural differences
- Privacy and the participants right to privacy. How to protect their privacy. Their privacy can be protected at any time, you hold no right to their information. Be careful where you record information and that no one’s conversations are being recorded and published.
- Deception, how to achieve interesting results without deceiving your participants or acting unethically. Essentially, deception must be justified by necessity and worldly good. (Falmouth University 2020)
Challenge Activity
Taking what we learnt from Research and Ethics this week, we were asked to assess the risk of the following scenarios:
Scenario 1
Scenario 1 A researcher plans to interview eight artists/curators/designers for her thesis. She offers a letter of introduction about the project, gains are written informed consent for the interview from each interviewee, later checks the contents of the transcription with each interviewee, allows the interviewee to withdraw comments / approve the interview record. The interviews will be used as attributed statements within the thesis. A recognised approach from oral history/social sciences/ethnography/art and design criticism and history is part of the methodology. The interviews will involve travel in the UK and abroad, the researcher has discussed her travel plans and personal safety with her supervisors. (Falmouth University 2020)
Medium-risk
- Research involves individuals
- Research involving access to records of personal or confidential information concerning identifiable individuals
- Potentially research involves the interaction with individuals or communities where different cultural perceptions of ethics might result in misunderstanding
The project sounds interesting and the researcher has considered how to protect her participant’s privacy and wellbeing. The research involves interaction with individuals who may be a part of a community or culture and misunderstanding’s might occur.
Scenario 2
Scenario 2 A researcher plans to interview around 30 producers of legitimate graffiti at the Southbank Undercroft. Participants were to be interviewed about their opinions and ideas regarding activities and future possibilities for the Undercroft, and also where relevant, their own graffiti habits and key trends in graffiti practices. (Falmouth University 2020)
Medium-risk to High Risk
- Research involving individuals or groups
- Research involving access to records of personal or confidential information concerning identifiable individuals
- Potentially involving the interaction with individuals or communities where different cultural perceptions of ethics might result in misunderstanding
Initially, I marked this as ‘medium risk’. The research again sounds interesting but the main risk factor for me is that the individuals will be identifiable. Someone may wish those individuals harm based on their beliefs and the community they belong to and the research has identified them (although unlikely). They also might reveal something on film that they later regret or causes them distress.
If the research takes a similar approach to Scenario 1, whereby they seek to protect the participants by checking the ‘contents’ of the interviews with the persons involved, allowing them to withdraw comments and/or from the project completely and approve the final interview then this would lessen the risk’s involved. I think it would also be important to stay focused on intellectual topics and not delve into personal and sensitive information with the participants of the study.
Scenario 3
Scenario 3 The research, for a practice-based PhD, involves engaging online presences in social networking sites under a pseudonym. It aims to explore the ways in which identity is constructed online. The research is such that it cannot be revealed in advance to those involved. The core of the research involves developing a community of online presences into a community of offline friends. (Falmouth University 2020)
High-risk
- Involves accessing and/or storing and/or disseminating material which may be regarded as unlawful
- Might induce psychological stress, anxiety or humiliation
- Might cause participants to reveal Information which causes concern to them at that time or later
Deception is rather at the heart of this scenario. Feigning friendship and essentially ‘cat-fishing’ individuals to form a community of offline friends doesn’t seem to fill the ‘end justifies the means’ criteria to me (if anything ever does).
Reflection
Week 8 Research has made me consider the moral implications of research and study. There were a few ill-thought out experiments as examples but it is surprisingly easy to step on moral boundaries in a proposal, without realising it.
‘The Wave’ is a perfect example of how a seemingly fun and informative lesson plan got out of a hand – how hard would it be for this to be repeated in a study proposal for research and feedback on games? If it did get out of hand would the results benefit the world at large, does the end justify the means?
To continue to reflect on ethics, integrity and the value of research futher reading materials were suggested by member’s of our cohort:
- The Ethics Toolkit
- Projects by If
- Individual Reflection
References
BJÖRK, Staffan. 2008. ‘Games, Gamers, and Gaming: Understanding Game Research’. In Proceedings of the 12th International Conference on Entertainment and Media in the Ubiquitous Era. 64–68. Available at: https://doi.org/10.1145/1457199.1457213 [accessed 12 November 2020].
CZARNOTA, Jedrzej. 2014. ‘Harnessing Your Players as a QA Resource: Benefits and Practice’ [online lecture]. GDC Vault. Available at: https://www.gdcvault.com/play/1020523/Harnessing-Your-Players-as-a [accessed 12 November 2020].
‘Ethics Kit | Methods & Tools for Ethics in the Design Process’. 2020. [online]. Available at: http://ethicskit.org/tools.html [accessed 12 November 2020].
Falmouth University. 2020. [online]. Available at: https://flex.falmouth.ac.uk/courses/872 [accessed 12 November 2020].
GEELHOED, Eric. 2020. ‘User and Audience research: An introduction‘ [online lecture].Falmouth University. Available at: https://flex.falmouth.ac.uk/courses/872/pages/week-8-user-and-audience-research?module_item_id=46492 [accessed 12 November 2020].
GUTTERMAN, Adam. 2017. ‘Best Practices for Launching Successfully on Google Play Panel (Presented by Google)’ [online lecture]. GDC Vault. Available at: https://www.gdcvault.com/play/1024374/Best-Practices-for-Launching-Successfully [accessed 12 November 2020].
HODENT, Celia, Kate EDWARDS, Carlos FIGUEIREDO and Kat LO. 2020. ‘Addressing Ethics & Content Responsibility as Game Developers’ [online lecture]. GDC Vault. Available at: https://www.gdcvault.com/play/1026774/Addressing-Ethics-Content-Responsibility-as [accessed 12 November 2020].
Humankind. 2020. Amplitude studios.
‘IF’. 2020. [online]. Available at: https://www.projectsbyif.com [accessed 12 November 2020].
‘Individual Reflection | Ethics Kit’. 2020. [online]. Available at: http://ethicskit.org/individual-reflection.html [accessed 12 November 2020].
MILGRAM, Stanley. 1963. ‘Behavioral Study of Obedience.’ The Journal of Abnormal and Social Psychology 67(4), 371–8.
PARKER, Alcwyn. 2020. ‘Integrity, Ethics and Policy’ [online lecture]. Falmouth University. Available at: https://flex.falmouth.ac.uk/courses/872/pages/week-8-integrity-ethics-and-policy?module_item_id=46493 [accessed 12 November 2020].
Starbound. 2016. Chucklefish.
Thea: The Awakening. 2015. MuHa Games.
The Wave. Available at: https://www.thewavehome.com/ [accessed 12 November 2020].
ZIMBARDO, Philip G. 1973. ‘On the Ethics of Intervention in Human Psychological Research: With Special Reference to the Stanford Prison Experiment’. Cognition 2(2), 243–56.
