2024
Martinez, Kim; Checa, David; Bustillo, Andres
In: Electronics, vol. 13, iss. 281, no. 2, pp. 281, 2024.
Abstract | Links | BibTeX | Tags: game design, game engagement, game evaluation, serious games, Virtual Reality
@article{martinez2024,
title = {Development of the Engagement Playability and User eXperience (EPUX) Metric for 2D-Screen and VR Serious Games: A Case-Study Validation of Hellblade: Senua’s Sacrifice },
author = {Kim Martinez and David Checa and Andres Bustillo},
url = {https://doi.org/10.3390/electronics13020281},
doi = {electronics13020281},
year = {2024},
date = {2024-01-08},
urldate = {2024-01-08},
journal = {Electronics},
volume = {13},
number = {2},
issue = {281},
pages = {281},
abstract = {Research into the design of serious games still lacks metrics to evaluate engagement with the experience so that users can achieve the learning aims. This study presents the new EPUX metric, based on playability and User eXperience (UX) elements, to measure the capability of any serious game to maintain the attention of players. The metric includes (1) playability aspects: game items that affect the emotions of users and that constitute the different layers of the game, i.e., mechanics, dynamics and aesthetics; and (2) UX features: motivation, meaningful choices, usability, aesthetics and balance both in the short and in the long term. The metric is also adapted to evaluate virtual reality serious games (VR-SGs), so that changes may be considered to features linked to playability and UX. The case study for the assessment of the EPUX metric is Hellblade, developed in two versions: one for 2D-screens and the other for VR devices. The comparison of the EPUX metric scores for both versions showed that (1) some VR dynamics augmented the impact of gameplay and, in consequence, engagement capacity; and (2) some game design flaws were linked to much lower scores. Among those flaws were low numbers of levels, missions, and items; no tutorial to enhance usability; and lack of strategies and rewards to increase motivation in the long term.},
keywords = {game design, game engagement, game evaluation, serious games, Virtual Reality},
pubstate = {published},
tppubtype = {article}
}
Research into the design of serious games still lacks metrics to evaluate engagement with the experience so that users can achieve the learning aims. This study presents the new EPUX metric, based on playability and User eXperience (UX) elements, to measure the capability of any serious game to maintain the attention of players. The metric includes (1) playability aspects: game items that affect the emotions of users and that constitute the different layers of the game, i.e., mechanics, dynamics and aesthetics; and (2) UX features: motivation, meaningful choices, usability, aesthetics and balance both in the short and in the long term. The metric is also adapted to evaluate virtual reality serious games (VR-SGs), so that changes may be considered to features linked to playability and UX. The case study for the assessment of the EPUX metric is Hellblade, developed in two versions: one for 2D-screens and the other for VR devices. The comparison of the EPUX metric scores for both versions showed that (1) some VR dynamics augmented the impact of gameplay and, in consequence, engagement capacity; and (2) some game design flaws were linked to much lower scores. Among those flaws were low numbers of levels, missions, and items; no tutorial to enhance usability; and lack of strategies and rewards to increase motivation in the long term.
2022
Martinez, Kim; Menéndez-Menéndez, María Isabel; Bustillo, Andres
A New Measure for Serious Games Evaluation: Gaming Educational Balanced (GEB) Model Journal Article
In: Applied Sciences, vol. 12, iss. 22, no. 11757, 2022, ISSN: 2076-3417.
Abstract | Links | BibTeX | Tags: game design, game evaluation, metrics, serious games
@article{kim2022,
title = {A New Measure for Serious Games Evaluation: Gaming Educational Balanced (GEB) Model},
author = {Kim Martinez and María Isabel Menéndez-Menéndez and Andres Bustillo},
editor = {Maya Satratzemi and Stelios Xinogalos},
doi = {10.3390/app122211757},
issn = {2076-3417},
year = {2022},
date = {2022-11-19},
urldate = {2022-11-19},
journal = {Applied Sciences},
volume = {12},
number = {11757},
issue = {22},
abstract = {Serious games have to meet certain characteristics relating to gameplay and educational content to be effective as educational tools. There are some models that evaluate these aspects, but they usually lack a good balance between both ludic and learning requirements, and provide no guide for the design of new games. This study develops the Gaming Educational Balanced (GEB) Model which addresses these two limitations. GEB is based on the Mechanics, Dynamics and Aesthetics framework and the Four Pillars of Educational Games theory. This model defines a metric to evaluate serious games, which can also be followed to guide their subsequent development. This rubric is tested with three indie serious games developed using different genres to raise awareness of mental illnesses. This evaluation revealed two main issues: the three games returned good results for gameplay, but the application of educational content was deficient, due in all likelihood to the lack of expert educators participating in their development. A statistical and machine learning validation of the results is also performed to ensure that the GEB metric features are clearly explained and the players are able to evaluate them correctly. These results underline the usefulness of the new metric tool for identifying game design strengths and weaknesses. Future works will apply this metric to more serious games to further test its effectiveness and to guide the design of new serious games.},
keywords = {game design, game evaluation, metrics, serious games},
pubstate = {published},
tppubtype = {article}
}
Serious games have to meet certain characteristics relating to gameplay and educational content to be effective as educational tools. There are some models that evaluate these aspects, but they usually lack a good balance between both ludic and learning requirements, and provide no guide for the design of new games. This study develops the Gaming Educational Balanced (GEB) Model which addresses these two limitations. GEB is based on the Mechanics, Dynamics and Aesthetics framework and the Four Pillars of Educational Games theory. This model defines a metric to evaluate serious games, which can also be followed to guide their subsequent development. This rubric is tested with three indie serious games developed using different genres to raise awareness of mental illnesses. This evaluation revealed two main issues: the three games returned good results for gameplay, but the application of educational content was deficient, due in all likelihood to the lack of expert educators participating in their development. A statistical and machine learning validation of the results is also performed to ensure that the GEB metric features are clearly explained and the players are able to evaluate them correctly. These results underline the usefulness of the new metric tool for identifying game design strengths and weaknesses. Future works will apply this metric to more serious games to further test its effectiveness and to guide the design of new serious games.