Introduction

Video games have become one of the most popular forms of entertainment in recent years, with the average consumer spending hundreds of hours in front of a console or computer. Video games are also known as computer games, and they are interactive electronic games that involve a user interface or input device, such as a joystick, controller, keyboard, or motion sensor, to produce audio and visual feedback as a result of the user's action. The user interface (UI) is the space for communication and interaction between humans and machines. The purpose of the user interface is for people to effectively control the machine and to receive feedback from the machine that helps them make decisions (Browne & Anand, 2012). As the gaming industry has grown and technology has advanced, the user interface of video games now involves many aspects, such as the use of images, texts, audio, and video. With that said, the user interface is virtual for the success of the video game, and this research will provide a comprehensive literature review for the application of multimedia UI in video games across platforms, benefiting future game makers.   

First of all, this paper discusses the human factors design process in video game development, highlighting the critical steps involved in understanding user requirements, designing effective solutions, and evaluating design outcomes. Furthermore, the paper examines the unique challenges and considerations for designing user interfaces and user experiences across various gaming platforms, such as PC, console, and mobile devices. By utilizing a combination of visual, auditory, and haptic feedback, designers can create an interactive visual system that aids gamers in decision-making and enhance player experience throughout gameplay. This paper will discuss the importance of visual, auditory, and haptic interfaces in video games, exploring their individual and combined effects on player performance, immersion, and satisfaction. In addition, this paper delves into the challenges faced by designers and explores potential tools and strategies to evaluate and enhance design effectiveness in the gaming industry. Designers and developers must consider various factors, such as user experience, human factors design, and usability, to gauge the effectiveness of their creations. Striking a balance between innovation and user expectations, while assessing and improving playability, is vital to a game's success.

The proposed research will provide valuable insights for game developers and designers, and it will help to create better gaming experiences for all players. In addition, the author believes that this research could provide valuable insight into how game design strategies can be applied to non-game products. Through a better understanding of these strategies, designers are able to create products that are more efficient, engaging, and enjoyable to use. 

Goals of Research

The existing knowledge of multi-sensory user interface design for video games, with a focus on human-centered design, is primarily concentrated on specific platforms and particular types of games. Previous studies have conducted in-depth empirical evaluations on user interface design for mobile video games and multiplayer arcade games. However, the types of games examined are limited to simple scrolling shooter games and educational games (Micheloni et al., 2019). The goal of this research paper is to bridge this gap by comparing and contrasting the existing literature and establishing a comprehensive guide for game UI designers and developers to apply human-centered design principles across various platforms and via different media.

To achieve this, the paper will examine the commonalities and differences across games and platforms, focusing on three types of video games: (1) mobile and tablet video games, (2) console games with joysticks or controllers, and (3) personal computer (PC) video games. The similarities and differences in terms of user preference, design strategies and implementation, while prioritizing the needs and preferences of users, will be addressed for each platform.

Furthermore, this research will delve into the primary components of multi-sensory user interfaces in video games on different platforms, such as auditory, visual, and haptic interfaces. As user interface design for video games represents "a balancing act between aesthetics and mechanics" (Llanos & Jørgensen, 2011), it is crucial to consider both the clarity of information and the elegance of presentation within the context of each game. The paper will investigate optimal solutions for mobile, console, and PC video games that adhere to human-centered design principles, as well as common practices that lead to successful user interfaces.

Moreover, the paper will explore the extent to which multi-sensory user interfaces in video games, designed with a human-centered approach, affect player engagement and satisfaction. By understanding the impact of user interface design on these crucial factors, the paper discusses design challenges, guidelines, and recommendations, which aim to help developers and designers to make more informed decisions when creating video games that not only captivate players, but also facilitate enjoyable and immersive experiences that cater to the needs and desires of the target audience.

Method

The primary research approach for this paper involves conducting a comprehensive literature review of scientific publications and research studies focused on the design of multimedia user interfaces across different video game platforms. To ensure access to a wide range of relevant resources, the author will utilize Tisch Library's JumboSearch and Google Scholar through Tufts University. To maintain the quality and credibility of the research, only peer-reviewed resources will be considered, encompassing books, academic journals, and articles authored by experts in the field. 

The literature review will examine various aspects of user interface design in video games, such as games across platforms, and the application of auditory, visual, and haptic user interfaces. Moreover, the paper would cover design challenges, means of measuring successful design, and recommendations for game designers. The goal is to synthesize the existing body of knowledge to identify trends, challenges, and opportunities within the field. 

It is important to note that this paper will solely rely on secondary research methods, and no primary research activities, such as observations, interviews, or surveys, will be carried out. By focusing on a thorough analysis of the existing literature, the author aims to contribute to the understanding of multimedia user interface design in video games and provide valuable insights for future research and development in the area. 

Findings of Research

Human Factors Design Process

Video games represent a category of software where the user's emotional experience is of utmost importance. A game's success largely depends on its ability to evoke positive emotions in players. While individuals play video games for various reasons, the primary motivation for most players is the pursuit of positive emotions. These emotions can manifest in numerous ways, such as satisfaction, a sense of accomplishment, amusement, or excitement (Johnson & Wiles, 2003). In the realm of video game design, designers employ human factors principles to concentrate on users, their needs, and demands, while utilizing human factors and usability methodologies. This approach boosts effectiveness and efficiency, enhances user well-being, satisfaction, accessibility, and sustainability, and mitigates potential negative impacts on human health, safety, and performance resulting from the use (Johnson & Wiles, 2003).

During the design of interactive systems like video games, there are four critical Human-Centered Design processes that should be carried out. Firstly, understanding and specifying the context of use involves recognizing the game's playing environment and the target audience's characteristics. Secondly, specifying user requirements entails identifying users' needs and wants from the game, which can be achieved through methods such as surveys, interviews, and focus groups. Thirdly, producing design solutions involves creating concepts that address the previously identified user requirements. To test their ideas, designers may develop prototypes or mockups. Lastly, evaluating the design requires testing it with users to determine if it meets their needs and is user-friendly. Designers may employ methods like usability testing or heuristic evaluation to assess the design's effectiveness (Johnson & Wiles, 2003).

Despite video games' popularity and immense revenue, progress in studying their effects and potential has been relatively slow. Video games fall within the realm of Human-Computer Interaction (HCI) research, but they differ from traditional HCI's focus, which is on productivity applications that facilitate user-defined tasks. Games, on the other hand, often assign tasks to players, deliberately making them challenging (Barr et al., 2007). 

The study concludes that “the defining characteristic of video games is the presence of value systems, interconnected values that guide gameplay” (Barr et al., 2007). In video games, there are typically "right" or "correct" ways to proceed, and user interfaces play a significant role in both shaping and conveying the value. Thus, examining video game interfaces can be rooted in understanding values and value systems and their connection to gameplay. A solid foundation in established theories of behavior and gameplay, in addition to in-depth analyses of actual video games using qualitative methods, is crucial. Through such analyses, scholars can comprehend the inner workings, effects, and potential of video games (Barr et al., 2007).

Video games Across Platforms

Incorporating human factors principles in video game design is crucial for creating an enjoyable and immersive gaming experience. To achieve this, designers must pay close attention to visual cues, minimize user errors, and facilitate comfortable interactions between players and the game. The user interface (UI) and user experience design (UXD) are integral components of a successful game. The user interface is responsible for ensuring smooth task execution and effective communication between the game and the player.  User experience design, on the other hand, focuses on delivering an engaging and enjoyable experience that keeps players invested in the game.

Each gaming platform, such as PC, console, and mobile, has its own unique characteristics and limitations, which have a direct impact on UI design. PC gaming is known for its versatility in terms of specifications and variety, as it can support a wide range of games without restrictions. Console gaming is more limited, as it can only play games specifically designed for the console. Mobile gaming, while convenient, has inferior specifications compared to the other two platforms. Despite these differences, each platform has its own strengths. Mobile gaming platforms excel in privacy and comfort, allowing players to enjoy games anywhere and anytime. PC gaming, however, provides the most satisfying gaming experience due to its adaptability and expansive game library (Galehantomo, 2015).

When designing UI for different platforms, it is essential to consider their specific features, such as input options and viewing distances. Console gaming relies on controllers with limited input options, while PC gaming offers more versatile inputs through mouse and keyboard. Additionally, console players generally sit further away from their screens, using TVs, whereas PC players sit closer to their monitors. These factors must be taken into account to optimize human factors in video game design, ultimately enhancing the gaming experience for players across all platforms (Galehantomo, 2015).

Types of User Interface

The user interface in video games plays a vital role in enabling interaction and conveying information to players. In gaming, UI design can be defined as an interactive visual system that offers information about potential actions or objects, aiding gamers in decision-making throughout gameplay via various user inputs. Multiple interface mediums are used in video games, including auditory, visual, and haptic feedback. By following human factors principles, these mediums are tailored to meet players' desires, necessities, and core motivations while they engage in gaming.

Overall, gamers tend to have improved experiences with multi-sensory displays and feel more confident in their performance when utilizing them, even if their actual performance remains unchanged (Nesbitt & Hoskens, 2008). The following sections will delve into a detailed examination of the three types of interfaces utilized in video games.

Visual Interface

Visual elements, such as on-screen instructions or maps, help users follow video game actions. Human factors principles can be applied to ensure visual elements are intuitive and user-friendly. As shown in Table 1, game UI design has four visual representation types: diegetic, heads-up displays (HUDs), spatial, and meta (Peacocke, 2018). 

  • Diegetic elements are visible or audible to characters and fit within the story context. 

  • heads-up displays (HUDs) exist outside the game or story and are only visible to players. 

  • Spatial elements are represented within the game space but aren't visible to characters. 

  • Meta elements are contextual to the game but aren't represented within the game space.


Drawing from several studies on different video games, the arthur discusses the implications of visual interfaces in video games, covering different aspects of the game, including the aesthetics, usability, and overall user satisfaction. A study by Llanos and Jørgensen examines scrolling text or colored overlays used to signify changes in a player's health status. This initial analysis reveals that the concept of "immersive" or minimal interfaces, which aim to incorporate all system information within the game world and present it as part of the narrative, is not always the best approach. Although players often find minimal UI aesthetically pleasing, they prefer sufficient information to interact meaningfully with the game mechanics and world. On the other hand, excess information can be annoying and disrupt immersion. Consequently, UI designers should present gameplay-relevant information clearly and precisely, consider the desired game experience, and reflect on specific UI approaches to achieve it (Llanos & Jørgensen, 2011).

In another study, Browne and Anand (2012) aimed to develop design heuristics for visual user interfaces by examining a specific mobile video game. While there are limitations of this study, including a small, non-diverse sample size and a focus on a single game type, the study identified five key design heuristics, which can be applied to create more engaging and user-friendly mobile user interfaces. These include offering an accelerometer-based interface, providing multiple user interfaces, using touch gestures for diagonal direction input, allowing configurable interface sensitivity, and directly translating physical gesture properties into virtual properties (Browne & Anand, 2012). By incorporating these heuristics, developers can improve the user experience for a range of applications, including video games of different types and on different platforms, as long as there are similar input requirements.

Finally, the study by Charbonneau and others highlight the importance of human factors in game design, specifically focusing on the effectiveness of different visual interfaces. The research paper focused on conveying dance sequence information for full-body rhythm dance games. By considering the user experience, the research emphasizes the critical role visuals play in shaping player performance and satisfaction. The findings reveal that Motion Lines and Beat Circles not only led to higher scores but also provided a more intuitive and less confusing experience compared to the conventional Timeline interface. This underscores the significance of carefully designing visual interfaces that cater to human factors, ultimately enhancing the overall gaming experience (Charbonneau et al., 2009).

Auditory Interface

In addition to the visual interface in video games, another key component is the auditory interface. Key audio elements in games include “vocalizations of game characters, sound effects relating to gameplay, ambient effects relating to atmosphere, and the music of the game” (Amiriparian et al., 2020). The blend of these audio elements within a specific game is closely tied to the gameplay mechanics and game genre. For instance, action and shooting games like the Call of Duty series feature intense, abrupt sounds such as punches and gunshots. On the other hand, sports games like the FIFA football series include continuous commentary voice-overs. Finally, racing games like the Forza series are characterized by prominent car noises, including powerful accelerations and squealing brakes (Amiriparian et al., 2020). Audio interface in games is powerful as it sets the tone for the game, and it also heavily impacts users’ immersion and emotional engagement. 

Furthermore, a study by Nesbitt and Hoskens focused on a similar topic, investigating the impact of game audio on immersion and emotional engagement. They conducted experiments to explore the significance of various audio elements and their compatibility within video games. Findings indicate that sound effects are the most crucial audio elements. The experiments reveal that game audio influences players' gameplay experiences in multiple ways, emphasizing audio's importance in video games. Although audio elements could be subtle and hardly noticeable for players, minor details such as gunshot sounds or enemy footsteps to background music alters player emotions (Nesbitt & Hoskens, 2008). 

Moreover, audio not only affects gameplay but also enhances players' satisfaction in performing actions, based on participant feedback. Game audio, particularly sound effects, is vital for creating an enjoyable experience. Designers and developers should consider aspects beyond gameplay. Nesbitt and Hoskensuture suggested that future research could utilize more specific questionnaire questions, such as each participant's gaming experience, and examine minor variables like playing the same type of game with different level designs. Audio cues can provide feedback on player actions or convey essential information about the game world, and human factors principles can be applied to ensure clarity and comprehension (Nesbitt & Hoskens, 2008).

Haptic Interface

A haptic interface describes the generated force in response to interactions within a virtual world (Park et al., 2004). In addition to video games, haptic devices have applications in various fields, such as telerobotics, military, medical, and entertainment systems. Traditional video games primarily relied on visual and auditory senses, using keyboards or joysticks for user interaction. However, when appropriate forces are applied based on factors such as collision speed and material properties in the game environment, alongside visuals and sound, players can experience a more realistic and immersive gaming experience (Park et al., 2004). Haptic interfaces are now widely used in shooters games, arcade games, and exergames. 

As mentioned above, haptic feedback conveys information through applied forces and vibrations, either as tactile (touch, texture, vibration) or kinesthetic (force, weight, resistance) feedback (Stach & Graham, 2011). Haptics have been used to improve interactions in various contexts, such as enhancing task performance and virtual presence in distributed groupware systems, and decreasing reaction times in various game genres. According to Stach and Graham, haptic feedback can address three critical exergame design areas: balancing group exercise for different physical abilities, guiding players to safe and healthy interaction levels, and providing immersive experiences to increase physical activity enjoyment. Haptic feedback enhances immersion by offering tactile sensations corresponding to in-game events. Designers can use human factors principles to ensure haptic feedback is effective and doesn't overwhelm or confuse players (Stach & Graham, 2011).

The previous sections discussed the application and importance of visual, auditory, and haptic interfaces. The question now is, from a human factors perspective, which one of them is most important in video games? Does a combination of all three work best? As the study by Nesbitt and Hoskens suggests, in general, players reported an enhanced experience when they were provided with multiple sensory cues. The combination of visual, auditory, and haptic feedback received the highest ratings in terms of immersion, confidence, and satisfaction. When comparing displays with a single extra sensory cue, players preferred visual/auditory over visual/haptic interfaces. In addition, most players emphasized the importance of sound in the gaming experience, mentioning reasons such as blocking external distractions and assisting in anticipating in-game movement and combat areas. Although some found haptic feedback is a significant factor to the game experience, it was not seen as useful for gameplay as auditory cues. These preferences were consistent across the three experience criteria and were supported by player comments (Nesbitt & Hoskens, 2008). The authors suggest that the combination methodology could also be useful for game designers who wish to use multi-sensory displays to provide a greater spectrum of information levels to the player. Better design of the way information is displayed to the different senses could be used to improve player performance, or alternatively to make some game tasks more challenging (Nesbitt & Hoskens, 2008).

Challenges and Measuring Success

The previous sections examined the state-of-art technologies for video games and the different types of interfaces that are frequently used, providing a toolbox for designers and developers. Beyond that, a more important question is, how do designers and developers measure the success of the design? How do designers know whether users will like, and stick to the game? Human factors design also plays a role in the evaluation. The constant evolution of technologies and design strategies presents new challenges for designers, requiring them to balance innovation with user expectations, conduct usability assessments, and employ playability heuristics. In this section of the paper, the author will discuss a few challenges for designers, and potential tools to evaluate design effectiveness.

Pre-learned Interaction Model

The first challenge for designers is that players usually have a pre-learned interaction model when they encounter a new game. Hence, players would attempt to interact with the game based on this model, and if a game doesn't accommodate these models, players can become confused and frustrated. As discussed in the paper “Facing the Human Factors Challenges in Game Design: A Discussion Panel”, the visual design, along with auditory and visual cues, can help guide players' interaction models (Portnoy et al., 2011). 

More specifically, Microsoft Game experiences excel when players can instantly access the experience without barriers, which requires a deep understanding of players' prior learning, expectations, and initial behaviors. However, it was mentioned that some games still need to push players to adjust their models, risking a low retention rate. It is a challenge for game designers to strike the perfect balance between making the game creative and original, and adhering to users’ pre-learned interaction models (Portnoy et al., 2011). 

Usability Assessment

In the gaming industry, usability assessment is characterized by its focus on user purpose and the role of the usability process. Video games emphasize entertainment and players' emotional states, like engagement and fun, over efficiency, whereas in other industries, the priority is task efficiency and user productivity (Portnoy et al., 2011). The usability process in gaming not only seeks to comprehend user experience but also aids in the iterative game design process. It is a useful tool for designers to evaluate design effectiveness.

The distinct nature of usability assessment in gaming affects the methods employed, such as psychophysiological techniques that continuously and objectively gauge players' emotional states without disrupting gameplay. Developers can generate emotional profiles to assess whether game features provoke the desired emotional response. In addition, eye tracking is another method that documents players' gaze patterns to evaluate attention focus and comprehend how visual information is processed in games (Portnoy et al., 2011).

Playability Heuristics 

Finally, the study by Soomro and others utilized two established playability heuristics to evaluate six Android games across various genres with fourteen test participants. Findings from the study indicate that the current playability heuristics fall short in identifying playability issues, such as “usability, mobility, gameplay, and multiplayer functionality” (Soomro et al., 2013) for touchscreen mobile game applications, as indicated in Table 2 below. While this list is not meant to be exhaustive, it provides meaningful metrics for video game design across platforms.  

Design Recommendations

Finally, the study "Evaluating Affective User-Centered Design of Video Games Using Qualitative Methods" by Ng and colleagues helped to establish a comprehensive set of guidelines for user-centered design in the context of video games. Affective user-centered design is an approach that assesses whether the emotions, meanings, and mental concepts conveyed through a video game's tangible and intangible elements align with the preferences and expectations of the intended users. In their research, Ng and others employed qualitative methods to thoroughly analyze data and identify key factors that contribute to an engaging and enjoyable gaming experience. As a result of their analysis, they proposed 15 recommendations for user-centered design in video games. These recommendations cover various aspects of game design. Note that the following list is adapted from the research paper by Ng et al.

  1. Diversity of Users

  2. User Capability

  3. Challenging Gameplay

  4. Provide Goals

  5. Provide Rewards

  6. Impressive Visuals

  7. Graphic Quality

  8. Creative Gameplay

  9. Good Narrative

  10. Interesting Game Characters

  11. Flow in Games

  12. Flexible Options

  13. Good Gaming Interface

  14. Tutorial and Hints

  15. Easy-to-Use Controls

Difficulties and Remedies

One difficulty is finding relevant literature on the topic. This can be especially challenging since the topic of the user interface for video games is relatively new and not well-researched. A solution would be exploring different sources for publishing studies. Google Scholar offers a comprehensive source for peer-reviewed journals and articles, and it is also worth investigating studies published in different countries and languages. However, given enough papers on this topic, a remaining significant challenge in conducting this literature review was the heterogeneity of studies in terms of research methods, design approaches, and platforms under consideration. This made it difficult to compare and synthesize results across studies. In addition, the limited amount of empirical research focusing specifically on human factors design and multi-sensory user interfaces in video games made it challenging to draw robust conclusions. Much of the literature was based on anecdotal evidence, case studies, or expert opinions, which limited the generalizability of the findings. Most quantitative studies included less than 50 participants in their studies. To address this issue, a systematic literature search using well-defined inclusion and exclusion criteria was employed. This approach allowed for the identification and inclusion of high-quality studies that met the research objectives.

In addition, the rapid pace of technological advancements in the gaming industry posed challenges in identifying studies that were still relevant and up-to-date. This required a constant review of recent publications and an assessment of their relevance to the current state of technology. To stay abreast of the latest developments in the field, it is essential to regularly update the literature review by incorporating recent studies and technological advancements. This will help maintain the relevance of the review and provide up-to-date insights.

Discussion

The findings of this literature review highlight the importance of human factors principles in video game design, as they significantly influence players' emotional experiences and overall satisfaction. The four critical Human-Centered Design processes, including understanding the context of use, specifying user requirements, producing design solutions, and evaluating the design, play crucial roles in creating an enjoyable and immersive gaming experience.

The study also emphasizes the need to consider the unique characteristics and limitations of different gaming platforms such as PC, console, and mobile when designing user interfaces. By understanding these platform-specific features, designers can optimize human factors in video game design, ultimately enhancing the gaming experience for players across all platforms. 

In terms of user interfaces, the study examines the importance of auditory, visual, and haptic feedback in providing an interactive and engaging experience for players. By following human factors principles, these interface mediums can be tailored to meet players' desires, necessities, and core motivations. 

The challenges faced by designers in the context of pre-learned interaction models, usability assessment, and playability heuristics are also examined. These challenges underscore the need for a deep understanding of players' prior learning, expectations, and initial behaviors, as well as the unique nature of usability assessment in the gaming industry. Finally, the study on affective user-centered design in video games offers 15 recommendations, covering various aspects of game design such as diversity of users, challenging gameplay, impressive visuals, creative gameplay, flow in games, good gaming interface, and difficulties and remedies. These recommendations serve as a valuable resource for designers seeking to create engaging and enjoyable gaming experiences that align with the preferences and expectations of their intended users.

Maturity of the Topic

The research topic, “Human Factors Design of Multi-sensory User Interfaces in Video Games Across Platforms,” is a relatively mature field of study, with its roots dating back to the early days of video gaming. However, as video game technology and design have evolved, so too has the research in this area. Multi-sensory User Interfaces are interfaces that combine multiple input modalities (such as keyboard, mouse, touch, voice, etc.) and output modalities (such as visual, auditory, haptic feedback, etc.) to provide a seamless and immersive experience for the player. Research on user interface in video games has been growing steadily since the 2000s, as developers have experimented with novel ways of integrating multiple modalities to create more immersive gameplay experiences. 

With the proliferation of gaming platforms like PC, consoles, mobile devices, and virtual reality systems, the study of user interfaces across these different platforms has become increasingly important. Researchers have been comparing and contrasting the unique challenges and opportunities presented by each platform, as well as identifying best practices for designing interfaces that can be easily adapted for different devices. This area of research has gained momentum over the past two decades and continues to evolve as new gaming platforms emerge.

While substantial progress has been made over the years, ongoing advancements in technology and gaming platforms, as well as the growing emphasis on usability and accessibility, ensure that this field will continue to develop and expand in the coming years.

Future Directions

Cross-platform Design Strategies

As gaming platforms continue to diversify, there is an increasing need for more effective cross-platform design strategies. Future research should focus on developing guidelines and best practices for designing user interfaces that can adapt seamlessly across different platforms while maintaining a consistent user experience.

Inclusive and Accessible Design

Inclusivity and accessibility should be key considerations in future research on user interface design for video games. This involves conducting research on how to cater to gamers with varying abilities, cultural backgrounds, and preferences, ensuring that games are enjoyable and accessible to a wider audience. This might include the development of adaptable user interfaces, alternative control schemes, and customizable difficulty settings.

Evaluation and Validation of Design Principles

In order to establish effective design principles for video game user interfaces, future research should prioritize the evaluation and validation of these principles through rigorous testing and user feedback. This will ensure that the design recommendations are grounded in empirical evidence and can contribute to creating more effective and user-friendly gaming experiences.

Conclusion

In conclusion, this paper provides a comprehensive literature review of the human factors design of Multi-sensory User Interfaces in Video Games Across Platforms. The effective implementation of human factors design principles is crucial in the development of successful video games that provide enjoyable and immersive experiences for players. By carefully considering user requirements, designing tailored solutions, and evaluating outcomes, designers can create games that resonate with their target audience and evoke positive emotions. Moreover, the paper emphasizes the importance of understanding the unique characteristics and limitations of different gaming platforms, as these factors directly impact user interface and user experience design. Furthermore, a well-designed video game user interface incorporating visual, auditory, and haptic feedback can significantly impact the gaming experience by enhancing player performance, immersion, and satisfaction. While each interface medium has its unique strengths, a multi-sensory approach is generally preferred by players, as evidenced by Nesbitt and Hoskens' study. Ultimately, striking the right balance between aesthetic appeal, usability, and immersion is crucial for creating a successful user interface that caters to players' diverse needs and preferences. Finally, creating a successful video game requires a deep understanding of user preferences, capabilities, and pre-learned interaction models. By incorporating usability assessments and playability heuristics, designers can fine-tune their creations to deliver an engaging and enjoyable gaming experience. Although substantial progress has been made in this field, future research should focus on cross-platform design strategies, inclusive and accessible design, and the evaluation and validation of design principles to ensure continued growth and relevance in this rapidly evolving industry.




Introduction

Video games have become one of the most popular forms of entertainment in recent years, with the average consumer spending hundreds of hours in front of a console or computer. Video games are also known as computer games, and they are interactive electronic games that involve a user interface or input device, such as a joystick, controller, keyboard, or motion sensor, to produce audio and visual feedback as a result of the user's action. The user interface (UI) is the space for communication and interaction between humans and machines. The purpose of the user interface is for people to effectively control the machine and to receive feedback from the machine that helps them make decisions (Browne & Anand, 2012). As the gaming industry has grown and technology has advanced, the user interface of video games now involves many aspects, such as the use of images, texts, audio, and video. With that said, the user interface is virtual for the success of the video game, and this research will provide a comprehensive literature review for the application of multimedia UI in video games across platforms, benefiting future game makers.   

First of all, this paper discusses the human factors design process in video game development, highlighting the critical steps involved in understanding user requirements, designing effective solutions, and evaluating design outcomes. Furthermore, the paper examines the unique challenges and considerations for designing user interfaces and user experiences across various gaming platforms, such as PC, console, and mobile devices. By utilizing a combination of visual, auditory, and haptic feedback, designers can create an interactive visual system that aids gamers in decision-making and enhance player experience throughout gameplay. This paper will discuss the importance of visual, auditory, and haptic interfaces in video games, exploring their individual and combined effects on player performance, immersion, and satisfaction. In addition, this paper delves into the challenges faced by designers and explores potential tools and strategies to evaluate and enhance design effectiveness in the gaming industry. Designers and developers must consider various factors, such as user experience, human factors design, and usability, to gauge the effectiveness of their creations. Striking a balance between innovation and user expectations, while assessing and improving playability, is vital to a game's success.

The proposed research will provide valuable insights for game developers and designers, and it will help to create better gaming experiences for all players. In addition, the author believes that this research could provide valuable insight into how game design strategies can be applied to non-game products. Through a better understanding of these strategies, designers are able to create products that are more efficient, engaging, and enjoyable to use. 

Goals of Research

The existing knowledge of multi-sensory user interface design for video games, with a focus on human-centered design, is primarily concentrated on specific platforms and particular types of games. Previous studies have conducted in-depth empirical evaluations on user interface design for mobile video games and multiplayer arcade games. However, the types of games examined are limited to simple scrolling shooter games and educational games (Micheloni et al., 2019). The goal of this research paper is to bridge this gap by comparing and contrasting the existing literature and establishing a comprehensive guide for game UI designers and developers to apply human-centered design principles across various platforms and via different media.

To achieve this, the paper will examine the commonalities and differences across games and platforms, focusing on three types of video games: (1) mobile and tablet video games, (2) console games with joysticks or controllers, and (3) personal computer (PC) video games. The similarities and differences in terms of user preference, design strategies and implementation, while prioritizing the needs and preferences of users, will be addressed for each platform.

Furthermore, this research will delve into the primary components of multi-sensory user interfaces in video games on different platforms, such as auditory, visual, and haptic interfaces. As user interface design for video games represents "a balancing act between aesthetics and mechanics" (Llanos & Jørgensen, 2011), it is crucial to consider both the clarity of information and the elegance of presentation within the context of each game. The paper will investigate optimal solutions for mobile, console, and PC video games that adhere to human-centered design principles, as well as common practices that lead to successful user interfaces.

Moreover, the paper will explore the extent to which multi-sensory user interfaces in video games, designed with a human-centered approach, affect player engagement and satisfaction. By understanding the impact of user interface design on these crucial factors, the paper discusses design challenges, guidelines, and recommendations, which aim to help developers and designers to make more informed decisions when creating video games that not only captivate players, but also facilitate enjoyable and immersive experiences that cater to the needs and desires of the target audience.

Method

The primary research approach for this paper involves conducting a comprehensive literature review of scientific publications and research studies focused on the design of multimedia user interfaces across different video game platforms. To ensure access to a wide range of relevant resources, the author will utilize Tisch Library's JumboSearch and Google Scholar through Tufts University. To maintain the quality and credibility of the research, only peer-reviewed resources will be considered, encompassing books, academic journals, and articles authored by experts in the field. 

The literature review will examine various aspects of user interface design in video games, such as games across platforms, and the application of auditory, visual, and haptic user interfaces. Moreover, the paper would cover design challenges, means of measuring successful design, and recommendations for game designers. The goal is to synthesize the existing body of knowledge to identify trends, challenges, and opportunities within the field. 

It is important to note that this paper will solely rely on secondary research methods, and no primary research activities, such as observations, interviews, or surveys, will be carried out. By focusing on a thorough analysis of the existing literature, the author aims to contribute to the understanding of multimedia user interface design in video games and provide valuable insights for future research and development in the area. 

Findings of Research

Human Factors Design Process

Video games represent a category of software where the user's emotional experience is of utmost importance. A game's success largely depends on its ability to evoke positive emotions in players. While individuals play video games for various reasons, the primary motivation for most players is the pursuit of positive emotions. These emotions can manifest in numerous ways, such as satisfaction, a sense of accomplishment, amusement, or excitement (Johnson & Wiles, 2003). In the realm of video game design, designers employ human factors principles to concentrate on users, their needs, and demands, while utilizing human factors and usability methodologies. This approach boosts effectiveness and efficiency, enhances user well-being, satisfaction, accessibility, and sustainability, and mitigates potential negative impacts on human health, safety, and performance resulting from the use (Johnson & Wiles, 2003).

During the design of interactive systems like video games, there are four critical Human-Centered Design processes that should be carried out. Firstly, understanding and specifying the context of use involves recognizing the game's playing environment and the target audience's characteristics. Secondly, specifying user requirements entails identifying users' needs and wants from the game, which can be achieved through methods such as surveys, interviews, and focus groups. Thirdly, producing design solutions involves creating concepts that address the previously identified user requirements. To test their ideas, designers may develop prototypes or mockups. Lastly, evaluating the design requires testing it with users to determine if it meets their needs and is user-friendly. Designers may employ methods like usability testing or heuristic evaluation to assess the design's effectiveness (Johnson & Wiles, 2003).

Despite video games' popularity and immense revenue, progress in studying their effects and potential has been relatively slow. Video games fall within the realm of Human-Computer Interaction (HCI) research, but they differ from traditional HCI's focus, which is on productivity applications that facilitate user-defined tasks. Games, on the other hand, often assign tasks to players, deliberately making them challenging (Barr et al., 2007). 

The study concludes that “the defining characteristic of video games is the presence of value systems, interconnected values that guide gameplay” (Barr et al., 2007). In video games, there are typically "right" or "correct" ways to proceed, and user interfaces play a significant role in both shaping and conveying the value. Thus, examining video game interfaces can be rooted in understanding values and value systems and their connection to gameplay. A solid foundation in established theories of behavior and gameplay, in addition to in-depth analyses of actual video games using qualitative methods, is crucial. Through such analyses, scholars can comprehend the inner workings, effects, and potential of video games (Barr et al., 2007).

Video games Across Platforms

Incorporating human factors principles in video game design is crucial for creating an enjoyable and immersive gaming experience. To achieve this, designers must pay close attention to visual cues, minimize user errors, and facilitate comfortable interactions between players and the game. The user interface (UI) and user experience design (UXD) are integral components of a successful game. The user interface is responsible for ensuring smooth task execution and effective communication between the game and the player.  User experience design, on the other hand, focuses on delivering an engaging and enjoyable experience that keeps players invested in the game.

Each gaming platform, such as PC, console, and mobile, has its own unique characteristics and limitations, which have a direct impact on UI design. PC gaming is known for its versatility in terms of specifications and variety, as it can support a wide range of games without restrictions. Console gaming is more limited, as it can only play games specifically designed for the console. Mobile gaming, while convenient, has inferior specifications compared to the other two platforms. Despite these differences, each platform has its own strengths. Mobile gaming platforms excel in privacy and comfort, allowing players to enjoy games anywhere and anytime. PC gaming, however, provides the most satisfying gaming experience due to its adaptability and expansive game library (Galehantomo, 2015).

When designing UI for different platforms, it is essential to consider their specific features, such as input options and viewing distances. Console gaming relies on controllers with limited input options, while PC gaming offers more versatile inputs through mouse and keyboard. Additionally, console players generally sit further away from their screens, using TVs, whereas PC players sit closer to their monitors. These factors must be taken into account to optimize human factors in video game design, ultimately enhancing the gaming experience for players across all platforms (Galehantomo, 2015).

Types of User Interface

The user interface in video games plays a vital role in enabling interaction and conveying information to players. In gaming, UI design can be defined as an interactive visual system that offers information about potential actions or objects, aiding gamers in decision-making throughout gameplay via various user inputs. Multiple interface mediums are used in video games, including auditory, visual, and haptic feedback. By following human factors principles, these mediums are tailored to meet players' desires, necessities, and core motivations while they engage in gaming.

Overall, gamers tend to have improved experiences with multi-sensory displays and feel more confident in their performance when utilizing them, even if their actual performance remains unchanged (Nesbitt & Hoskens, 2008). The following sections will delve into a detailed examination of the three types of interfaces utilized in video games.

Visual Interface

Visual elements, such as on-screen instructions or maps, help users follow video game actions. Human factors principles can be applied to ensure visual elements are intuitive and user-friendly. As shown in Table 1, game UI design has four visual representation types: diegetic, heads-up displays (HUDs), spatial, and meta (Peacocke, 2018). 

  • Diegetic elements are visible or audible to characters and fit within the story context. 

  • heads-up displays (HUDs) exist outside the game or story and are only visible to players. 

  • Spatial elements are represented within the game space but aren't visible to characters. 

  • Meta elements are contextual to the game but aren't represented within the game space.


Drawing from several studies on different video games, the arthur discusses the implications of visual interfaces in video games, covering different aspects of the game, including the aesthetics, usability, and overall user satisfaction. A study by Llanos and Jørgensen examines scrolling text or colored overlays used to signify changes in a player's health status. This initial analysis reveals that the concept of "immersive" or minimal interfaces, which aim to incorporate all system information within the game world and present it as part of the narrative, is not always the best approach. Although players often find minimal UI aesthetically pleasing, they prefer sufficient information to interact meaningfully with the game mechanics and world. On the other hand, excess information can be annoying and disrupt immersion. Consequently, UI designers should present gameplay-relevant information clearly and precisely, consider the desired game experience, and reflect on specific UI approaches to achieve it (Llanos & Jørgensen, 2011).

In another study, Browne and Anand (2012) aimed to develop design heuristics for visual user interfaces by examining a specific mobile video game. While there are limitations of this study, including a small, non-diverse sample size and a focus on a single game type, the study identified five key design heuristics, which can be applied to create more engaging and user-friendly mobile user interfaces. These include offering an accelerometer-based interface, providing multiple user interfaces, using touch gestures for diagonal direction input, allowing configurable interface sensitivity, and directly translating physical gesture properties into virtual properties (Browne & Anand, 2012). By incorporating these heuristics, developers can improve the user experience for a range of applications, including video games of different types and on different platforms, as long as there are similar input requirements.

Finally, the study by Charbonneau and others highlight the importance of human factors in game design, specifically focusing on the effectiveness of different visual interfaces. The research paper focused on conveying dance sequence information for full-body rhythm dance games. By considering the user experience, the research emphasizes the critical role visuals play in shaping player performance and satisfaction. The findings reveal that Motion Lines and Beat Circles not only led to higher scores but also provided a more intuitive and less confusing experience compared to the conventional Timeline interface. This underscores the significance of carefully designing visual interfaces that cater to human factors, ultimately enhancing the overall gaming experience (Charbonneau et al., 2009).

Auditory Interface

In addition to the visual interface in video games, another key component is the auditory interface. Key audio elements in games include “vocalizations of game characters, sound effects relating to gameplay, ambient effects relating to atmosphere, and the music of the game” (Amiriparian et al., 2020). The blend of these audio elements within a specific game is closely tied to the gameplay mechanics and game genre. For instance, action and shooting games like the Call of Duty series feature intense, abrupt sounds such as punches and gunshots. On the other hand, sports games like the FIFA football series include continuous commentary voice-overs. Finally, racing games like the Forza series are characterized by prominent car noises, including powerful accelerations and squealing brakes (Amiriparian et al., 2020). Audio interface in games is powerful as it sets the tone for the game, and it also heavily impacts users’ immersion and emotional engagement. 

Furthermore, a study by Nesbitt and Hoskens focused on a similar topic, investigating the impact of game audio on immersion and emotional engagement. They conducted experiments to explore the significance of various audio elements and their compatibility within video games. Findings indicate that sound effects are the most crucial audio elements. The experiments reveal that game audio influences players' gameplay experiences in multiple ways, emphasizing audio's importance in video games. Although audio elements could be subtle and hardly noticeable for players, minor details such as gunshot sounds or enemy footsteps to background music alters player emotions (Nesbitt & Hoskens, 2008). 

Moreover, audio not only affects gameplay but also enhances players' satisfaction in performing actions, based on participant feedback. Game audio, particularly sound effects, is vital for creating an enjoyable experience. Designers and developers should consider aspects beyond gameplay. Nesbitt and Hoskensuture suggested that future research could utilize more specific questionnaire questions, such as each participant's gaming experience, and examine minor variables like playing the same type of game with different level designs. Audio cues can provide feedback on player actions or convey essential information about the game world, and human factors principles can be applied to ensure clarity and comprehension (Nesbitt & Hoskens, 2008).

Haptic Interface

A haptic interface describes the generated force in response to interactions within a virtual world (Park et al., 2004). In addition to video games, haptic devices have applications in various fields, such as telerobotics, military, medical, and entertainment systems. Traditional video games primarily relied on visual and auditory senses, using keyboards or joysticks for user interaction. However, when appropriate forces are applied based on factors such as collision speed and material properties in the game environment, alongside visuals and sound, players can experience a more realistic and immersive gaming experience (Park et al., 2004). Haptic interfaces are now widely used in shooters games, arcade games, and exergames. 

As mentioned above, haptic feedback conveys information through applied forces and vibrations, either as tactile (touch, texture, vibration) or kinesthetic (force, weight, resistance) feedback (Stach & Graham, 2011). Haptics have been used to improve interactions in various contexts, such as enhancing task performance and virtual presence in distributed groupware systems, and decreasing reaction times in various game genres. According to Stach and Graham, haptic feedback can address three critical exergame design areas: balancing group exercise for different physical abilities, guiding players to safe and healthy interaction levels, and providing immersive experiences to increase physical activity enjoyment. Haptic feedback enhances immersion by offering tactile sensations corresponding to in-game events. Designers can use human factors principles to ensure haptic feedback is effective and doesn't overwhelm or confuse players (Stach & Graham, 2011).

The previous sections discussed the application and importance of visual, auditory, and haptic interfaces. The question now is, from a human factors perspective, which one of them is most important in video games? Does a combination of all three work best? As the study by Nesbitt and Hoskens suggests, in general, players reported an enhanced experience when they were provided with multiple sensory cues. The combination of visual, auditory, and haptic feedback received the highest ratings in terms of immersion, confidence, and satisfaction. When comparing displays with a single extra sensory cue, players preferred visual/auditory over visual/haptic interfaces. In addition, most players emphasized the importance of sound in the gaming experience, mentioning reasons such as blocking external distractions and assisting in anticipating in-game movement and combat areas. Although some found haptic feedback is a significant factor to the game experience, it was not seen as useful for gameplay as auditory cues. These preferences were consistent across the three experience criteria and were supported by player comments (Nesbitt & Hoskens, 2008). The authors suggest that the combination methodology could also be useful for game designers who wish to use multi-sensory displays to provide a greater spectrum of information levels to the player. Better design of the way information is displayed to the different senses could be used to improve player performance, or alternatively to make some game tasks more challenging (Nesbitt & Hoskens, 2008).

Challenges and Measuring Success

The previous sections examined the state-of-art technologies for video games and the different types of interfaces that are frequently used, providing a toolbox for designers and developers. Beyond that, a more important question is, how do designers and developers measure the success of the design? How do designers know whether users will like, and stick to the game? Human factors design also plays a role in the evaluation. The constant evolution of technologies and design strategies presents new challenges for designers, requiring them to balance innovation with user expectations, conduct usability assessments, and employ playability heuristics. In this section of the paper, the author will discuss a few challenges for designers, and potential tools to evaluate design effectiveness.

Pre-learned Interaction Model

The first challenge for designers is that players usually have a pre-learned interaction model when they encounter a new game. Hence, players would attempt to interact with the game based on this model, and if a game doesn't accommodate these models, players can become confused and frustrated. As discussed in the paper “Facing the Human Factors Challenges in Game Design: A Discussion Panel”, the visual design, along with auditory and visual cues, can help guide players' interaction models (Portnoy et al., 2011). 

More specifically, Microsoft Game experiences excel when players can instantly access the experience without barriers, which requires a deep understanding of players' prior learning, expectations, and initial behaviors. However, it was mentioned that some games still need to push players to adjust their models, risking a low retention rate. It is a challenge for game designers to strike the perfect balance between making the game creative and original, and adhering to users’ pre-learned interaction models (Portnoy et al., 2011). 

Usability Assessment

In the gaming industry, usability assessment is characterized by its focus on user purpose and the role of the usability process. Video games emphasize entertainment and players' emotional states, like engagement and fun, over efficiency, whereas in other industries, the priority is task efficiency and user productivity (Portnoy et al., 2011). The usability process in gaming not only seeks to comprehend user experience but also aids in the iterative game design process. It is a useful tool for designers to evaluate design effectiveness.

The distinct nature of usability assessment in gaming affects the methods employed, such as psychophysiological techniques that continuously and objectively gauge players' emotional states without disrupting gameplay. Developers can generate emotional profiles to assess whether game features provoke the desired emotional response. In addition, eye tracking is another method that documents players' gaze patterns to evaluate attention focus and comprehend how visual information is processed in games (Portnoy et al., 2011).

Playability Heuristics 

Finally, the study by Soomro and others utilized two established playability heuristics to evaluate six Android games across various genres with fourteen test participants. Findings from the study indicate that the current playability heuristics fall short in identifying playability issues, such as “usability, mobility, gameplay, and multiplayer functionality” (Soomro et al., 2013) for touchscreen mobile game applications, as indicated in Table 2 below. While this list is not meant to be exhaustive, it provides meaningful metrics for video game design across platforms.  

Design Recommendations

Finally, the study "Evaluating Affective User-Centered Design of Video Games Using Qualitative Methods" by Ng and colleagues helped to establish a comprehensive set of guidelines for user-centered design in the context of video games. Affective user-centered design is an approach that assesses whether the emotions, meanings, and mental concepts conveyed through a video game's tangible and intangible elements align with the preferences and expectations of the intended users. In their research, Ng and others employed qualitative methods to thoroughly analyze data and identify key factors that contribute to an engaging and enjoyable gaming experience. As a result of their analysis, they proposed 15 recommendations for user-centered design in video games. These recommendations cover various aspects of game design. Note that the following list is adapted from the research paper by Ng et al.

  1. Diversity of Users

  2. User Capability

  3. Challenging Gameplay

  4. Provide Goals

  5. Provide Rewards

  6. Impressive Visuals

  7. Graphic Quality

  8. Creative Gameplay

  9. Good Narrative

  10. Interesting Game Characters

  11. Flow in Games

  12. Flexible Options

  13. Good Gaming Interface

  14. Tutorial and Hints

  15. Easy-to-Use Controls

Difficulties and Remedies

One difficulty is finding relevant literature on the topic. This can be especially challenging since the topic of the user interface for video games is relatively new and not well-researched. A solution would be exploring different sources for publishing studies. Google Scholar offers a comprehensive source for peer-reviewed journals and articles, and it is also worth investigating studies published in different countries and languages. However, given enough papers on this topic, a remaining significant challenge in conducting this literature review was the heterogeneity of studies in terms of research methods, design approaches, and platforms under consideration. This made it difficult to compare and synthesize results across studies. In addition, the limited amount of empirical research focusing specifically on human factors design and multi-sensory user interfaces in video games made it challenging to draw robust conclusions. Much of the literature was based on anecdotal evidence, case studies, or expert opinions, which limited the generalizability of the findings. Most quantitative studies included less than 50 participants in their studies. To address this issue, a systematic literature search using well-defined inclusion and exclusion criteria was employed. This approach allowed for the identification and inclusion of high-quality studies that met the research objectives.

In addition, the rapid pace of technological advancements in the gaming industry posed challenges in identifying studies that were still relevant and up-to-date. This required a constant review of recent publications and an assessment of their relevance to the current state of technology. To stay abreast of the latest developments in the field, it is essential to regularly update the literature review by incorporating recent studies and technological advancements. This will help maintain the relevance of the review and provide up-to-date insights.

Discussion

The findings of this literature review highlight the importance of human factors principles in video game design, as they significantly influence players' emotional experiences and overall satisfaction. The four critical Human-Centered Design processes, including understanding the context of use, specifying user requirements, producing design solutions, and evaluating the design, play crucial roles in creating an enjoyable and immersive gaming experience.

The study also emphasizes the need to consider the unique characteristics and limitations of different gaming platforms such as PC, console, and mobile when designing user interfaces. By understanding these platform-specific features, designers can optimize human factors in video game design, ultimately enhancing the gaming experience for players across all platforms. 

In terms of user interfaces, the study examines the importance of auditory, visual, and haptic feedback in providing an interactive and engaging experience for players. By following human factors principles, these interface mediums can be tailored to meet players' desires, necessities, and core motivations. 

The challenges faced by designers in the context of pre-learned interaction models, usability assessment, and playability heuristics are also examined. These challenges underscore the need for a deep understanding of players' prior learning, expectations, and initial behaviors, as well as the unique nature of usability assessment in the gaming industry. Finally, the study on affective user-centered design in video games offers 15 recommendations, covering various aspects of game design such as diversity of users, challenging gameplay, impressive visuals, creative gameplay, flow in games, good gaming interface, and difficulties and remedies. These recommendations serve as a valuable resource for designers seeking to create engaging and enjoyable gaming experiences that align with the preferences and expectations of their intended users.

Maturity of the Topic

The research topic, “Human Factors Design of Multi-sensory User Interfaces in Video Games Across Platforms,” is a relatively mature field of study, with its roots dating back to the early days of video gaming. However, as video game technology and design have evolved, so too has the research in this area. Multi-sensory User Interfaces are interfaces that combine multiple input modalities (such as keyboard, mouse, touch, voice, etc.) and output modalities (such as visual, auditory, haptic feedback, etc.) to provide a seamless and immersive experience for the player. Research on user interface in video games has been growing steadily since the 2000s, as developers have experimented with novel ways of integrating multiple modalities to create more immersive gameplay experiences. 

With the proliferation of gaming platforms like PC, consoles, mobile devices, and virtual reality systems, the study of user interfaces across these different platforms has become increasingly important. Researchers have been comparing and contrasting the unique challenges and opportunities presented by each platform, as well as identifying best practices for designing interfaces that can be easily adapted for different devices. This area of research has gained momentum over the past two decades and continues to evolve as new gaming platforms emerge.

While substantial progress has been made over the years, ongoing advancements in technology and gaming platforms, as well as the growing emphasis on usability and accessibility, ensure that this field will continue to develop and expand in the coming years.

Future Directions

Cross-platform Design Strategies

As gaming platforms continue to diversify, there is an increasing need for more effective cross-platform design strategies. Future research should focus on developing guidelines and best practices for designing user interfaces that can adapt seamlessly across different platforms while maintaining a consistent user experience.

Inclusive and Accessible Design

Inclusivity and accessibility should be key considerations in future research on user interface design for video games. This involves conducting research on how to cater to gamers with varying abilities, cultural backgrounds, and preferences, ensuring that games are enjoyable and accessible to a wider audience. This might include the development of adaptable user interfaces, alternative control schemes, and customizable difficulty settings.

Evaluation and Validation of Design Principles

In order to establish effective design principles for video game user interfaces, future research should prioritize the evaluation and validation of these principles through rigorous testing and user feedback. This will ensure that the design recommendations are grounded in empirical evidence and can contribute to creating more effective and user-friendly gaming experiences.

Conclusion

In conclusion, this paper provides a comprehensive literature review of the human factors design of Multi-sensory User Interfaces in Video Games Across Platforms. The effective implementation of human factors design principles is crucial in the development of successful video games that provide enjoyable and immersive experiences for players. By carefully considering user requirements, designing tailored solutions, and evaluating outcomes, designers can create games that resonate with their target audience and evoke positive emotions. Moreover, the paper emphasizes the importance of understanding the unique characteristics and limitations of different gaming platforms, as these factors directly impact user interface and user experience design. Furthermore, a well-designed video game user interface incorporating visual, auditory, and haptic feedback can significantly impact the gaming experience by enhancing player performance, immersion, and satisfaction. While each interface medium has its unique strengths, a multi-sensory approach is generally preferred by players, as evidenced by Nesbitt and Hoskens' study. Ultimately, striking the right balance between aesthetic appeal, usability, and immersion is crucial for creating a successful user interface that caters to players' diverse needs and preferences. Finally, creating a successful video game requires a deep understanding of user preferences, capabilities, and pre-learned interaction models. By incorporating usability assessments and playability heuristics, designers can fine-tune their creations to deliver an engaging and enjoyable gaming experience. Although substantial progress has been made in this field, future research should focus on cross-platform design strategies, inclusive and accessible design, and the evaluation and validation of design principles to ensure continued growth and relevance in this rapidly evolving industry.




Introduction

Video games have become one of the most popular forms of entertainment in recent years, with the average consumer spending hundreds of hours in front of a console or computer. Video games are also known as computer games, and they are interactive electronic games that involve a user interface or input device, such as a joystick, controller, keyboard, or motion sensor, to produce audio and visual feedback as a result of the user's action. The user interface (UI) is the space for communication and interaction between humans and machines. The purpose of the user interface is for people to effectively control the machine and to receive feedback from the machine that helps them make decisions (Browne & Anand, 2012). As the gaming industry has grown and technology has advanced, the user interface of video games now involves many aspects, such as the use of images, texts, audio, and video. With that said, the user interface is virtual for the success of the video game, and this research will provide a comprehensive literature review for the application of multimedia UI in video games across platforms, benefiting future game makers.   

First of all, this paper discusses the human factors design process in video game development, highlighting the critical steps involved in understanding user requirements, designing effective solutions, and evaluating design outcomes. Furthermore, the paper examines the unique challenges and considerations for designing user interfaces and user experiences across various gaming platforms, such as PC, console, and mobile devices. By utilizing a combination of visual, auditory, and haptic feedback, designers can create an interactive visual system that aids gamers in decision-making and enhance player experience throughout gameplay. This paper will discuss the importance of visual, auditory, and haptic interfaces in video games, exploring their individual and combined effects on player performance, immersion, and satisfaction. In addition, this paper delves into the challenges faced by designers and explores potential tools and strategies to evaluate and enhance design effectiveness in the gaming industry. Designers and developers must consider various factors, such as user experience, human factors design, and usability, to gauge the effectiveness of their creations. Striking a balance between innovation and user expectations, while assessing and improving playability, is vital to a game's success.

The proposed research will provide valuable insights for game developers and designers, and it will help to create better gaming experiences for all players. In addition, the author believes that this research could provide valuable insight into how game design strategies can be applied to non-game products. Through a better understanding of these strategies, designers are able to create products that are more efficient, engaging, and enjoyable to use. 

Goals of Research

The existing knowledge of multi-sensory user interface design for video games, with a focus on human-centered design, is primarily concentrated on specific platforms and particular types of games. Previous studies have conducted in-depth empirical evaluations on user interface design for mobile video games and multiplayer arcade games. However, the types of games examined are limited to simple scrolling shooter games and educational games (Micheloni et al., 2019). The goal of this research paper is to bridge this gap by comparing and contrasting the existing literature and establishing a comprehensive guide for game UI designers and developers to apply human-centered design principles across various platforms and via different media.

To achieve this, the paper will examine the commonalities and differences across games and platforms, focusing on three types of video games: (1) mobile and tablet video games, (2) console games with joysticks or controllers, and (3) personal computer (PC) video games. The similarities and differences in terms of user preference, design strategies and implementation, while prioritizing the needs and preferences of users, will be addressed for each platform.

Furthermore, this research will delve into the primary components of multi-sensory user interfaces in video games on different platforms, such as auditory, visual, and haptic interfaces. As user interface design for video games represents "a balancing act between aesthetics and mechanics" (Llanos & Jørgensen, 2011), it is crucial to consider both the clarity of information and the elegance of presentation within the context of each game. The paper will investigate optimal solutions for mobile, console, and PC video games that adhere to human-centered design principles, as well as common practices that lead to successful user interfaces.

Moreover, the paper will explore the extent to which multi-sensory user interfaces in video games, designed with a human-centered approach, affect player engagement and satisfaction. By understanding the impact of user interface design on these crucial factors, the paper discusses design challenges, guidelines, and recommendations, which aim to help developers and designers to make more informed decisions when creating video games that not only captivate players, but also facilitate enjoyable and immersive experiences that cater to the needs and desires of the target audience.

Method

The primary research approach for this paper involves conducting a comprehensive literature review of scientific publications and research studies focused on the design of multimedia user interfaces across different video game platforms. To ensure access to a wide range of relevant resources, the author will utilize Tisch Library's JumboSearch and Google Scholar through Tufts University. To maintain the quality and credibility of the research, only peer-reviewed resources will be considered, encompassing books, academic journals, and articles authored by experts in the field. 

The literature review will examine various aspects of user interface design in video games, such as games across platforms, and the application of auditory, visual, and haptic user interfaces. Moreover, the paper would cover design challenges, means of measuring successful design, and recommendations for game designers. The goal is to synthesize the existing body of knowledge to identify trends, challenges, and opportunities within the field. 

It is important to note that this paper will solely rely on secondary research methods, and no primary research activities, such as observations, interviews, or surveys, will be carried out. By focusing on a thorough analysis of the existing literature, the author aims to contribute to the understanding of multimedia user interface design in video games and provide valuable insights for future research and development in the area. 

Findings of Research

Human Factors Design Process

Video games represent a category of software where the user's emotional experience is of utmost importance. A game's success largely depends on its ability to evoke positive emotions in players. While individuals play video games for various reasons, the primary motivation for most players is the pursuit of positive emotions. These emotions can manifest in numerous ways, such as satisfaction, a sense of accomplishment, amusement, or excitement (Johnson & Wiles, 2003). In the realm of video game design, designers employ human factors principles to concentrate on users, their needs, and demands, while utilizing human factors and usability methodologies. This approach boosts effectiveness and efficiency, enhances user well-being, satisfaction, accessibility, and sustainability, and mitigates potential negative impacts on human health, safety, and performance resulting from the use (Johnson & Wiles, 2003).

During the design of interactive systems like video games, there are four critical Human-Centered Design processes that should be carried out. Firstly, understanding and specifying the context of use involves recognizing the game's playing environment and the target audience's characteristics. Secondly, specifying user requirements entails identifying users' needs and wants from the game, which can be achieved through methods such as surveys, interviews, and focus groups. Thirdly, producing design solutions involves creating concepts that address the previously identified user requirements. To test their ideas, designers may develop prototypes or mockups. Lastly, evaluating the design requires testing it with users to determine if it meets their needs and is user-friendly. Designers may employ methods like usability testing or heuristic evaluation to assess the design's effectiveness (Johnson & Wiles, 2003).

Despite video games' popularity and immense revenue, progress in studying their effects and potential has been relatively slow. Video games fall within the realm of Human-Computer Interaction (HCI) research, but they differ from traditional HCI's focus, which is on productivity applications that facilitate user-defined tasks. Games, on the other hand, often assign tasks to players, deliberately making them challenging (Barr et al., 2007). 

The study concludes that “the defining characteristic of video games is the presence of value systems, interconnected values that guide gameplay” (Barr et al., 2007). In video games, there are typically "right" or "correct" ways to proceed, and user interfaces play a significant role in both shaping and conveying the value. Thus, examining video game interfaces can be rooted in understanding values and value systems and their connection to gameplay. A solid foundation in established theories of behavior and gameplay, in addition to in-depth analyses of actual video games using qualitative methods, is crucial. Through such analyses, scholars can comprehend the inner workings, effects, and potential of video games (Barr et al., 2007).

Video games Across Platforms

Incorporating human factors principles in video game design is crucial for creating an enjoyable and immersive gaming experience. To achieve this, designers must pay close attention to visual cues, minimize user errors, and facilitate comfortable interactions between players and the game. The user interface (UI) and user experience design (UXD) are integral components of a successful game. The user interface is responsible for ensuring smooth task execution and effective communication between the game and the player.  User experience design, on the other hand, focuses on delivering an engaging and enjoyable experience that keeps players invested in the game.

Each gaming platform, such as PC, console, and mobile, has its own unique characteristics and limitations, which have a direct impact on UI design. PC gaming is known for its versatility in terms of specifications and variety, as it can support a wide range of games without restrictions. Console gaming is more limited, as it can only play games specifically designed for the console. Mobile gaming, while convenient, has inferior specifications compared to the other two platforms. Despite these differences, each platform has its own strengths. Mobile gaming platforms excel in privacy and comfort, allowing players to enjoy games anywhere and anytime. PC gaming, however, provides the most satisfying gaming experience due to its adaptability and expansive game library (Galehantomo, 2015).

When designing UI for different platforms, it is essential to consider their specific features, such as input options and viewing distances. Console gaming relies on controllers with limited input options, while PC gaming offers more versatile inputs through mouse and keyboard. Additionally, console players generally sit further away from their screens, using TVs, whereas PC players sit closer to their monitors. These factors must be taken into account to optimize human factors in video game design, ultimately enhancing the gaming experience for players across all platforms (Galehantomo, 2015).

Types of User Interface

The user interface in video games plays a vital role in enabling interaction and conveying information to players. In gaming, UI design can be defined as an interactive visual system that offers information about potential actions or objects, aiding gamers in decision-making throughout gameplay via various user inputs. Multiple interface mediums are used in video games, including auditory, visual, and haptic feedback. By following human factors principles, these mediums are tailored to meet players' desires, necessities, and core motivations while they engage in gaming.

Overall, gamers tend to have improved experiences with multi-sensory displays and feel more confident in their performance when utilizing them, even if their actual performance remains unchanged (Nesbitt & Hoskens, 2008). The following sections will delve into a detailed examination of the three types of interfaces utilized in video games.

Visual Interface

Visual elements, such as on-screen instructions or maps, help users follow video game actions. Human factors principles can be applied to ensure visual elements are intuitive and user-friendly. As shown in Table 1, game UI design has four visual representation types: diegetic, heads-up displays (HUDs), spatial, and meta (Peacocke, 2018). 

  • Diegetic elements are visible or audible to characters and fit within the story context. 

  • heads-up displays (HUDs) exist outside the game or story and are only visible to players. 

  • Spatial elements are represented within the game space but aren't visible to characters. 

  • Meta elements are contextual to the game but aren't represented within the game space.


Drawing from several studies on different video games, the arthur discusses the implications of visual interfaces in video games, covering different aspects of the game, including the aesthetics, usability, and overall user satisfaction. A study by Llanos and Jørgensen examines scrolling text or colored overlays used to signify changes in a player's health status. This initial analysis reveals that the concept of "immersive" or minimal interfaces, which aim to incorporate all system information within the game world and present it as part of the narrative, is not always the best approach. Although players often find minimal UI aesthetically pleasing, they prefer sufficient information to interact meaningfully with the game mechanics and world. On the other hand, excess information can be annoying and disrupt immersion. Consequently, UI designers should present gameplay-relevant information clearly and precisely, consider the desired game experience, and reflect on specific UI approaches to achieve it (Llanos & Jørgensen, 2011).

In another study, Browne and Anand (2012) aimed to develop design heuristics for visual user interfaces by examining a specific mobile video game. While there are limitations of this study, including a small, non-diverse sample size and a focus on a single game type, the study identified five key design heuristics, which can be applied to create more engaging and user-friendly mobile user interfaces. These include offering an accelerometer-based interface, providing multiple user interfaces, using touch gestures for diagonal direction input, allowing configurable interface sensitivity, and directly translating physical gesture properties into virtual properties (Browne & Anand, 2012). By incorporating these heuristics, developers can improve the user experience for a range of applications, including video games of different types and on different platforms, as long as there are similar input requirements.

Finally, the study by Charbonneau and others highlight the importance of human factors in game design, specifically focusing on the effectiveness of different visual interfaces. The research paper focused on conveying dance sequence information for full-body rhythm dance games. By considering the user experience, the research emphasizes the critical role visuals play in shaping player performance and satisfaction. The findings reveal that Motion Lines and Beat Circles not only led to higher scores but also provided a more intuitive and less confusing experience compared to the conventional Timeline interface. This underscores the significance of carefully designing visual interfaces that cater to human factors, ultimately enhancing the overall gaming experience (Charbonneau et al., 2009).

Auditory Interface

In addition to the visual interface in video games, another key component is the auditory interface. Key audio elements in games include “vocalizations of game characters, sound effects relating to gameplay, ambient effects relating to atmosphere, and the music of the game” (Amiriparian et al., 2020). The blend of these audio elements within a specific game is closely tied to the gameplay mechanics and game genre. For instance, action and shooting games like the Call of Duty series feature intense, abrupt sounds such as punches and gunshots. On the other hand, sports games like the FIFA football series include continuous commentary voice-overs. Finally, racing games like the Forza series are characterized by prominent car noises, including powerful accelerations and squealing brakes (Amiriparian et al., 2020). Audio interface in games is powerful as it sets the tone for the game, and it also heavily impacts users’ immersion and emotional engagement. 

Furthermore, a study by Nesbitt and Hoskens focused on a similar topic, investigating the impact of game audio on immersion and emotional engagement. They conducted experiments to explore the significance of various audio elements and their compatibility within video games. Findings indicate that sound effects are the most crucial audio elements. The experiments reveal that game audio influences players' gameplay experiences in multiple ways, emphasizing audio's importance in video games. Although audio elements could be subtle and hardly noticeable for players, minor details such as gunshot sounds or enemy footsteps to background music alters player emotions (Nesbitt & Hoskens, 2008). 

Moreover, audio not only affects gameplay but also enhances players' satisfaction in performing actions, based on participant feedback. Game audio, particularly sound effects, is vital for creating an enjoyable experience. Designers and developers should consider aspects beyond gameplay. Nesbitt and Hoskensuture suggested that future research could utilize more specific questionnaire questions, such as each participant's gaming experience, and examine minor variables like playing the same type of game with different level designs. Audio cues can provide feedback on player actions or convey essential information about the game world, and human factors principles can be applied to ensure clarity and comprehension (Nesbitt & Hoskens, 2008).

Haptic Interface

A haptic interface describes the generated force in response to interactions within a virtual world (Park et al., 2004). In addition to video games, haptic devices have applications in various fields, such as telerobotics, military, medical, and entertainment systems. Traditional video games primarily relied on visual and auditory senses, using keyboards or joysticks for user interaction. However, when appropriate forces are applied based on factors such as collision speed and material properties in the game environment, alongside visuals and sound, players can experience a more realistic and immersive gaming experience (Park et al., 2004). Haptic interfaces are now widely used in shooters games, arcade games, and exergames. 

As mentioned above, haptic feedback conveys information through applied forces and vibrations, either as tactile (touch, texture, vibration) or kinesthetic (force, weight, resistance) feedback (Stach & Graham, 2011). Haptics have been used to improve interactions in various contexts, such as enhancing task performance and virtual presence in distributed groupware systems, and decreasing reaction times in various game genres. According to Stach and Graham, haptic feedback can address three critical exergame design areas: balancing group exercise for different physical abilities, guiding players to safe and healthy interaction levels, and providing immersive experiences to increase physical activity enjoyment. Haptic feedback enhances immersion by offering tactile sensations corresponding to in-game events. Designers can use human factors principles to ensure haptic feedback is effective and doesn't overwhelm or confuse players (Stach & Graham, 2011).

The previous sections discussed the application and importance of visual, auditory, and haptic interfaces. The question now is, from a human factors perspective, which one of them is most important in video games? Does a combination of all three work best? As the study by Nesbitt and Hoskens suggests, in general, players reported an enhanced experience when they were provided with multiple sensory cues. The combination of visual, auditory, and haptic feedback received the highest ratings in terms of immersion, confidence, and satisfaction. When comparing displays with a single extra sensory cue, players preferred visual/auditory over visual/haptic interfaces. In addition, most players emphasized the importance of sound in the gaming experience, mentioning reasons such as blocking external distractions and assisting in anticipating in-game movement and combat areas. Although some found haptic feedback is a significant factor to the game experience, it was not seen as useful for gameplay as auditory cues. These preferences were consistent across the three experience criteria and were supported by player comments (Nesbitt & Hoskens, 2008). The authors suggest that the combination methodology could also be useful for game designers who wish to use multi-sensory displays to provide a greater spectrum of information levels to the player. Better design of the way information is displayed to the different senses could be used to improve player performance, or alternatively to make some game tasks more challenging (Nesbitt & Hoskens, 2008).

Challenges and Measuring Success

The previous sections examined the state-of-art technologies for video games and the different types of interfaces that are frequently used, providing a toolbox for designers and developers. Beyond that, a more important question is, how do designers and developers measure the success of the design? How do designers know whether users will like, and stick to the game? Human factors design also plays a role in the evaluation. The constant evolution of technologies and design strategies presents new challenges for designers, requiring them to balance innovation with user expectations, conduct usability assessments, and employ playability heuristics. In this section of the paper, the author will discuss a few challenges for designers, and potential tools to evaluate design effectiveness.

Pre-learned Interaction Model

The first challenge for designers is that players usually have a pre-learned interaction model when they encounter a new game. Hence, players would attempt to interact with the game based on this model, and if a game doesn't accommodate these models, players can become confused and frustrated. As discussed in the paper “Facing the Human Factors Challenges in Game Design: A Discussion Panel”, the visual design, along with auditory and visual cues, can help guide players' interaction models (Portnoy et al., 2011). 

More specifically, Microsoft Game experiences excel when players can instantly access the experience without barriers, which requires a deep understanding of players' prior learning, expectations, and initial behaviors. However, it was mentioned that some games still need to push players to adjust their models, risking a low retention rate. It is a challenge for game designers to strike the perfect balance between making the game creative and original, and adhering to users’ pre-learned interaction models (Portnoy et al., 2011). 

Usability Assessment

In the gaming industry, usability assessment is characterized by its focus on user purpose and the role of the usability process. Video games emphasize entertainment and players' emotional states, like engagement and fun, over efficiency, whereas in other industries, the priority is task efficiency and user productivity (Portnoy et al., 2011). The usability process in gaming not only seeks to comprehend user experience but also aids in the iterative game design process. It is a useful tool for designers to evaluate design effectiveness.

The distinct nature of usability assessment in gaming affects the methods employed, such as psychophysiological techniques that continuously and objectively gauge players' emotional states without disrupting gameplay. Developers can generate emotional profiles to assess whether game features provoke the desired emotional response. In addition, eye tracking is another method that documents players' gaze patterns to evaluate attention focus and comprehend how visual information is processed in games (Portnoy et al., 2011).

Playability Heuristics 

Finally, the study by Soomro and others utilized two established playability heuristics to evaluate six Android games across various genres with fourteen test participants. Findings from the study indicate that the current playability heuristics fall short in identifying playability issues, such as “usability, mobility, gameplay, and multiplayer functionality” (Soomro et al., 2013) for touchscreen mobile game applications, as indicated in Table 2 below. While this list is not meant to be exhaustive, it provides meaningful metrics for video game design across platforms.  

Design Recommendations

Finally, the study "Evaluating Affective User-Centered Design of Video Games Using Qualitative Methods" by Ng and colleagues helped to establish a comprehensive set of guidelines for user-centered design in the context of video games. Affective user-centered design is an approach that assesses whether the emotions, meanings, and mental concepts conveyed through a video game's tangible and intangible elements align with the preferences and expectations of the intended users. In their research, Ng and others employed qualitative methods to thoroughly analyze data and identify key factors that contribute to an engaging and enjoyable gaming experience. As a result of their analysis, they proposed 15 recommendations for user-centered design in video games. These recommendations cover various aspects of game design. Note that the following list is adapted from the research paper by Ng et al.

  1. Diversity of Users

  2. User Capability

  3. Challenging Gameplay

  4. Provide Goals

  5. Provide Rewards

  6. Impressive Visuals

  7. Graphic Quality

  8. Creative Gameplay

  9. Good Narrative

  10. Interesting Game Characters

  11. Flow in Games

  12. Flexible Options

  13. Good Gaming Interface

  14. Tutorial and Hints

  15. Easy-to-Use Controls

Difficulties and Remedies

One difficulty is finding relevant literature on the topic. This can be especially challenging since the topic of the user interface for video games is relatively new and not well-researched. A solution would be exploring different sources for publishing studies. Google Scholar offers a comprehensive source for peer-reviewed journals and articles, and it is also worth investigating studies published in different countries and languages. However, given enough papers on this topic, a remaining significant challenge in conducting this literature review was the heterogeneity of studies in terms of research methods, design approaches, and platforms under consideration. This made it difficult to compare and synthesize results across studies. In addition, the limited amount of empirical research focusing specifically on human factors design and multi-sensory user interfaces in video games made it challenging to draw robust conclusions. Much of the literature was based on anecdotal evidence, case studies, or expert opinions, which limited the generalizability of the findings. Most quantitative studies included less than 50 participants in their studies. To address this issue, a systematic literature search using well-defined inclusion and exclusion criteria was employed. This approach allowed for the identification and inclusion of high-quality studies that met the research objectives.

In addition, the rapid pace of technological advancements in the gaming industry posed challenges in identifying studies that were still relevant and up-to-date. This required a constant review of recent publications and an assessment of their relevance to the current state of technology. To stay abreast of the latest developments in the field, it is essential to regularly update the literature review by incorporating recent studies and technological advancements. This will help maintain the relevance of the review and provide up-to-date insights.

Discussion

The findings of this literature review highlight the importance of human factors principles in video game design, as they significantly influence players' emotional experiences and overall satisfaction. The four critical Human-Centered Design processes, including understanding the context of use, specifying user requirements, producing design solutions, and evaluating the design, play crucial roles in creating an enjoyable and immersive gaming experience.

The study also emphasizes the need to consider the unique characteristics and limitations of different gaming platforms such as PC, console, and mobile when designing user interfaces. By understanding these platform-specific features, designers can optimize human factors in video game design, ultimately enhancing the gaming experience for players across all platforms. 

In terms of user interfaces, the study examines the importance of auditory, visual, and haptic feedback in providing an interactive and engaging experience for players. By following human factors principles, these interface mediums can be tailored to meet players' desires, necessities, and core motivations. 

The challenges faced by designers in the context of pre-learned interaction models, usability assessment, and playability heuristics are also examined. These challenges underscore the need for a deep understanding of players' prior learning, expectations, and initial behaviors, as well as the unique nature of usability assessment in the gaming industry. Finally, the study on affective user-centered design in video games offers 15 recommendations, covering various aspects of game design such as diversity of users, challenging gameplay, impressive visuals, creative gameplay, flow in games, good gaming interface, and difficulties and remedies. These recommendations serve as a valuable resource for designers seeking to create engaging and enjoyable gaming experiences that align with the preferences and expectations of their intended users.

Maturity of the Topic

The research topic, “Human Factors Design of Multi-sensory User Interfaces in Video Games Across Platforms,” is a relatively mature field of study, with its roots dating back to the early days of video gaming. However, as video game technology and design have evolved, so too has the research in this area. Multi-sensory User Interfaces are interfaces that combine multiple input modalities (such as keyboard, mouse, touch, voice, etc.) and output modalities (such as visual, auditory, haptic feedback, etc.) to provide a seamless and immersive experience for the player. Research on user interface in video games has been growing steadily since the 2000s, as developers have experimented with novel ways of integrating multiple modalities to create more immersive gameplay experiences. 

With the proliferation of gaming platforms like PC, consoles, mobile devices, and virtual reality systems, the study of user interfaces across these different platforms has become increasingly important. Researchers have been comparing and contrasting the unique challenges and opportunities presented by each platform, as well as identifying best practices for designing interfaces that can be easily adapted for different devices. This area of research has gained momentum over the past two decades and continues to evolve as new gaming platforms emerge.

While substantial progress has been made over the years, ongoing advancements in technology and gaming platforms, as well as the growing emphasis on usability and accessibility, ensure that this field will continue to develop and expand in the coming years.

Future Directions

Cross-platform Design Strategies

As gaming platforms continue to diversify, there is an increasing need for more effective cross-platform design strategies. Future research should focus on developing guidelines and best practices for designing user interfaces that can adapt seamlessly across different platforms while maintaining a consistent user experience.

Inclusive and Accessible Design

Inclusivity and accessibility should be key considerations in future research on user interface design for video games. This involves conducting research on how to cater to gamers with varying abilities, cultural backgrounds, and preferences, ensuring that games are enjoyable and accessible to a wider audience. This might include the development of adaptable user interfaces, alternative control schemes, and customizable difficulty settings.

Evaluation and Validation of Design Principles

In order to establish effective design principles for video game user interfaces, future research should prioritize the evaluation and validation of these principles through rigorous testing and user feedback. This will ensure that the design recommendations are grounded in empirical evidence and can contribute to creating more effective and user-friendly gaming experiences.

Conclusion

In conclusion, this paper provides a comprehensive literature review of the human factors design of Multi-sensory User Interfaces in Video Games Across Platforms. The effective implementation of human factors design principles is crucial in the development of successful video games that provide enjoyable and immersive experiences for players. By carefully considering user requirements, designing tailored solutions, and evaluating outcomes, designers can create games that resonate with their target audience and evoke positive emotions. Moreover, the paper emphasizes the importance of understanding the unique characteristics and limitations of different gaming platforms, as these factors directly impact user interface and user experience design. Furthermore, a well-designed video game user interface incorporating visual, auditory, and haptic feedback can significantly impact the gaming experience by enhancing player performance, immersion, and satisfaction. While each interface medium has its unique strengths, a multi-sensory approach is generally preferred by players, as evidenced by Nesbitt and Hoskens' study. Ultimately, striking the right balance between aesthetic appeal, usability, and immersion is crucial for creating a successful user interface that caters to players' diverse needs and preferences. Finally, creating a successful video game requires a deep understanding of user preferences, capabilities, and pre-learned interaction models. By incorporating usability assessments and playability heuristics, designers can fine-tune their creations to deliver an engaging and enjoyable gaming experience. Although substantial progress has been made in this field, future research should focus on cross-platform design strategies, inclusive and accessible design, and the evaluation and validation of design principles to ensure continued growth and relevance in this rapidly evolving industry.




Introduction

Video games have become one of the most popular forms of entertainment in recent years, with the average consumer spending hundreds of hours in front of a console or computer. Video games are also known as computer games, and they are interactive electronic games that involve a user interface or input device, such as a joystick, controller, keyboard, or motion sensor, to produce audio and visual feedback as a result of the user's action. The user interface (UI) is the space for communication and interaction between humans and machines. The purpose of the user interface is for people to effectively control the machine and to receive feedback from the machine that helps them make decisions (Browne & Anand, 2012). As the gaming industry has grown and technology has advanced, the user interface of video games now involves many aspects, such as the use of images, texts, audio, and video. With that said, the user interface is virtual for the success of the video game, and this research will provide a comprehensive literature review for the application of multimedia UI in video games across platforms, benefiting future game makers.   

First of all, this paper discusses the human factors design process in video game development, highlighting the critical steps involved in understanding user requirements, designing effective solutions, and evaluating design outcomes. Furthermore, the paper examines the unique challenges and considerations for designing user interfaces and user experiences across various gaming platforms, such as PC, console, and mobile devices. By utilizing a combination of visual, auditory, and haptic feedback, designers can create an interactive visual system that aids gamers in decision-making and enhance player experience throughout gameplay. This paper will discuss the importance of visual, auditory, and haptic interfaces in video games, exploring their individual and combined effects on player performance, immersion, and satisfaction. In addition, this paper delves into the challenges faced by designers and explores potential tools and strategies to evaluate and enhance design effectiveness in the gaming industry. Designers and developers must consider various factors, such as user experience, human factors design, and usability, to gauge the effectiveness of their creations. Striking a balance between innovation and user expectations, while assessing and improving playability, is vital to a game's success.

The proposed research will provide valuable insights for game developers and designers, and it will help to create better gaming experiences for all players. In addition, the author believes that this research could provide valuable insight into how game design strategies can be applied to non-game products. Through a better understanding of these strategies, designers are able to create products that are more efficient, engaging, and enjoyable to use. 

Goals of Research

The existing knowledge of multi-sensory user interface design for video games, with a focus on human-centered design, is primarily concentrated on specific platforms and particular types of games. Previous studies have conducted in-depth empirical evaluations on user interface design for mobile video games and multiplayer arcade games. However, the types of games examined are limited to simple scrolling shooter games and educational games (Micheloni et al., 2019). The goal of this research paper is to bridge this gap by comparing and contrasting the existing literature and establishing a comprehensive guide for game UI designers and developers to apply human-centered design principles across various platforms and via different media.

To achieve this, the paper will examine the commonalities and differences across games and platforms, focusing on three types of video games: (1) mobile and tablet video games, (2) console games with joysticks or controllers, and (3) personal computer (PC) video games. The similarities and differences in terms of user preference, design strategies and implementation, while prioritizing the needs and preferences of users, will be addressed for each platform.

Furthermore, this research will delve into the primary components of multi-sensory user interfaces in video games on different platforms, such as auditory, visual, and haptic interfaces. As user interface design for video games represents "a balancing act between aesthetics and mechanics" (Llanos & Jørgensen, 2011), it is crucial to consider both the clarity of information and the elegance of presentation within the context of each game. The paper will investigate optimal solutions for mobile, console, and PC video games that adhere to human-centered design principles, as well as common practices that lead to successful user interfaces.

Moreover, the paper will explore the extent to which multi-sensory user interfaces in video games, designed with a human-centered approach, affect player engagement and satisfaction. By understanding the impact of user interface design on these crucial factors, the paper discusses design challenges, guidelines, and recommendations, which aim to help developers and designers to make more informed decisions when creating video games that not only captivate players, but also facilitate enjoyable and immersive experiences that cater to the needs and desires of the target audience.

Method

The primary research approach for this paper involves conducting a comprehensive literature review of scientific publications and research studies focused on the design of multimedia user interfaces across different video game platforms. To ensure access to a wide range of relevant resources, the author will utilize Tisch Library's JumboSearch and Google Scholar through Tufts University. To maintain the quality and credibility of the research, only peer-reviewed resources will be considered, encompassing books, academic journals, and articles authored by experts in the field. 

The literature review will examine various aspects of user interface design in video games, such as games across platforms, and the application of auditory, visual, and haptic user interfaces. Moreover, the paper would cover design challenges, means of measuring successful design, and recommendations for game designers. The goal is to synthesize the existing body of knowledge to identify trends, challenges, and opportunities within the field. 

It is important to note that this paper will solely rely on secondary research methods, and no primary research activities, such as observations, interviews, or surveys, will be carried out. By focusing on a thorough analysis of the existing literature, the author aims to contribute to the understanding of multimedia user interface design in video games and provide valuable insights for future research and development in the area. 

Findings of Research

Human Factors Design Process

Video games represent a category of software where the user's emotional experience is of utmost importance. A game's success largely depends on its ability to evoke positive emotions in players. While individuals play video games for various reasons, the primary motivation for most players is the pursuit of positive emotions. These emotions can manifest in numerous ways, such as satisfaction, a sense of accomplishment, amusement, or excitement (Johnson & Wiles, 2003). In the realm of video game design, designers employ human factors principles to concentrate on users, their needs, and demands, while utilizing human factors and usability methodologies. This approach boosts effectiveness and efficiency, enhances user well-being, satisfaction, accessibility, and sustainability, and mitigates potential negative impacts on human health, safety, and performance resulting from the use (Johnson & Wiles, 2003).

During the design of interactive systems like video games, there are four critical Human-Centered Design processes that should be carried out. Firstly, understanding and specifying the context of use involves recognizing the game's playing environment and the target audience's characteristics. Secondly, specifying user requirements entails identifying users' needs and wants from the game, which can be achieved through methods such as surveys, interviews, and focus groups. Thirdly, producing design solutions involves creating concepts that address the previously identified user requirements. To test their ideas, designers may develop prototypes or mockups. Lastly, evaluating the design requires testing it with users to determine if it meets their needs and is user-friendly. Designers may employ methods like usability testing or heuristic evaluation to assess the design's effectiveness (Johnson & Wiles, 2003).

Despite video games' popularity and immense revenue, progress in studying their effects and potential has been relatively slow. Video games fall within the realm of Human-Computer Interaction (HCI) research, but they differ from traditional HCI's focus, which is on productivity applications that facilitate user-defined tasks. Games, on the other hand, often assign tasks to players, deliberately making them challenging (Barr et al., 2007). 

The study concludes that “the defining characteristic of video games is the presence of value systems, interconnected values that guide gameplay” (Barr et al., 2007). In video games, there are typically "right" or "correct" ways to proceed, and user interfaces play a significant role in both shaping and conveying the value. Thus, examining video game interfaces can be rooted in understanding values and value systems and their connection to gameplay. A solid foundation in established theories of behavior and gameplay, in addition to in-depth analyses of actual video games using qualitative methods, is crucial. Through such analyses, scholars can comprehend the inner workings, effects, and potential of video games (Barr et al., 2007).

Video games Across Platforms

Incorporating human factors principles in video game design is crucial for creating an enjoyable and immersive gaming experience. To achieve this, designers must pay close attention to visual cues, minimize user errors, and facilitate comfortable interactions between players and the game. The user interface (UI) and user experience design (UXD) are integral components of a successful game. The user interface is responsible for ensuring smooth task execution and effective communication between the game and the player.  User experience design, on the other hand, focuses on delivering an engaging and enjoyable experience that keeps players invested in the game.

Each gaming platform, such as PC, console, and mobile, has its own unique characteristics and limitations, which have a direct impact on UI design. PC gaming is known for its versatility in terms of specifications and variety, as it can support a wide range of games without restrictions. Console gaming is more limited, as it can only play games specifically designed for the console. Mobile gaming, while convenient, has inferior specifications compared to the other two platforms. Despite these differences, each platform has its own strengths. Mobile gaming platforms excel in privacy and comfort, allowing players to enjoy games anywhere and anytime. PC gaming, however, provides the most satisfying gaming experience due to its adaptability and expansive game library (Galehantomo, 2015).

When designing UI for different platforms, it is essential to consider their specific features, such as input options and viewing distances. Console gaming relies on controllers with limited input options, while PC gaming offers more versatile inputs through mouse and keyboard. Additionally, console players generally sit further away from their screens, using TVs, whereas PC players sit closer to their monitors. These factors must be taken into account to optimize human factors in video game design, ultimately enhancing the gaming experience for players across all platforms (Galehantomo, 2015).

Types of User Interface

The user interface in video games plays a vital role in enabling interaction and conveying information to players. In gaming, UI design can be defined as an interactive visual system that offers information about potential actions or objects, aiding gamers in decision-making throughout gameplay via various user inputs. Multiple interface mediums are used in video games, including auditory, visual, and haptic feedback. By following human factors principles, these mediums are tailored to meet players' desires, necessities, and core motivations while they engage in gaming.

Overall, gamers tend to have improved experiences with multi-sensory displays and feel more confident in their performance when utilizing them, even if their actual performance remains unchanged (Nesbitt & Hoskens, 2008). The following sections will delve into a detailed examination of the three types of interfaces utilized in video games.

Visual Interface

Visual elements, such as on-screen instructions or maps, help users follow video game actions. Human factors principles can be applied to ensure visual elements are intuitive and user-friendly. As shown in Table 1, game UI design has four visual representation types: diegetic, heads-up displays (HUDs), spatial, and meta (Peacocke, 2018). 

  • Diegetic elements are visible or audible to characters and fit within the story context. 

  • heads-up displays (HUDs) exist outside the game or story and are only visible to players. 

  • Spatial elements are represented within the game space but aren't visible to characters. 

  • Meta elements are contextual to the game but aren't represented within the game space.


Drawing from several studies on different video games, the arthur discusses the implications of visual interfaces in video games, covering different aspects of the game, including the aesthetics, usability, and overall user satisfaction. A study by Llanos and Jørgensen examines scrolling text or colored overlays used to signify changes in a player's health status. This initial analysis reveals that the concept of "immersive" or minimal interfaces, which aim to incorporate all system information within the game world and present it as part of the narrative, is not always the best approach. Although players often find minimal UI aesthetically pleasing, they prefer sufficient information to interact meaningfully with the game mechanics and world. On the other hand, excess information can be annoying and disrupt immersion. Consequently, UI designers should present gameplay-relevant information clearly and precisely, consider the desired game experience, and reflect on specific UI approaches to achieve it (Llanos & Jørgensen, 2011).

In another study, Browne and Anand (2012) aimed to develop design heuristics for visual user interfaces by examining a specific mobile video game. While there are limitations of this study, including a small, non-diverse sample size and a focus on a single game type, the study identified five key design heuristics, which can be applied to create more engaging and user-friendly mobile user interfaces. These include offering an accelerometer-based interface, providing multiple user interfaces, using touch gestures for diagonal direction input, allowing configurable interface sensitivity, and directly translating physical gesture properties into virtual properties (Browne & Anand, 2012). By incorporating these heuristics, developers can improve the user experience for a range of applications, including video games of different types and on different platforms, as long as there are similar input requirements.

Finally, the study by Charbonneau and others highlight the importance of human factors in game design, specifically focusing on the effectiveness of different visual interfaces. The research paper focused on conveying dance sequence information for full-body rhythm dance games. By considering the user experience, the research emphasizes the critical role visuals play in shaping player performance and satisfaction. The findings reveal that Motion Lines and Beat Circles not only led to higher scores but also provided a more intuitive and less confusing experience compared to the conventional Timeline interface. This underscores the significance of carefully designing visual interfaces that cater to human factors, ultimately enhancing the overall gaming experience (Charbonneau et al., 2009).

Auditory Interface

In addition to the visual interface in video games, another key component is the auditory interface. Key audio elements in games include “vocalizations of game characters, sound effects relating to gameplay, ambient effects relating to atmosphere, and the music of the game” (Amiriparian et al., 2020). The blend of these audio elements within a specific game is closely tied to the gameplay mechanics and game genre. For instance, action and shooting games like the Call of Duty series feature intense, abrupt sounds such as punches and gunshots. On the other hand, sports games like the FIFA football series include continuous commentary voice-overs. Finally, racing games like the Forza series are characterized by prominent car noises, including powerful accelerations and squealing brakes (Amiriparian et al., 2020). Audio interface in games is powerful as it sets the tone for the game, and it also heavily impacts users’ immersion and emotional engagement. 

Furthermore, a study by Nesbitt and Hoskens focused on a similar topic, investigating the impact of game audio on immersion and emotional engagement. They conducted experiments to explore the significance of various audio elements and their compatibility within video games. Findings indicate that sound effects are the most crucial audio elements. The experiments reveal that game audio influences players' gameplay experiences in multiple ways, emphasizing audio's importance in video games. Although audio elements could be subtle and hardly noticeable for players, minor details such as gunshot sounds or enemy footsteps to background music alters player emotions (Nesbitt & Hoskens, 2008). 

Moreover, audio not only affects gameplay but also enhances players' satisfaction in performing actions, based on participant feedback. Game audio, particularly sound effects, is vital for creating an enjoyable experience. Designers and developers should consider aspects beyond gameplay. Nesbitt and Hoskensuture suggested that future research could utilize more specific questionnaire questions, such as each participant's gaming experience, and examine minor variables like playing the same type of game with different level designs. Audio cues can provide feedback on player actions or convey essential information about the game world, and human factors principles can be applied to ensure clarity and comprehension (Nesbitt & Hoskens, 2008).

Haptic Interface

A haptic interface describes the generated force in response to interactions within a virtual world (Park et al., 2004). In addition to video games, haptic devices have applications in various fields, such as telerobotics, military, medical, and entertainment systems. Traditional video games primarily relied on visual and auditory senses, using keyboards or joysticks for user interaction. However, when appropriate forces are applied based on factors such as collision speed and material properties in the game environment, alongside visuals and sound, players can experience a more realistic and immersive gaming experience (Park et al., 2004). Haptic interfaces are now widely used in shooters games, arcade games, and exergames. 

As mentioned above, haptic feedback conveys information through applied forces and vibrations, either as tactile (touch, texture, vibration) or kinesthetic (force, weight, resistance) feedback (Stach & Graham, 2011). Haptics have been used to improve interactions in various contexts, such as enhancing task performance and virtual presence in distributed groupware systems, and decreasing reaction times in various game genres. According to Stach and Graham, haptic feedback can address three critical exergame design areas: balancing group exercise for different physical abilities, guiding players to safe and healthy interaction levels, and providing immersive experiences to increase physical activity enjoyment. Haptic feedback enhances immersion by offering tactile sensations corresponding to in-game events. Designers can use human factors principles to ensure haptic feedback is effective and doesn't overwhelm or confuse players (Stach & Graham, 2011).

The previous sections discussed the application and importance of visual, auditory, and haptic interfaces. The question now is, from a human factors perspective, which one of them is most important in video games? Does a combination of all three work best? As the study by Nesbitt and Hoskens suggests, in general, players reported an enhanced experience when they were provided with multiple sensory cues. The combination of visual, auditory, and haptic feedback received the highest ratings in terms of immersion, confidence, and satisfaction. When comparing displays with a single extra sensory cue, players preferred visual/auditory over visual/haptic interfaces. In addition, most players emphasized the importance of sound in the gaming experience, mentioning reasons such as blocking external distractions and assisting in anticipating in-game movement and combat areas. Although some found haptic feedback is a significant factor to the game experience, it was not seen as useful for gameplay as auditory cues. These preferences were consistent across the three experience criteria and were supported by player comments (Nesbitt & Hoskens, 2008). The authors suggest that the combination methodology could also be useful for game designers who wish to use multi-sensory displays to provide a greater spectrum of information levels to the player. Better design of the way information is displayed to the different senses could be used to improve player performance, or alternatively to make some game tasks more challenging (Nesbitt & Hoskens, 2008).

Challenges and Measuring Success

The previous sections examined the state-of-art technologies for video games and the different types of interfaces that are frequently used, providing a toolbox for designers and developers. Beyond that, a more important question is, how do designers and developers measure the success of the design? How do designers know whether users will like, and stick to the game? Human factors design also plays a role in the evaluation. The constant evolution of technologies and design strategies presents new challenges for designers, requiring them to balance innovation with user expectations, conduct usability assessments, and employ playability heuristics. In this section of the paper, the author will discuss a few challenges for designers, and potential tools to evaluate design effectiveness.

Pre-learned Interaction Model

The first challenge for designers is that players usually have a pre-learned interaction model when they encounter a new game. Hence, players would attempt to interact with the game based on this model, and if a game doesn't accommodate these models, players can become confused and frustrated. As discussed in the paper “Facing the Human Factors Challenges in Game Design: A Discussion Panel”, the visual design, along with auditory and visual cues, can help guide players' interaction models (Portnoy et al., 2011). 

More specifically, Microsoft Game experiences excel when players can instantly access the experience without barriers, which requires a deep understanding of players' prior learning, expectations, and initial behaviors. However, it was mentioned that some games still need to push players to adjust their models, risking a low retention rate. It is a challenge for game designers to strike the perfect balance between making the game creative and original, and adhering to users’ pre-learned interaction models (Portnoy et al., 2011). 

Usability Assessment

In the gaming industry, usability assessment is characterized by its focus on user purpose and the role of the usability process. Video games emphasize entertainment and players' emotional states, like engagement and fun, over efficiency, whereas in other industries, the priority is task efficiency and user productivity (Portnoy et al., 2011). The usability process in gaming not only seeks to comprehend user experience but also aids in the iterative game design process. It is a useful tool for designers to evaluate design effectiveness.

The distinct nature of usability assessment in gaming affects the methods employed, such as psychophysiological techniques that continuously and objectively gauge players' emotional states without disrupting gameplay. Developers can generate emotional profiles to assess whether game features provoke the desired emotional response. In addition, eye tracking is another method that documents players' gaze patterns to evaluate attention focus and comprehend how visual information is processed in games (Portnoy et al., 2011).

Playability Heuristics 

Finally, the study by Soomro and others utilized two established playability heuristics to evaluate six Android games across various genres with fourteen test participants. Findings from the study indicate that the current playability heuristics fall short in identifying playability issues, such as “usability, mobility, gameplay, and multiplayer functionality” (Soomro et al., 2013) for touchscreen mobile game applications, as indicated in Table 2 below. While this list is not meant to be exhaustive, it provides meaningful metrics for video game design across platforms.  

Design Recommendations

Finally, the study "Evaluating Affective User-Centered Design of Video Games Using Qualitative Methods" by Ng and colleagues helped to establish a comprehensive set of guidelines for user-centered design in the context of video games. Affective user-centered design is an approach that assesses whether the emotions, meanings, and mental concepts conveyed through a video game's tangible and intangible elements align with the preferences and expectations of the intended users. In their research, Ng and others employed qualitative methods to thoroughly analyze data and identify key factors that contribute to an engaging and enjoyable gaming experience. As a result of their analysis, they proposed 15 recommendations for user-centered design in video games. These recommendations cover various aspects of game design. Note that the following list is adapted from the research paper by Ng et al.

  1. Diversity of Users

  2. User Capability

  3. Challenging Gameplay

  4. Provide Goals

  5. Provide Rewards

  6. Impressive Visuals

  7. Graphic Quality

  8. Creative Gameplay

  9. Good Narrative

  10. Interesting Game Characters

  11. Flow in Games

  12. Flexible Options

  13. Good Gaming Interface

  14. Tutorial and Hints

  15. Easy-to-Use Controls

Difficulties and Remedies

One difficulty is finding relevant literature on the topic. This can be especially challenging since the topic of the user interface for video games is relatively new and not well-researched. A solution would be exploring different sources for publishing studies. Google Scholar offers a comprehensive source for peer-reviewed journals and articles, and it is also worth investigating studies published in different countries and languages. However, given enough papers on this topic, a remaining significant challenge in conducting this literature review was the heterogeneity of studies in terms of research methods, design approaches, and platforms under consideration. This made it difficult to compare and synthesize results across studies. In addition, the limited amount of empirical research focusing specifically on human factors design and multi-sensory user interfaces in video games made it challenging to draw robust conclusions. Much of the literature was based on anecdotal evidence, case studies, or expert opinions, which limited the generalizability of the findings. Most quantitative studies included less than 50 participants in their studies. To address this issue, a systematic literature search using well-defined inclusion and exclusion criteria was employed. This approach allowed for the identification and inclusion of high-quality studies that met the research objectives.

In addition, the rapid pace of technological advancements in the gaming industry posed challenges in identifying studies that were still relevant and up-to-date. This required a constant review of recent publications and an assessment of their relevance to the current state of technology. To stay abreast of the latest developments in the field, it is essential to regularly update the literature review by incorporating recent studies and technological advancements. This will help maintain the relevance of the review and provide up-to-date insights.

Discussion

The findings of this literature review highlight the importance of human factors principles in video game design, as they significantly influence players' emotional experiences and overall satisfaction. The four critical Human-Centered Design processes, including understanding the context of use, specifying user requirements, producing design solutions, and evaluating the design, play crucial roles in creating an enjoyable and immersive gaming experience.

The study also emphasizes the need to consider the unique characteristics and limitations of different gaming platforms such as PC, console, and mobile when designing user interfaces. By understanding these platform-specific features, designers can optimize human factors in video game design, ultimately enhancing the gaming experience for players across all platforms. 

In terms of user interfaces, the study examines the importance of auditory, visual, and haptic feedback in providing an interactive and engaging experience for players. By following human factors principles, these interface mediums can be tailored to meet players' desires, necessities, and core motivations. 

The challenges faced by designers in the context of pre-learned interaction models, usability assessment, and playability heuristics are also examined. These challenges underscore the need for a deep understanding of players' prior learning, expectations, and initial behaviors, as well as the unique nature of usability assessment in the gaming industry. Finally, the study on affective user-centered design in video games offers 15 recommendations, covering various aspects of game design such as diversity of users, challenging gameplay, impressive visuals, creative gameplay, flow in games, good gaming interface, and difficulties and remedies. These recommendations serve as a valuable resource for designers seeking to create engaging and enjoyable gaming experiences that align with the preferences and expectations of their intended users.

Maturity of the Topic

The research topic, “Human Factors Design of Multi-sensory User Interfaces in Video Games Across Platforms,” is a relatively mature field of study, with its roots dating back to the early days of video gaming. However, as video game technology and design have evolved, so too has the research in this area. Multi-sensory User Interfaces are interfaces that combine multiple input modalities (such as keyboard, mouse, touch, voice, etc.) and output modalities (such as visual, auditory, haptic feedback, etc.) to provide a seamless and immersive experience for the player. Research on user interface in video games has been growing steadily since the 2000s, as developers have experimented with novel ways of integrating multiple modalities to create more immersive gameplay experiences. 

With the proliferation of gaming platforms like PC, consoles, mobile devices, and virtual reality systems, the study of user interfaces across these different platforms has become increasingly important. Researchers have been comparing and contrasting the unique challenges and opportunities presented by each platform, as well as identifying best practices for designing interfaces that can be easily adapted for different devices. This area of research has gained momentum over the past two decades and continues to evolve as new gaming platforms emerge.

While substantial progress has been made over the years, ongoing advancements in technology and gaming platforms, as well as the growing emphasis on usability and accessibility, ensure that this field will continue to develop and expand in the coming years.

Future Directions

Cross-platform Design Strategies

As gaming platforms continue to diversify, there is an increasing need for more effective cross-platform design strategies. Future research should focus on developing guidelines and best practices for designing user interfaces that can adapt seamlessly across different platforms while maintaining a consistent user experience.

Inclusive and Accessible Design

Inclusivity and accessibility should be key considerations in future research on user interface design for video games. This involves conducting research on how to cater to gamers with varying abilities, cultural backgrounds, and preferences, ensuring that games are enjoyable and accessible to a wider audience. This might include the development of adaptable user interfaces, alternative control schemes, and customizable difficulty settings.

Evaluation and Validation of Design Principles

In order to establish effective design principles for video game user interfaces, future research should prioritize the evaluation and validation of these principles through rigorous testing and user feedback. This will ensure that the design recommendations are grounded in empirical evidence and can contribute to creating more effective and user-friendly gaming experiences.

Conclusion

In conclusion, this paper provides a comprehensive literature review of the human factors design of Multi-sensory User Interfaces in Video Games Across Platforms. The effective implementation of human factors design principles is crucial in the development of successful video games that provide enjoyable and immersive experiences for players. By carefully considering user requirements, designing tailored solutions, and evaluating outcomes, designers can create games that resonate with their target audience and evoke positive emotions. Moreover, the paper emphasizes the importance of understanding the unique characteristics and limitations of different gaming platforms, as these factors directly impact user interface and user experience design. Furthermore, a well-designed video game user interface incorporating visual, auditory, and haptic feedback can significantly impact the gaming experience by enhancing player performance, immersion, and satisfaction. While each interface medium has its unique strengths, a multi-sensory approach is generally preferred by players, as evidenced by Nesbitt and Hoskens' study. Ultimately, striking the right balance between aesthetic appeal, usability, and immersion is crucial for creating a successful user interface that caters to players' diverse needs and preferences. Finally, creating a successful video game requires a deep understanding of user preferences, capabilities, and pre-learned interaction models. By incorporating usability assessments and playability heuristics, designers can fine-tune their creations to deliver an engaging and enjoyable gaming experience. Although substantial progress has been made in this field, future research should focus on cross-platform design strategies, inclusive and accessible design, and the evaluation and validation of design principles to ensure continued growth and relevance in this rapidly evolving industry.