Section: Module 4: Methodology and Recommendations | Digital Reality in Foreign Language Education for medical Professionals

DR FLEMP

Course index

Section outline

- Select activity Module 4: Methodology and Recommendations
  
  User-centered design is an approach to product design and development that focuses on the needs of the end-users. It is based on the principle of involving users in the design process in order to create products that are more usable and effective. User-centered design involves activities like user research, persona development, prototyping, and testing, aiming to meet the users’ needs. Module includes the following aspects: the process description of the productive collaboration, the occurring problems during that process and how the process should be organised. Out of these aspects should then result in requirement types and their specific implications.
- Select activity 4.1 Summary
  
  Students must
  
  Mark as done
  
  4.1 Summary
  
  User-centered design is an approach to product design and development that focuses on the needs of the end-users. It is based on the principle of involving users in the design process in order to create products that are more usable and effective. User-centered design involves activities like user research, persona development, prototyping, and testing, aiming to meet the users’ needs.
  
  Purpose and scope
  
  The intent of this analysis is to give an overview of all the relevant collaboration aspects between the partners. The result should be visualized appropriately, e.g., with a mind map. Section includes the following aspects: the process description of the productive collaboration, the occurring problems during that process and how the process should be organised. Out of these aspects should then result in requirement types and their specific implications.
Select activity 4.2 Defining successful productive collaboration

Students must

Mark as done

4.2 Defining successful productive collaboration

Productive collaboration is a process-based approach which encompasses the following characteristics: the clear and structured exchange of information, delivery methods like visuals or text should be included depending on appropriateness, iteration of the collaboration process with agile methods (e.g. SCRUM), each collaboration member is aligned with the other members to reach a common define outcome, the role and tasks of each member is defined and understood.

4.3 Goal of productive collaboration between the language and healthcare teachers and the developers.

The goal of these criteria is to complete tasks on time. Additionally, productive collaboration can result in modelling and scaffolding. Modelling helps partners to have a reference that is more tangible than an abstract idea for the project. Each member is likely to understand what the outcome of a specific task should be. For example, the developers demonstrate a prototype which is close to the project’s result. This demonstration is able to clarify the objectives and helps to avoid confusion of the healthcare/language teachers. Scaffolding supports that understanding. Over time, this support is reduced and ideally not required anymore. As a result members become more autonomous during the process and ensure successful task completion within the defined timeframe.

4.4 Describing the general process of productive collaboration

Following the productive collaboration process we can outline two separate processes: the process of defining a scenario (content) and the process of defining the required tools to generate a concrete VR scenario.

First, we analyse the process of defining the scenario. An initial step is to check the status quo of VR scenarios in nursing. The step includes the scenarios, their selection criteria, their limitations, advantages and the overall acceptance of the VR experiences from the nurses. Then, based on the previous knowledge, learning outcomes should be formulated. Furthermore, concrete requirements to the content should be derived from the defined learning outcomes, e.g., level of details of the graphics, level of accuracy of the interactions, movement in VR space and more.

Note: Especially, interactions can be tricky as VR-headsets do not always give the necessary level of accuracy when interaction with the VR content. For example, you grab a syringe and you have a displacement of a few millimetres which might confuse the user. Generally, accuracy is a concern when doing object manipulation. Social interaction in VR tends to be less affected by slightly lower accuracy.

Second, we decide on the tools to achieve successful productive collaboration. Prior knowledge of tools is one possibility on deciding on the tool as it will reduce time for acceptance and training. Tools for collaboration could be Zoom, Miro or Slack. Also, during the development of the training prototype professionals (e.g. nurses), as well as instructors and students must be able to access the prototype to give regular feedback to the developers. Using an agile method like SCRUM helps professionals and instructors and the developers to achieve the required solution. Both processes on the scenario development and the tools usage should be documented to be able to describe challenges and give recommendations for future work.

4.5 Defining the requirements for productive collaboration

There are several types of requirements that need to be taken into account. Initially, stakeholders need to define early on a common language of communication that can enable smooth communication and collaboration. This could mean for example that a role-play scenario could involve a real-human and a bot, the two roles need to be explicit in the scenarios for the developers team to make appropriate adjustments in the design.

There is also the work process and management of the personnel, finances and the time frame for the project. Then, there are the communication tools which need to be adopted by each member. Using free and open-source software might be a good choice to reduce cost, whereas proprietary software while requiring a budget might be easier to use. Privacy is also an important topic which might be considered as open-source software often does not require registration, hence giving away personal information. To actually experience the training VR equipment and peripherals are required. Depending on the VR headset an extra computer might be required to run the application, e.g., HTC VIVE. The Oculus Quest is self-sustaining. When the equipment is available the teachers need to engage with the equipment and the VR experience. Ready-to-use applications, i.e. starter applications are ideal to get everyone on board. This will ease future discussion in the process as the teacher already got a sense on what can work for the learning experience in VR.

Concretely, the teachers need to understand what would be the added value. Only then will they be able to identify the right lessons or tasks from their curriculum that will benefit from a VR adaptation. Therefore, the developers should early on encourage all the other stakeholders involved to experience VR. Conversely, strategies to keep the teachers motivated and involved with VR applications are essential. These could be any app, game or experience that is in the context of the project. It is also possible to focus on specific interactions of a particular app. For example, starter applications usually have tasks where users move objects from one location to another. This already gives a sense of the movement in VR and the experience of grabbing a virtual object.

4.6 Overview of Design Best Practices and User Centered Design

Design of the VR application should be done according to the best practices described in the Oculus Best Practices document as well as those on the Oculus website.

The Oculus Best Practices is a guide released by Oculus for VR game development. Much of the guide is dedicated to technical aspects of game development such as rendering and tracking which is largely taken care of by the chosen game engine and VR framework or by the Quest 2 operating system. The guide will not be replicated entirely in this document, but a short summary of the most relevant practices will be included.

4.7 Oculus Best Practices Summary

Some of the key practices more related to design are:

· Never disable head movement tracking, even in menus.

· Minimise player acceleration time and frequency. Prefer instant changes in velocity if possible. If acceleration does happen, it is preferable if it is controlled by the player.

· User Interface should float 2-3 metres from the user

Prefer a user interface integrated in the 3d world over a HUD.

4.8 User Centered Design

The process of designing with the users needs in focus in an iterative process. Meaning that design is not something one once, up front, and then handed over for implementation. Instead, domain experts, in this case experts in medical and Language educators should be involved regularly in the design process. The domain experts should take part in the review meetings and their feedback during those will form the basis for the continued design and development.

We should also aim to frequently deliver functional builds of the project which can be tested on the actual hardware by the individual nursing educators in the project.

4.9 Requirements for development

· Have a scenario

· Appoint a Product Owner

· Work with Product Owner to produce:

o user stories

o flow diagram.

o Reference materials (Images, video, etc.)

· Make sure the Product Owner will have access to the stakeholders to continuously have dialog during the development process, for Q&A’s. This can lead to new user-stories and added features over time.

Play time

How long the scenario is meant to take (ideally not longer than 20 min).

Protocol of scenario

Detailed protocol of the actual scenario. As the design and development team is unlikely to be familiar with the details of nursing, a nursing professional needs to take part in the process and be available for explanations.

User flow diagram

User flow diagrams. These can be created in collaboration between the design team and the nursing team.

Materials

List of objects the student needs to interact with in the scenario.

· The level of detail needed for effective learning

· How they are used

· Where in the environment the objects are located

Environment

· Description of the environment the scenario is expected to take place in.

· The size of the space.

· How students are expected to move about the environment.

Tools/Interactions

· How the student interacts with the scenario. Examples:

o Picking up objects with their hands/controllers

o Interacting with a computer

o Interacting with a patient using dialog trees/menus

· How much precision is required for a satisfactory level/successful learning

o Does it need precise

· Are the VR controllers or optionally hand tracking enough to provide an accurate simulation of this scenario? If not, what additional equipment should be used?

Testing and iteration

Availability of nursing experts to participate in review progress of the development of the scenario and offer feedback. It is important that issues are caught early before a lot of resources are spent on developing something.

4.10 Tools

Table 9 - Tools for user-centric design

Approach / Tool

Purpose / area

Relevant Tools

Description

Wireframing

Prototyping

Whiteboard

Brainstorming

Miro

Figma

Real world whiteboard, paper and post-its

Start making early prototypes, so the first version of the application can be visualised (can also be used for later versions).

User Stories

Perspective of the end user

Communication entries between stakeholders and developers

General explanation of a software feature written from the perspective of the end user.

4.10.1 Tools used during product development

Combination of tools that can be used by the development team to build the MR applications.

Table 10 Tools used during product development

Approach / Tool	Purpose / area	Relevant Tools	Description
Kanban	Task manager	Monday Jira Trello Asana Github issues MS Planner	Helps organise and prioritise development work.
Game Engine	XR/VR development	Unity Unreal Engine Godot	Develop 3D applications for target platforms.
Version Control	Manage and share project codebase	GitHub Bitbucket Gitlab	Helps the team manage versions of and share the application codebase

4.10.2 Chatbot

Combination of tools that can be used by the team to build the chatbots.

Table 11 Tools for chatbots

Approach / Tool	Purpose / area	Relevant Tools	Description
Speech-to-text	Human voice into text data	VoiceSDK Whisper	Make voice/sound data into text data.
Text-to-speech	Language models text response into sound data	VoiceSDK Google TTS Microsoft TTS	Take the text response from the Language model and give it a voice.
Language Model	Natural Language Processing (NLP)	Wit.ai Botpress ChatGPT	Take the text and create a response.
Vector Embedding	Semantic Search Prompt Engineering	OpenAI Embeddings Open-source embedding models	Create vector embeddings from the knowledge base for creating optimal prompts.

4.11 Selection and customization

Table 12 Selection and customization

Area	Tool	Reasons
Early Prototyping	Figma	- Has Free version - Easy to use and to chare - Easy to collaborate on
User Stories and Task Manage	Monday	- Easy to use. - Members already had experience with it. - Has free version
Game Engine	Unity	- Has free version - Large community - Well supported for XR development - Large library of assets in stores.
Version Control	Git (GitHub)	- Easy to use. - Members already had experience with it. - Has free version
Text-To-Speech & Speech-To-Text	VoiceSDK	- Easy to use - Free - Good support for Oculus/Meta headsets. - Good support for Unity - Well documented
Chatbot / Language Model	OpenAI’s ChatGPT API	- Easy to use - Powerful when it comes to acting as a character - Saves time
Vector Embedding	OpenAI Embeddings	- Easy to use - Ranks High on the MTEB

4.12 New elements and update of the pathwork of conceptual elements

4.12.1 Instructional design and scenario writing

When it comes to designing simulations for mixed reality (MR) in the context of foreign language education (FLE), there are several aspects that need to be taken into consideration. These include:

1 The use of MR technology. The technology adds a level of immersion and interactivity to the simulation. This means that instructional designers need to be familiar with the capabilities and limitations of the technology to ensure that the simulations are both effective and engaging.

2 Language proficiency levels: The simulations need to be designed for learners at different language proficiency levels, from beginner to advanced. This means that the instructional strategies and materials need to be tailored to the specific needs of each group of learners.

3 Nursing scenarios: The simulations need to be designed to simulate real-world nursing/healthcare scenarios, such as patient assessments, medication administration, and communication with patients and colleagues. This means that the instructional materials and scenarios need to be developed in collaboration with nursing professionals to ensure accuracy and relevance.

4.12.2 Applying the design practices to scenario writing

The design process should prioritise the needs of students, which can be identified through user research, personas, and user stories. Prototyping and testing should be used to validate the design.

4.12.2.1 Primary learning outcome

In a more complex scenario, the simulation will include a variety of aspects. The level of detail of all those cannot necessarily be kept high. The primary education objective of the scenario should be to select the subset of the simulation that requires the most accuracy and care.

4.12.2.2 Scenario duration

How long the scenario is meant to take. Ideally the duration of each scenario should not exceed 20 minutes.

4.12.2.3 Tools/Interactions

How the student interacts with the scenario. Examples:

1 Picking up objects with their hands/controllers

2 Interacting with a computer

3 Interacting with a patient (real person or a bot) using dialog trees/menus

4 How much precision is required for a satisfactory level/successful learning.

5 Does it need precision?

6 Are the VR controllers or optionally hand tracking enough to provide an accurate simulation of this scenario? If not, what additional equipment should be used?

4.12.2.4 Protocol of scenario

4.12.2.5 Evaluation of student action

A detailed description of the evaluation of the student’s action is required.

4.12.2.6 User flow diagram

User flow diagrams. These can be created in collaboration between the design team and the nursing team.

4.12.2.7 Materials

List of objects the student needs to interact with in the scenario.

· The level of detail needed for effective learning.

· How they are used

· Where in the environment the objects are located

4.12.2.8 Environment

· Description (or visualisation) of the environment the scenario is expected to take place in.

· The size of the space.

· How students are expected to move around the environment.

4.12.3 Testing and iteration

Availability of language education experts to participate in the review progress of the development of the scenario and offer feedback. It is important that issues are caught early before a lot of resources are spent on development.

4.13 Designing a Simulation Scenario

Construct measurable objectives.

4.13.1 Key Components:

· Create broad and detailed objectives to address identified needs and maximise the attainment of desired outcomes.

· Broad and specific objectives together form the blueprint for designing a simulation-based experience.

o Broad objectives reflect the overall goal of the simulation-based experience and align with organisational objectives.

o Specific objectives are related to participant performance metrics.

· During the design phase, decide which objectives will be made known to the participants before the experience and which will be kept hidden.

o Objectives that provide general information and context for the participants should be disclosed (e.g., provide care for a patient with heart failure).

o Participant performance metrics or critical action checklists should not be disclosed.

· Use the measurable objectives to guide the design, development, and implementation of the simulation-based experience (as per the INACSL Standard: Objectives and Outcomes[5]).

· The facilitator is responsible for ensuring the achievement of all objectives throughout the simulation-based experience (as per the INACSL Standard: Facilitation[6]).

4.14 Simulation Recipe

4.14.1 Who is your target audience?

Medical professionals working in a different language than they were originally trained in.

4.14.2 Learning Objectives

What are the learning objectives? How are they to be evaluated? Can they be automatically evaluated by the application or does a teacher need to look at students’ performance?

4.15 What else to figure out:

4.15.1 Medical record

(Optional) – patient notes, history & physical, medication list, ECG’s, labs, x-rays

4.15.2 Teaching Points

· Critical actions - series of steps necessary to successfully complete tasks & demonstrate understanding of the teaching points.

· Common Pitfalls - common mistakes that your learners may make that you specifically want to remind them to avoid.

4.15.3 Debriefing – most important part

· based on the teaching points outlined above

· based on what you observed during the simulation

4.16 Development of guidelines for conceptual design, scenario writing, development and adaptation of new simulation scenarios for VR in nursing education and training

Follow Criterion 3, from INACSL Standard of Best Practice: Simulation Design.[7] Construction of a chatbot using Large language models (LLMs) based technology.

4.16.1 Basic principles of LLMs

Large language models are a type of neural networks with a large number of parameters trained on large sets of unlabeled text data. They can be used for language tasks such as predictive generation of text or similarity search of string of text. Recent developments (i.e. GPT-3 and GPT-4) have shown a high capability of generating plausible, human-like text responses.

4.16.2 Prompt engineering

Prompt engineering is the process of designing prompts provided to a language model to achieve desired behaviour. It involves crafting the initial input to the language model in a way that produces the desired response. Prompt engineering is critical for ensuring that the chatbot responds appropriately.

LLMs makes prompt engineering relatively accessible. The prompts can be written in plain text English, with no programming component.

The written prompt could contain instructions for how to behave and specific instructions on how to answer the questions, such as specific moods.

4.16.3 Techniques to enhance prompt and knowledge

Embeddings are a way to compare two pieces of text using LLMs. Each string of text is turned into a large dimensionality vector and different distance measures like euclidean distance or angle between the vectors can be used to compare the semantic similarity of two texts.

LLMs have a limit of how much text it can process and respond to, and the service offered is monetized by length of text processed. For these reasons it’s in our interest to keep the prompts short. A large background representing a character’s knowledge and history can be split up into chunks and then stored in a database with their embedding vector as a key. When a question from the student is received the similarity between the student question and the entries of the database is used to construct a prompt of only the information relevant to the question instead of including the whole character background.

4.16.4 Prompt components

4.16.4.1 Conversation history

Conversation history has to be included as context for the model. For example to enable students to ask follow up questions. This part of the prompt is automatically generated and no writing is necessary.

4.16.4.2 Background

Detailed description of the character background. All relevant facts should be detailed here. It does not need to be in any particular format, a simple list of facts will work.

An example of how the background section could look:

Elisa Johnson was born on March 1, 1983. She is 39 years old. She has not felt good recently. She has weakness and sweating and stomach pain. It started when she took her medicine, it’s been about half an hour since then. Another complaint she has is that she feels tired, and that’s unusual. She was diagnosed with type-2 diabetes three years ago.

4.16.4.3 Behaviour prompt

Contains the commands for how the character should respond. Care has to be taken to write one that produces a desired result.

4.17 List of requirements, workflow charts, for successful interaction and collaboration between teachers and developers

4.17.1 Understand the target technology that you are designing for

Designing becomes challenging when you're unaware of the strengths and constraints of the technology for which you're designing for. Here are some recommended steps that one can follow to understand Extended Reality (XR) technology better:

1 Research: Learn the basics of XR through books and online resources.

2 Attend workshops: Join hands-on XR workshops or courses to experience it firsthand.

3 Explore XR applications: Try different XR experiences like games or simulations.

4 Join online communities: Engage with XR enthusiasts online to learn and discuss (example: XR4Europe member)

5 Stay updated: Follow XR news, conferences, and podcasts to stay informed. (example: XR4Europe newsletter)

Table 13 - Recommended sources for getting familiar with XR

Name	More Information
Extended Reality (XR)	https://www.interaction-design.org/literature/topics/extended-reality-xr
XR4Europe – What is XR?	https://xr4europe.eu/whatisxr/
EU Publication: Extended reality Opportunities, success stories and challenges (health, education): final report	https://op.europa.eu/en/publication-detail/-/publication/f242f605-a82e-11ed-b508-01aa75ed71a1

Some recommended applications to try out:

Table 14 Recommended applications

Name	Category	More Information
First Steps	Game	https://www.oculus.com/experiences/quest/1863547050392688/
Oxford Medical Simulation	Medical Simulation	https://oxfordmedicalsimulation.com/
MondlyVR	Language	https://www.mondly.com/vr
MondlyAR	Language	https://www.mondly.com/ar
ImmerseMe	Language	https://immerseme.co/
Vermillion	Game	https://www.oculus.com/experiences/quest/4900967296622279/
Job Simulator	Game	https://www.oculus.com/experiences/quest/3235570703151406/

4.17.2 Follow well established standards for designing language learning in medical scenarios

Follow well established frameworks and guidelines like the COMMON EUROPEAN FRAMEWORK OF REFERENCE FOR LANGUAGES: LEARNING, TEACHING, ASSESSMENT (CEFR) for designing the learning objective within language learning. But also use standards for healthcare, like the INACSL - Healthcare Simulation Standards of Best Practices(TM) to make sure that the simulation contains the medical aspects also.

List of standards for medical simulation:

Table 15 Standards for medical simulation

Name	Organization	More Information
Healthcare Simulation Standards of Best Practices	INACSL	https://www.inacsl.org/healthcare-simulation-standards
Standards for Simulation-Based Education	ASPiH	https://aspih.org.uk/standards-framework-for-sbe/
Capability Framework for healthcare Simulation Technology Specialists	SimGHOST	https://cdn.ymaws.com/simghosts.org/resource/resmgr/capability_framework/a4_final_simghosts_capabilit.pdf
Standards of Best Practice	ASPE	https://www.aspeducators.org/standards-of-best-practice
SSH Code of Ethics	SSH	https://www.ssih.org/SSH-Resources/Code-of-Ethics
SSH Dictionary	SSH	https://www.ssih.org/dictionary

4.17.3 Collaborate with the Development/Agile Team

When the learning scenario is defined it is time to involve the developers. The status quo of the software development process is agile. Meaning it is an iterative process. A good rule for the educators to keep in mind when collaborating with software developers is that the domain should own the product (the MR application), and that they are the ones that decides when this is good enough for usage. To do this, make sure to appoint a Product Owner (PO). In complex use cases, like education and healthcare, keep in mind that the developer does not fully understand the use-case, only the educators (stakeholders) does, hence it is the PO’s mission to be the bridge between the stakeholders and the agile team.

4.17.3.1 Process Overview

When your initial vision, research and design feels ready for a more complex prototype (example a VR application). Now is a good idea to involve an agile time.

Figure 21 Overview of the overall development process of the MR application

4.17.4 Appoint a Product Owner

The Product Owner (PO) is accountable for maximising the value of the product (the simulated scenario) developed by an agile Development Team. They serve as the representative of stakeholders, including customers, and determine the deliverables required. In short:

· The PO communicates with the stakeholders and breaks down their needs into understandable assignments (preferably user stories, see Figure 29).

· The user stories are then given to the agile team to be placed in their backlog.

· The agile team will then break down those user stories into features to be implemented into the virtual simulation.

The PO can also be one of the stakeholders (i.e. an educator).

Figure 22 Role of the Product Owner

4.17.5 Product Backlog and task manager

A product backlog and task manager for an Agile team is the tools used to organise and track tasks within the team. It helps prioritise work, manage tasks, and identify problems in the workflow. It can be a simple board with sticky notes or a digital system like Jira, Trello, or Asana. The goal is to increase efficiency and productivity by making it clear who is doing what and how progress is going.

Figure 23 The process of communicating with stakeholders, formulating user stories, developing a virtual feature, which is subsequently tested and reviewed for feedback.

4.17.6 User Stories and Story Points

If you would like to know more about user stories and good practices for producing them, you can read more about that topic here. But in short it is the product owner's job to make sure the user story represents what the stakeholders want and to prioritise them for the development team.

User stories will later be broken down into implementation tasks (also known as story points) by the agile team. It is also their task to estimate the time and complexity it will take for each story point (hence the whole User Story). The 3 main factors that is considered when estimating story points are:

· Complexity: How difficult is it to develop this story?

· Risks: Does there any external dependencies (ex: third-party libraries) that can affect the development?

· Repetition: How extensive is the workload? Are there tasks that can be easily replicated?

Figure 24 Story Points

How to estimate User Points?

There are multiple methods that can be used to estimate story points. Some popular are:

· Using the Fibonacci sequence. Read more here!

· Using T-Shirt sizing. Read more here!

· Planning Poker. Read more here!

Pick the must suitable for your use-case. Just remember story points are based on the complexity of tasks, the effort needed to complete them, and the uncertainty and risk involved. Do not size the story based on only one or two of these parameters. It's not accurate to equate story points with hours because team members work at different speeds.

Story points aren't exact but provide a rough idea for project planning. Small tasks may not need story points because they take a short time to complete.

If the team changes significantly, consider re-evaluating the story point system, as the new members might work differently.

Estimation should be a group effort with everyone having a chance to share their opinion. Retrospectives can help improve the accuracy of future estimations.

Some tools (applications) that can be used for task management

Table 16 Tools for task management

Name	Organisation	More Information
Monday task management software.	Monday	https://monday.com/s/task-management-software
Jira	Atlassian	https://www.atlassian.com/software/jira
Trello	Atlassian	https://trello.com/
Asana	Asana	https://asana.com/
GitHub	GitHub issues	https://github.com/features/issues
Microsoft Planner	Microsoft	https://www.microsoft.com/en-ww/microsoft-365/business/task-management-software

Note that there are plenty more options out there, if you have another one that your team prefers, use that one.

4.17.7 Make a Product Roadmap

A product roadmap serves as a strategic guide that shows the development path of a product or solution over time. It's a tool used by product owners to specify upcoming product features and their release timeline. In agile development, a roadmap offers vital background for the team's daily tasks and must be adaptable to changes.

The product roadmap is a fundamental communication tool that aligns short-term actions with long-term objectives. Grasping the function of a roadmap, along with knowing how to create an effective one, is crucial to ensure all team members are moving towards the same goal.

More information about product roadmaps can be found here!

4.17.8 Create a User Flow Diagram

A user flow diagram, otherwise referred to as an interaction or task flow diagram, is a visual representation of the sequential steps a user takes to accomplish a task or achieve a goal using a particular product or service. These diagrams serve as a critical tool for product and UX teams, enabling them to determine the most effective interaction methods with an application following the identification of user needs. In order to comprehend these needs fully and envision the desired user experience, mapping and visualisation are essential steps.

More information about User Flow Diagrams can be found here!

Some tools (applications) that can be used for creating flow diagrams

Table 17 Tools for flow diagrams

Name	More Information
Miro Board	https://miro.com/
Draw.io	https://www.drawio.com/
Visio	https://www.microsoft.com/en-ww/microsoft-365/visio/flowchart-software

Note that there are plenty more options out there, if you have another one that your team prefers, use that one.

4.17.9 Collect Reference Materials

Reference materials are a very useful and viable thing for the agile team to have access to. This could be materials like: images of medical devices, hospital rooms, video recordings of procedures or interaction, 3D scanned objects, real physical objects, and more.

3D scanning objects

Today it is possible and cheap to 3D scan objects and environments, using only a smartphone. 3D scanning is the process of analysing an object or environment and collecting data about its appearance. The scanning tool (ex smartphone) then creates a 3D digital object that later can be used for different purposes, like placing it in a virtual environment. Some popular mobile applications for 3D scanning are KIRI Engine, WIDAR, and Polycam. There are plenty more, and depending on the time of reading, other applications on the market might work better.

There are also other techniques for 3D scanning objects that involve more expensive equipment. If custom 3D objects of high quality are of high priority for your simulations, this topic might be something to spend more time and money researching.

Some tools (applications) for 3D scanning:

Table 18 Tools for 3D scanning

Name	Platforms	More Information
KIRI Engine	iOS, Android, and web browser	https://www.kiriengine.com/
WIDAR	iOS and Android	https://www.widar.io/
Polycam	iOS, Android, and web browser	https://poly.cam/

Note that there are plenty more options out there, if you have another one that your team prefers, use that one

4.17.10 Collaborating on chatbots based on realistic personas that uses Large Language Models

Using Large Language Models (LLMs) as an engine has revolutionised the way to create chatbots. However the base-models themself (like GPT-4) have the bad habit of hallucinating.

To avoid this the domain can create their own knowledge bases and/or personas to limit what responses the Large Language Model will say. The steps are:

· Use a data driven framework for creating character persona(s)[8]

· Make the persona into a character by aligning the language to follow instructions[9]

· See this GitHub repository for examples and instructions following templates.

Figure 25 From persona into an instruction following the initial prompt.

The initial prompt (that can be based on one of the instructions following templates) can be delivered to the development/agile team for implementing the chatbot into the application.

Simple example of an initial prompt:

name: "Jane Doe"

context: "Jane Doe's Persona: Jane Doe is a 42 year old woman, living in Germany Berlin. She has two young children and is a construction worker. She has suffered from type 2 diabetes since 6 years back and is taking 50 mg of insulin tablets every day. She has been suffering from fuzzy eyesight for the last 3 days."

greeting: |-

*Jane Doe is sitting in an examination room at the hospital, she has been waiting for 15 minutes and is starting to get a bit annoyed. She wants to know what can be the cause of her fuzzy vision.

example_dialogue: |-

{{user}}: Hello, how are you today?

{{char}}: Hi doctor, overall I’m good, but something is wrong with my eyes.

{{user}}: Okej, what was your name again?

{{char}}: My name is Jane Doe

Good practice would be to add more information to Jane Doe’s character and a longer example dialogue.

Some tools that can be used for creating and sharing instruction following characters:

Table 19 Tools for creating and sharing instruction following characters

Name	Platforms	More Information
Text Generation Webui, from oobabooga	Self-host, web browser	https://github.com/oobabooga/text-generation-webui/blob/main/docs/Chat-mode.md
Files (.txt, .json, .yaml) with instruction following format.	Localhost, Google docs, any cloud file system	https://github.com/oobabooga/text-generation-webui/tree/main/characters/instruction-following
Flowise	Self-host, web browser	https://flowiseai.com/

Table 20 Methodological and Design Guidelines and Recommendations for MR applications

Element	Guidelines
Effective and successful A “good practice” has proven its strategic relevance as the most effective way in achieving a specific objective; it has been successfully adopted and has had a positive impact on individuals and/or communities. ● What are the conditions (institutional, economic, social and environmental) needed for the successful implementation of MR in healthcare?	Guideline 1. Flexibility and control Design Thinking Workshops (DTWs) demonstrated that participants desired more flexibility, more control of their learning in MR. Participants voiced their desire for control and being able to communicate with healthcare professionals and patients in an authentic environment. Thus, flexibility and control can be broken into two sub-guidelines: Guideline 1.1 Learner-initiated communication Guideline 1.2 Flexibility and control in using the MR communication channel(s) Guideline 2: Align MR with Learning Objectives From the DTWs it was extracted that the activities and scenarios used in the app should add value to the classroom implementation. In this context, the MR functionalities should be applied for meeting the learning objectives of the language course. While it can be helpful to focus on all language skills as they very rarely occur in isolation in real-life communication, we recommend focusing on tasks that support vocabulary, pronunciation, grammar, pragmatics, listening, speaking and reading. Writing in MR is time-consuming, demotivating and there is no added value to writing in MR. Guideline 2.1: The scenario allows for both productive and receptive skills to be practiced Guideline 2.2: The scenarios allows for students to engage in interactive language production, introduction of new vocabulary (e.g. role play, dialogue between two nurses talking about the medical equipment or a patient) Guideline 2.3: The scenarios allows for students to interact with audiovisual elements instead of only text, as suggested by the Cognitive Theory of Multimedia Learning (e.g. a nurse approaches a monitor and it expands with its features) Guideline 2.4: The scenarios and the immersive nature of MR provide context to the students learning experience, as suggested by the situated learning theory Guideline 2.5: The content should be useful and entertaining. It should differ from what we already know in order to motivate and inspire learners to learn a foreign language more effectively. Guideline 3: Instructors and students training and familiarity with MR The DTWs demonstrated that instructors and students need to get basic training for MR in order to be able to perform activities with it and not get demotivated due to their unfamiliarity with the technology. Provide a short training of the technology features within the app (guided activities, mini games, equipment and features) Guideline 3.1 Need for students to perform a task (grab something, move around, turn right) Guideline 4: Implementation of real-life scenarios The DTWs and focus group discussions demonstrated that in healthcare it is important to have realistic scenarios and situations for practice and decision making. Guideline 4.1 Provide realistic state of the art hospital environment (Implementation of state-of-the-art hospital equipment) as suggested by the situated learning theory. Guideline 5: Promote Learner Autonomy and Independence From the DTWs, the focus group discussions and the inquiry-based learning theory it was found that the application should allow the user to learn independently and progress autonomously e.g. make decisions and see their impact, receive feedback, get help when stuck, use of speech recognition, pre-recorded dialogues provided by the bot, grab tools inside the hospital, give and take tools between nurses/doctors etc. Guideline 6: Multiplier Mode The DTWs and focus groups discussions extracted that a multiplayer mode would make the application more engaging and will promote collaboration and enhance language and decision-making skills Guideline 7: Define clearly the learning outcomes From the DTWs it was found that it is very important to know which skills or knowledge we want to teach in order to select the means as targetfully as possible.
Replicating and upscaling A “good practice” should have the potential for replication and should therefore be adaptable to similar objectives in varying situations. It needs to be methodologically transparent to successfully scale up or replicate. ● How can DR FLEMP be replicated in similar and/or different contexts? ● What are the required conditions to successfully replicate and adapt the practice in another context/geographical area? ● What are the required conditions to be able to replicate this practice on a larger scale (national, regional, international)? ● What is your vision for replicating or upscaling this practice in the framework of DRFLEMP?	Guideline 8: Existing content can be modified to meet the specific learning objectives From the desk research it was found that applications using MR technologies for LL can be modified to meet the learning objectives of ESP such as nursing and healthcare Guideline 9: Identify the needs and possibilities of the target audience From the DTWs, it was concluded that there is a need for clear understanding of the needs according to language level, country of origin and occupation. This will help to create a better content and a better product in general Guideline 10: Indicating clear instructions for language instructors and learners The DTWs demonstrated the importance of clear and understandable instruction in order to reduce possible problems while using the app, understand the tool better and provide more qualitative lessons. This will also help to understand what the learner and instructor are doing and why.
Inclusive (in terms of gender and other underrepresented groups) A description of the practice must show how actors, men and women, involved in the process, were able to improve their experience, wellbeing, objectives. ● Explain how DRFLEMP is participatory for all and inclusive (inclusive of gender and other underrepresented groups)?	Guideline 11: Access to MR equipment From the DTWs it was discussed that MR equipment for should be accessible for everyone to use and practice Guideline 12: Consider the population while designing the app From the DTWs it was found to be important to take into account all population groups in terms of gender, origin, religion, accents while developing the app. 15.1 Promote interaction between people from different backgrounds (personalisation of avatars, bots of multiple skin colours etc).
Compliant with data protection and privacy The good practice must adhere to legislative and university standards on data protection and privacy. In particular it would need to be understood how such issues are addressed in the replication or scale up of a practice.	Guideline 13: Control malicious behavior Implement visuals, speech recognition and artificial intelligence to avoid malicious behaviours in the app (recognition of disrespectful and bad words, red warnings, report users when a malicious behaviour is repetitive).
Usability Confirmation by the beneficiaries that the DRFLEMP addresses the needs properly.	Guideline 14: Provide user support From the DTWs it was found that help and assistance should be provided within the oculus at any stage of learning in order to have the learner feel safe both academically and technically. Guideline 15: Provide feedback From the focus group discussions it was found that when corrective feedback is provided e.g. explaining why an action/answer was correct or incorrect, makes more sense to users. Guideline 15.1 Provide visual feedback (graphical indicators, score, colour, sound) Guideline 15.2 Provide feedback on progress (graphical indicators, score, colour, sound, suggestions) Guideline 15.3 Provision of immediate and overall feedback. From the DTWs, it was found that both immediate (e.g. through automated response) and overall feedback are necessary in order to have an opportunity for immediate correction and have an overall assessment of progress. Guideline 16: Simple and easy to use From the focus group discussions, it was found that when too much information is provided in written format in the application, provokes confusion and destruction to the users Guideline 17: Provide help and hints From the DTWs and the Focus Group Discussions, it was noted that when users get stuck, should be able to get a form of help or receive some hints (oral/visual) in order to be able to learn and proceed within the app (e.g. help button, pictures, text, sounds).

[1] Hoadley, C. M. (2004). Methodological alignment in design-based research. Educational psychologist, 39(4), 203-212.

[2] Abdlkarim, D., Di Luca, M., Aves, P., Yeo, S. H., Miall, R. C., Holland, P., & Galea, J. M. (2022). A methodological framework to assess the accuracy of virtual reality hand-tracking systems: A case study with the oculus quest 2. BioRxiv, 2022-02.

[3] https://developer.oculus.com/resources/

[4] INACSL Standards Committee, Miller, C., Deckers, C., Jones, M., Wells-Beede, E., & McGee, E. (2021). Healthcare Simulation Standards of Best PracticeTM Outcomes and Objectives. Clinical Simulation in Nursing, https://doi.org/10.1016/j.ecns.2021.08.013.

[5] INACSL Standards Committee, Miller, C., Deckers, C., Jones, M., Wells-Beede, E., & McGee, E. (2021). Healthcare Simulation Standards of Best PracticeTM Outcomes and Objectives. Clinical Simulation in Nursing, https://doi.org/10.1016/j.ecns.2021.08.013.

[6] INACSL Standards Committee, Persico, L., Belle, A., DiGregorio, H., Wilson-Keates, B., & Shelton, C. (2021). Healthcare Simulation Standards of Best PracticeTM Facilitation. Clinical Simulation in Nursing, https://doi.org/10.1016/j.ecns.2021.08.010.

[7] INACSL Standards Committee, Watts, P.I, McDermott, D.S., Alinier, G., Charnetski, M., Ludlow, J., Horsley, E., Meakim, C., & Nawathe, P. (2021). Healthcare Simulation Standards of Best PracticeTM Simulation Design. Clinical Simulation in Nursing, https://doi.org/10.1016/j.ecns.2021.08.009.

[8] Jansen Bernard J. (2020) Data-Driven Personas for Enhanced User Understanding: Combining Empathy with Rationality for Better Insights to Analytics. https://www.sciencedirect.com/science/article/pii/S2543925122000560

[9] OpenAI (2022), Aligning language models to follow instructions https://openai.com/research/instruction-following

Quiz 3