The Services of Electro-mechanical Care Agencies (SELEMCA) lab was a research facility in Amsterdam dedicated to exploring novel assistive technologies for groups such as elderly persons and people suffering from (mental) health disorders. More specifically, our mandate included the development of systems built around human interactions with robots, computer-based agents, and virtual worlds. We refer to these systems as Caredroids, devices or software that will provide a variety of medical services.

As traditional channels for secondary health care become scarcer due to aging populations and smaller family sizes, supplementing or replacing existing support networks with technology such as Caredroids will become increasingly critical. The increasing demand for care services cannot be solved by productivity improvements alone.

A new approach is the use of creative technological solutions to supplement and replace existing care-services. These solutions include agents, robots, ambient and virtual worlds; mechanotronic robots that we call Caredroids - Product-Service Systems (PSSs) that create a better fit between carer and patient. In SELEMCA, a PSS is a value proposition consisting of a marketable set of robots and avatars that provide services such as brokerage, coaching, tutoring, conversation, and companionship as related to healthcare. They were designed and combined to optimally fulfill healthcare needs from the perspectives of the patient, caretaker, care professional, and care managers. Clients may not attach particular value to the electro-mechanics or virtual environments themselves but rather to the functionality that robots and their digital counterparts offer. SELEMCA focused on care brokerage, exercise coaching, Q&A on wellbeing, companionship, and health education.

What questions were answered?
Teams from 3 universities and industry partners were working together to address the questions:

How do users of Caredroid experience these novel product-service combinations – both in terms of having humanoid company and as interactive healthcare tools?
How to improve the relationship between Caredroids and humans in terms of creativity and aesthetics?
How to establish trust in the application of Caredroids in the medical domain?

What have the teams achieved?
The team has established a transdisciplinary design theory of human-android interaction by investigating the human affective system and emotion regulation. This lead to the development of generic knowledge instruments and design guidelines to create care services that are emotionally intelligent and human-oriented.

Timeframe and communication:
SELEMCA started in 2011 and finished in 2015. The project generated innovative scientific knowledge in the forms of new learning technology facilitating brainstorming and problemsolving. Scientific output was distributed in journal publications and conference proceedings. Throughout the project, we have communicated information and results via this website.

Who was involved?
Scientific partners: Vrije Universiteit Amsterdam, Delft University of Technology, University of Twente, Inholland;
Creative and Industry partners: Osira, GermansMedia, IC3D Media, Roessingh Enschede, Aernout Mik, Waag Society, Lost Boys, Mentrum/Arkin, Appsterdam, Hanson Robotics.

Project leaders
Dr. Dr. Johan F. Hoorn, CAMeRA@VU


Talking CT Head

The Talking CT Head is an integration of ps-medtech desktop virtual reality, IC3D Media game technology, and the Luminis PulseOn e-learning system. SELEMCA agent technology will be applied in the next phase.
Ps-medtech is the company that developed the VR hardware and made the 3D CT scans navigable. Medical Technology Inholland provided the expert knowledge on, for example, radiology. Their students and teachers are the end-users of this system. IC3D Media is a company dedicated to the development of game engines and avatar technology. They designed the mouth layer over the CT scan head. They moreover developed the audio so that the head can talk about itself. Luminis/PulseOn is a company that delivered the e-learning platform for personalised learning.

What is it, for whom?
The Talking CT Head is a product-service system that supports e-learning in higher education, designed in a student and tutor-centered way, with great opportunities for scientific outreach to the public at large (e.g., NEMO, Corpus) and public health education. It induces the much-wanted “wow” effect, raising curiosity and instigating the motivation to learn.

Commonly, e-learning in higher education is associated with downloadable lectures, online tutorials, Podcasts, tablets, dedicated discussion forums, and sometimes even e-mail. The benefits are sought in solving logistical problems of the organisation (e.g., large-scale distribution of content and large-scale assessment of students) as well as measurable cost savings. The students can work more efficiently, because they are not constricted by a timetable and do self-assessments independent of time, place, or tutor. True and desirable as this may be, SELEMCA studies and designs an e-learning system that goes beyond the ‘digital filing cabinet,’ concentrating on pedagogical interventions on the fly.
In collaboration with Inholland and companies from the Creative Industries, SELEMCA developed a whole new learning concept for students of Medical Imaging and Radiation Therapy (MBRT). Here, a 3D Virtual Reality technique is used to visualise several scans of the human body (MRI, CT, etc.) such that the scans can be manipulated, while being educated through SELEMCA-designed emotionally adaptive “agent technology” (work in progress). In the 3D scans, students can “touch” organs and body parts while the system interprets and explains what they see. This way, the students learn anatomy and how it is represented on different scans. Knowledge can be tested, using the same system. This innovation makes students much more aware of the 3D aspect of treatment (i.e. radiotherapy) and diagnosis in the body. This applies not only to students of MBRT, but also for higher-vocational students of HBO-V and the medical masters ANP and PA. In time, this project offers good chances to qualify for external funding and exporting the concept to various other medical schools outside the university. The system will also attract media attention, serving scientific outreach and general health education.

How it works
The ps-medtech C-Station is a desktop virtual reality system - a 3D viewing station - designed for the medical professional for the analysis of complex 3D en 4D data such as MRI, CT, and Ultrasound in a way not previously possible in a traditional computing environment. Through optical tracking of multifaceted “dice,” the student can explore the brain architecture (or any other body part) in 3D. Enriched with IC3D Media animation and sound, Luminis’ logging and adaptation tools for personalised e-learning, and SELEMCA’s affective understanding of the student (work in progress), the educational content becomes self-explicating, interesting, and decidedly motivating.
SELEMCA’s emotion-simulation software will be applied to an animated talking head built of CT-scans, explaining what the student sees and what to look for. Enriched with exercises, the student’s progress can be logged by the PulseOn system and content matter can be adapted to personalised learning styles. The emotion simulations will help to motivate the student. SELEMCA’s inhouse designed creativity-software connects the educational contents to special interest areas of the student (e.g., a journey through the winding lobes is framed as a hike through canyons and gulches).

Next step
While the virtual reality system offers educational content (e.g., to diagnose trauma in a CT scan), in the next phase, agent technology will be connected to electrodes and sensors, measuring and responding to the student’s behavioural and electro-cortical patterns. Suppose, certain teaching styles become predictable after a while, then alpha-wave activity increases. Traditionally, this would lead to a wake-up call to stir arousal and insisting on focused attention, not changing the teaching strategy but changing the student’s state of mind. In creative teaching, however, the moment of reverie is where problems are solved in an associative style rather than rationally so that the system changes its teaching strategy and offers associatively related contents from which the student can solve the problem from a completely different angle. For example, if the challenge is to recognise brain-tissue softening in a range of CT scans and the student fails repeatedly or falls asleep, the system offers a cooking app to bake a pudding with gelatine and tofu. Then it returns to the original question, asking the student to mentally apply the molten gelatine images to the CT scan. Performance is measured through PulseOn and the strategy that renders most correct responses is continued. In other words, e-learning new style is a merger between neuroscience, interactive multimedia, pedagogy, and creativity.