JOURNAL TRANSCRIPT
A Survey on HCI Applications for Aiding Children with Mental Disorders
Rudraraju Praneeth Varma
Department Of Computer Science Michigan Technological University CS5760 Topic Assignment 2
[email protected]
Abstract In this paper, we will survey on HCI applications for aiding children with mental disorders. These disorders not only affects the person’s communication but also the way he interacts with peers or crowd. We don’t have any knowledge on cause of these disorders but we know that it is not a psychological cause but it is a biological cause from research. Till now there is no procedure to treat these disorders, so we have come up with an excellent idea of using HCI applications for aiding children with mental disorders. So we have come up with an application of HCI which supplies the user with virtual reality environment where we could also use ultrasonic gesture recognition for detecting and recognizing gestures using Google’s Day Dream software. Index Terms- Human Computer Interaction(HCI), Autism, Bipolar disorder, Daydream, Ultrasonic sensor(US sensor) and Virtual Reality(VR)
Introduction People suffer from mental disorders. Even children might suffer from the same problem. It becomes difficult for adults to identify if their children are suffering from the mental disorders especially bipolar disorder and autism. As we also know that the children might not possess the ability to express their concern. This acts as a barrier between adult and child. Due to these mental disorders, the person often experiences mood swings, behavioural swings and poor concentration. Even though few mental disorders can be cured by proper treatment strategies, autism and bipolar disorders don’t have treatment to cure them.
The application of HCI can have a huge impact on our behaviour. HCI plays a key role in providing support for examining a person’s emotion in the form of tools. This can be used to examine a person’s mood. Though there are different mental disorders, HCI provides a common path to tackle these mental disorders. In the next phase we will discuss history of mental disorders.
Background and Related Work A mental disorder is a state where person suffering from this has lack of potential to function. We don’t know the exact reason behind the cause of mental disorder. Based on a person’s behaviour, his thoughts and perception combined together we can actually come up with a procedure to calculate if a person suffers from mental disorder. Mental disorder is a phase of an absence of mental illness. People having mental disorders are often depressed and tend to be introverts. So they would never like to communicate or socialize with people. Mental disorder is also known as mental illness in few countries.The mental disorders can be genetically inherent, so the family has to take proper care of offsprings by making a visit to hospital for check-up so that the mental disorders can be identified in the child in initial stage itself so that proper treatment can be done to child. Otherwise, it becomes difficult for the doctors as the mental disorders may be in secondary or tertiary stages where we cannot treat the person accordingly. Apart from mental disorders there are neurological disorders and we also have intellectual disorders, thought disorders, eating disorders and sleeping disorders. Disorders may last temporarily or they may last for the rest of life. Foreign studies on mental disorders have shown that nearly 30 to 40 percent of people suffering from mental disorders recover from the symptoms of mental disorder without taking any medicines. We are mainly focussing on two types of mental disorders namely bipolar disorder and autism. The person who is identified with bipolar disorder has a fair chance of no longer being identified with it within 2 months. Almost everyone diagnosed with bipolar disorder are not identified with it after 2 years. But half of them encounter some or other form of serious depression in the span of next 2 years. In an article published in the year 2011, it is brought to account that nearly half of the population having mental disorders are in age between 10 to 24. Also accidents were one of the major cause of mental disorders which accounted to about 12 percent of total mental disorders. Autistic disorder is a type of mental disorder that cannot be treated and the person with this disorder suffers for
the lifetime. Male is more likely prone to this disorder rather than a female as much a 4 times. It belongs to a type of disability that affects the development of a person. As much as three-forth of the people suffering from Autism tend to have an Intelligence Quotient of less than 70. Remaining one-fourth of the people have IQ of normal people so these people are having functional Autism. This disorder ranges from mild to serious.
Our Proposed System Our proposed system is based on an application of HCI which supplies the user with virtual reality environment which has beautiful scenery like nature park, forest which has greenery in abundance. Also the user is supplied with music that matches the virtual environment. The user can also interact with various objects that are provided with the virtual reality environment through the gestures. We have come up with an idea to use ultrasonic gesture recognition that is having a size of chip scale and also low power consumption. We can develop application using DayDream software. The user gets to choose from a wide range of available environment.
B) Software used for our application Fig 1 :- Components of our proposed system
Every environment is integrated with suitable music and effects so that the user enjoys the mix of environment and music that matches environment so that the user especially children can have a lot of entertainment interacting with the surroundings of created environment. Depending on the user selection, the output of the HCI application can be seen on Google Virtual Reality Headset. This Human Computer Interaction inculcates the user to interact with various surroundings so the user will also improve the interaction with peers or crowd. In the application we also create a simulated environment where the user meets new characters and he gets rewards if he interacts with the characters in application. The user’s voice is detected by the ultrasonic sensor so the user is made to have conversation for the user to reach next phase of the application. If user does not have conversation, the ultrasonic sensor cannot detect sound so the user cannot move to the next phase of the application. This will improve the user’s communication and interaction. In this we will include applications that inculcate the user with game based learning. So the user feels as if he is playing games but the user would also learn new things. As we know that the IQ for most users of this application would be less than 70, we will create apps that are full of gestures. The games in the application will be having various levels and as the user completes each level he is promoted to next level. As the levels in the game increase, the user have to put more intelligence to complete the level. The user can also play good games like spot the differences in our application. So the user will also improve his IQ by playing the games. Our aim to create an application at low cost and useful to many people.
Fig 2 :- Block diagram of our proposed system
Components of our block diagram:1) Mobile Application:Firstly, the user has to wear the Google VR headset connected to Ultrasonic sensor. Then the user has to open the application, select the environment he is interested in and then the music that is relevant to the environment is played so that the user would have a great experience using the mobile application. Apart from selecting environment, the application will also contain lot of games where user would have lot of interactions within the application.
Fig 3:- Spot the differences game in our application
2) Virtual Reality headset connected to ultrasonic sensor:The user must wear the virtual reality headset connected to ultrasonic sensor. Google virtual reality headset is recommended as the we are developing an application using google’s daydream software. The user could also use google cardboard virtual reality handset as it is cost effective. The user’s voice is detected by the ultrasonic sensor so the user is made to have conversation for the user to reach next phase of the application. This will indeed improve the way user interacts and communicates in outside world as well.
Fig 4:- User wearing VR headset connected to US sensor
Comparison of existing systems with our proposed system In this section, we will compare our proposed system to existing system. The existing system is developed based on Kinect Framework v1.5 and XNA Game Framework 4.0. A Kinect sensor is used for user tracking and gesture detection. Based on the input from
the Kinect sensor, the HCI application displays the outcome of user interaction via a projector and a speaker. The design of the existing application allows the therapist to choose between different backgrounds, musical sounds and effects presented in the application to create a virtual scenario where children are able to interact with the surroundings while observing different effects according to the formerly selection. This existing system gives the therapist more control and flexibility over different treatments for specific individuals. But this system consumes lot of power. Another existing system was developed to aid autism that manipulates raw sensory information that the child would perceive by increasing saliency of social stimuli. For the sake of simplicity, the authors have limited their study to visual attention. Possible hardware for sensing environment and projecting the manipulated information to child's eyes might be a lightweight head mounted camera and display set. The degree of saliency intensification depends on how far his or her current attention point. Therefore, someone whose attention quickly gets to social stimuli does not experience so much intensification as a person who is not socially stimulated gets. Such social eye glasses can be used when attending to people is very desirable. The idea here is that the authors could influence saliency in a given location by manipulating its feature maps. For instance, magnifying the intensity level in that location could increase saliency of a location. This scheme can cause a low-level pressure on a child’s attention through this bottom-up process. The goal is to put this pressure at socially stimulating locations that autistic children do not normally find attractive. Even this consumes lot of power and the system setup is large. Another existing system was developed to aid bipolar disorder that uses image processing and pattern recognition techniques to recognize child's emotional state. This data is then fitted to an emotion model which predicts future states of the user. In order to stabilize user's mood, the system tries project the opposite mood of the child on his or her state. This projection is achieved by using an attractive and influential synthetic character capable of expressing some basic emotions. This time hardware is simpler; a normal monitor and a camera mounted somewhere in front of the user are enough but the power consumption is high.
So we have come with an idea to use ultrasonic sensor connected to VR headset and we are developing the application using daydreams, a software provided by google. The US sensor uses sound as a source for gesture recognition and detection. This
application would be more interactive with the user, as the ultimate aim of this application is to improve communication and interaction of the children suffering from autism or bipolar disorder. We are also concentrating on game based learning that can aid the child to improve his intelligence as most of the people suffering from mental disorders have low intelligence. The proposed setup consumes less power and is size of a chip scale.
Challenges We ought to develop our proposed project in such a way that it could help to users suffering from any mental disorder as our proposed application helps in aiding only bipolar disorder and autism. We should also take into account of people who are physically disabled and include haptic displays in proposed setup. We could include Support Vector Machines or Artificial Neural Network in the proposed system to get continuous feedback, but embedding either of them in to our system is difficult. We could also ask the user for feedback while the user uses the application. Even though we have embedded every environment with music, we should consult a music specialist and take his valuable suggestions for continuous improvement of the application. By improving our application step by step, the proposed system would become robust.
Conclusion We have provided a solution for aiding children with mental disorders using HCI application. The proposed system intends to develop a method and model that aids children suffering from autism and bipolar disorder. The proposed system is cost-effective and it works effectively. It is implemented by using ultrasonic sensors connected to VR headset. We would develop the application using daydreams, a software provided by google. The US sensor uses sound as a source for gesture recognition and detection. This application would be more interactive with the user, as the ultimate aim of this application is to improve communication and interaction of the children suffering from autism or bipolar disorder. We are also concentrating on game based learning that can aid the child to improve his intelligence as most of the people suffering from mental disorders have low intelligence. The proposed setup consumes less power and is size of a chip scale.
References [1] Surapa, T., Swetha, D., “A HCI Application for Aiding Children with Mental Disorders” Proceedings of the 7th International Convention on Rehabilitation Engineering and Assistive Technology, 2013 [2] CDC, Prevalence of Autism Spectrum Disorders — Autism and Developmental Disabilities Monitoring Network, 14 Sites, United States, 2008, Morbidity and Mortality Weekly Report, vol. 61, pp. 1-19, 2012. [3] Baluja, S., Pomerleau, D., "Non-intrusive gaze tracking using artificial neural networks", Technical Report CMU-CS-94-102, Carnegie Mellon University, 1994. [4] J. Hulsegge and A. Verheul, Snoezelen: Another World, 1987. [5] Braun, J., Julesz, B., "Withdrawing attention at little or no cost:detection and discrimination tasks", Percept Psychophys, 1998,60:1-23. [6] Rice CE, Baio J, Van Naarden Braun K, Doernberg N, Meaney F J, Kirby RS, for the ADDM Network. A public health collaboration for the surveillance of autism spectrum disorders. Paediatr Perinat Epidemiol 2007;21:179–90. [7] G. E. Lancioni, A. J. Cuvo, and M. F. O'Reilly, Snoezelen: an Overview of Research with People with Developmental Disabilities and Dementia, Disability and Rehabilitation, vol. 24, pp. 175-184, 2002. [8] Dilsaver, S.C., Chen, Y., Swann, A.C., Shoaib, A.M., Krajewski, K.J., ,"Suicidality in patients with pure and depressive mania", American Journal of Psychiatry. 151, 1312–1315, 1994. [9] Christian, J., "Autism and Related Disorders Handbook", USD School of Medicine and Health Science Center for Disabilities, 2002. [10] A. Rosenzweigh, What is Snoezelen Therapy and is it Helpful in Alzheimer's Disease? , About.com, 2011.
[11] Dawson, G., Meltzoff, A. N., Osterling, J., Rinaldi, J., Brown, E., "Children with autism fail to orient to naturally occurring social stimuli", Journal of Autism and Developmental Disorders, 28, 479–485, 1998. [12] Griswold, K., "Management of Bipolar Disorder", American Family Physician, September 15, 2000. [13] Koch, C., Ullman, S. ,. "Shifts in selective visual attention: towards the underlying neural circuitry", Human Neurobiology, 4(4), 219-227, 1985. [14] Mobahi, H., "Building an Interactive Robot Face from Scratch", Bachelor of Engineering Final Project Report, Azad University, Tehran-South Campus, Tehran, Iran, May 2003. [15] Norcia, N., "Innovative research advances knowledge and treatment of mood disorders", Sunnybrook & Women's News, Vol. 14, Aug. 16, 2004. [16] Mundy, P., Sigman, M., Ungerer, J., Sherman, T., "Defining the social deficits of autism: The contribution of non-verbal communication measures", Journal of Child Psychology and Psychiatry and Allied Disciplines, 27, 657–669, 1986. [17] Papolos, D., Papolos, J., "The Bipolar Child", New York: Broadway Books, December, 1999. [18] Yoon, H., "Using Robots to End AI: Artificial Inference", Yale Scientific Magazine Issue 78.2, Winter 2004. [19] Stiefelhagen, R., Yang, J., Waibel, A., "Tracking eyes and monitoring eye gaze", Workshop on Perceptual User Interfaces, Banff, Canada, 1997. [20] Picard, R. W., "Affective Computing", MIT Press, Cambridge, MA ,1997.
Bibliography Praneeth Varma Rudraraju (
[email protected]) is graduate student in the department of Computer Science at Michigan Technological University. His research interests include Computer Architecture, Artificial Intelligernce.