A research team of Professor Inseok Hwang (CSE) and Integrated Ph.D Candidate Jungeun Lee, Suwon Yoon, and Kyoosik Lee (CSE) has developed a personalized storybook generation system designed to assist children’s language education. The team collaborated with Prof. Dongsun Yim’s team of Ewha Womans University to create this system using generative AI (GAI) and home IoT-based technology. The research was presented at the ACM Conference on Human Factors in Computer System (CHI) where it received the Honorable Mention Award, a recognition given to the top 5% of papers.
Language skills in children are crucial as they influence not only cognitive and academic abilities but also social interactions. It is essential to continually assess whether children’s language abilities are developing appropriately and provide language interventions. However, the challenge lies in the fact that: although children grow up in diverse environments, conventional methods have used standardized vocabulary lists to assess language abilities alongside pre-existing storybooks and toys for intervention.
The research team identified the limitations of these uniform and standardized tools. In response, they designed an effective educational system that considers each child’s unique environment. By utilizing home IoT devices, the team collected and monitored the speech that children hear and produce in their homes. They then analyzed the words and calculated scores for linguistically significant elements associated with each word.
The team employed GAI, including GPT-4 and Stable Diffusion, to create personalized storybooks that incorporate target vocabulary for each child. By integrating linguistic theories with experiences of professionals in the field, the team developed an effective personalized language education system for children.
To account for individual differences in language development, the system was designed to allow for modulation of various criteria in vocabulary selection, tailored to each child’s needs. The entire process is automated, enabling continuous updates as children’s language skills and environments evolve. In a four-week trial of nine in families, the system proved effective in-home environments.
The research team aims to create personalized guides tailored to the needs of more diverse individuals using GAI. The team anticipates that their research will contribute to forming education that respects the diversity of children’s learning environments and goals.
This research was supported by the National Research Foundation of Korea, the Institute of Information & Communications Technology Planning & Evaluation’s University ICT Research Center, and the ICT R&D Innovation Voucher Support Program.
