-->ILLUMINATIO
-->Intelligent Illuminating Product Design Based on Machine Learning (original version of ILLUMINATIO)

Environmental Illumination plays a crucial role in human health and work efficiency, impacting mood and circadian rhythm. Advancements in computer vision and machine learning create new opportunities for designing adaptive pervasive computing devices situated in the home and workplace. In this work in progress, we apply literature on the human impact of lighting,identifying key design factors that we used to develop ILLUMINATIO: a pervasive computing artificial illumination  device that utilizes live camera input and a biologically-informed ML system to learn from user behaviors and autonomously adjust its lighting and projection configurations, offering the user engaged, yet adaptive control on the environment. The system centers alignment with users’ personalized habits while promoting key factors for wellbeing. Through this prototype, we explore the implications of designing for adaptive, situated, pervasive illumination devices with AI-integrations, and consider the broader implications for smart home environments and ubiquitous computing lifestyles.

In smart homes, illumination significantly improves the convenience and comfort of family life. However, current smart home products are limited to basic functionalities, such as employing user-preset scenes and operating appliances based on sensor detection. These processes merely change the manner in which operations are executed[1,2,3,4]. AI, with its ability to dynamically adjust outcomes based on evolving data[5,6,7,17], holds considerable promise for surpassing the intelligence of current smart home automation.
‍
Therefore, this project aims to explore AI-driven adaptive illumination device to better support user health and productivity in a personalized manner. Following research into ubiquitous computing and illumination, we developed a prototype of ILLUMINATIO, utilizing AI-driven technology to adapt lighting to the user's circadian rhythm and meet the requirements of various scenarios.

Ubiquitous Computing/Smart Home
There is a long history of HCI literature that merges digital interactions with physical environmental systems to improve user experiences in the home and in the workplace, including Weiser’s original visions of a future characterized by “calm technology” – invisible, responsive interfaces embedded into the environment that act in our periphery[15,16]. Rogers motivates a vision where pervasive technologies might be specifically situated, serving specific purposes, and actively engaged with[14]. Rather than being invisibly embedded into the environment, users can initiate interactions with pervasive technologies to empower their influence over the environment. These ideals motivate the design of environmental control systems that is both intuitively adaptive, and that invites active user engagement – a promising application for AI-driven, adaptive systems that are embedded into adjustable everyday objects: like an artificial light source. Currently available smart lighting products[1,4] can adjust brightness and color temperature throughout the day, however, these products fail to leverage the rich information we can learn each time a user adjusts their illumination settings.
‍
Illumination
In an extensive examination of how office workers interact with and control their lighting environments Moore et al.  linked office productivity with illumination, and found that workers preferred a wide variety of lighting conditions, influenced by several factors (age, time spent using a screen)[13].

Studies on light as a factor of health have found that illumination can affect hormone levels in the human body, synchronize biological clock, and impact the coordination of bodily functions and rhythms[8,9,10,11,12]. A biologically biologically-informed artificial lighting system should adapt to the user's circadian rhythm by supporting the processes that define the body’s active and rest phases, using both visual and non-visual light effects[11,12].

Using AI-Driven Interactions for Responsive Devices
Other ubiquitous computing products and prototypes have utilized AI and Machine Learning to drive more responsive and adaptable environmental interactions [5,6,7,17]. This project leverages AI-driven, biologically informed adaptive interactions with environmental illumination, with a focus on dynamically adjusting the projection angle and lighting characteristics of a situated desktop artificial light source based on the user's habits. This approach attempts to create alignment with the user’s preferences and active interactions with the system, as well as their passively expressed preferences and biological rhythms, towards a system that allows us to explore the benefits of responsive, personalized, and engaged experiences with situated, biologically informed ubiquitous computing devices.

To motivate our prototype, consider the following scenario:

Bob works from home. After over a month of use, ILLUMINATIO has learned his work habits and understands his circadian rhythm from the data recorded during his interactions with it, providing him with dynamically adjusted lighting throughout the day to enhance his work efficiency and stabilize his biological clock.
‍
In the morning, as Bob starts working on his computer, ILLUMINATIO's camera captures the computer's image and automatically provides bright, cool-toned lighting from an angle that doesn't produce glare. This promotes the secretion of cortisol[9], fully awakening Bob and getting him into a work mode[10], while also providing ambient light reducing the contrast between the work area and environment to minimize visual fatigue. During midday break, ILLUMINATIO offers bright warm light, slightly reducing Bob's tension while maintaining his circadian rhythm, aiding in relaxation[8]. In the afternoon, as he resumes work, IILLUMINATIO's light gradually becomes cooler, activating Bob's work state. Later, when Bob places a book on the desk, ILLUMINATIO's camera captures and recognizes the book, automatically adjusting the lighting angle to evenly illuminate the book, with the color temperature adjusted to daylight for optimal color rendering. Later, when Bob wants to focus on reading, he activates the Focus Mode, and ILLUMINATIO turns off the ambient light and provides focused illumination on the area where the book is located, reducing visual distractions from the surroundings. In the evening, ILLUMINATIO gradually reduces brightness and warms the color temperature, encouraging his body to produce more melatonin and ensuring a smooth transition into rest[8]. In the evening, as he starts using his phone, ILLUMINATIO dims the workspace lighting, and provides dimmer warm light as ambient lighting, making the environment suitable for evening use of mobile devices and promoting sleep.

ILLUMINATIO uses biologically effective artificial lighting to create a personalized natural lighting environment within Bob's circadian rhythm, enhancing his health and work efficiency[9]. Throughout the process, the system autonomously adjusts to suit Bob’s illumination needs throughout the day; there is no need for manual adjustment of lighting brightness or angle(1). However, Bob can choose to actively interact with the system and make an adjustment to their lighting or angle, and ILLUMINATIO will use this additional information to better respond to the his future behavior patterns.

Hardware Design of Intelligent Lighting Products

We opted for a microcontroller to handle processing calculations and control the hardware. To cater to both desktop and ambient lighting needs, the ILLUMINATIO system incorporates 2 color LED lights for desktop and ambient illumination; a camera for object recognition (identifying work scenes); a light engine for focus mode lighting; and 4 servo motors for multi-angle adjustment of the lamp arm.

To facilitate capturing images with the camera and performing calculations over the internet, we selected the Raspberry Pi 4 Model B. Four 30kg servos provide stable support and flexible rotation; additional hardware includes commonly available open-source LED modules and cameras, and the light engine is adapted from an existing product, a LeJiaDa YG200 Mini Projector. For ease of prototyping, we designed the drawings in AutoCAD and used laser cutting on 3mm aluminum plates to mount the hardware components and a 360-degree turntable for the base (2).

The design of the product casing and structure starts from functionality; its dimensions were defined by our desktop lighting experiments and the smallest volume necessary to accommodate the integrated circuits.
‍
The final structure is designed for the rotation angle of the lamp arm, affording (360º) rotation during use, and to overall aesthetically convey a sense of anthropomorphic charm and intelligence.
‍

The design of the product casing and structure starts from functionality; its dimensions were defined by our desktop lighting experiments and the smallest volume necessary to accommodate the integrated circuits.
‍
The final structure is designed for the rotation angle of the lamp arm, affording (360º) rotation during use, and to overall aesthetically convey a sense of anthropomorphic charm and intelligence(2).

We modeled ILLUMINATIO with Rhino, focusing on advanced surfaces, the overall form is simple and elegant, with square tops and bottoms blending into a circular cross-section in the middle. The middle part of the lamp arm, devoid of electronic components, is hollowed out, supported by three dynamic G4 continuous curves, with wiring running through them, the microcontroller installed at the bottom, and 4 servos within the bottom and up to the third section of the lamp arm(4).

Focus Mode
We used OpenCV for contour extraction from desktop images captured by the camera, to identify the location of books, thereby achieving accurate projection of the light source.

Using an open-source paid interface from Baidu, and further processing the obtained information: after extracting edge point coordinates, a new image is created with a black background and then draw the required color of light for illumination needs on the image and output to the light engine, providing illumination for focus mode. The result of the illumination contour matches expectations(5).

Intelligent Illumination Adapted to User Work Habits and Needs
To evaluate the algorithm, we created a partially synthetic dataset using data on the second author’s actual habits using a normal adjustable lamp. This dataset included 8 features and 1000 individual entries, each  representing the optimal illuminance and light angle under various conditions and taking into account such factors as time, ambient light, and the specific task being performed.

A K-Nearest Neighbor model is employed to forecast both illuminance and direction. This machine learning model exhibits notable efficacy when applied to the behavioral dataset thanks to its advantage for accommodating clustered data(6). Furthermore, its robustness to outliers could mitigate the disturbance of noise within real-world scenarios.

The training process emulates the device's assimilation of user behavior patterns from streaming data. Before the device has been used, the training set comprises three predefined observations. Upon each user activation of the device, a new observation is appended to the dataset for each adjustment to the illuminance and light direction. Subsequently, the model leverages both the latest input and historical data to forecast forthcoming illuminance and light direction configurations. The prediction accuracy is evaluated by computing the absolute difference between the predicted values and the actual values(8). The error quickly diminished to a negligible level to the human eye after 16 rounds of updates(7).

Our future work will further explore the how user behavior might be impacted by the presence of such adaptive, pervasive computing devices in their environment -- as the devices gradually learn their user's behavioral patterns, necessitating less adjustments over time, will the devices fade into invisibility as Weiser predicted[16], especially as the performance of learning algorithms progress or will the messy unpredictability inherent to everyday life entail that users remain in an engaged role, continually taking intentional actions for devices to continually respond to and extend?

In this project, we developed ILLUMINATIO and discovered the potential of AI-driven lighting systems in achieving adaptive environmental control based on biological information. By embedding an AI-driven learning system within an everyday object, we empower users with the choice to interact with the pervasive computing system. We grant the environmental illumination device the capability to utilize user adjustments and our understanding of biologically effective light to learn their behavior patterns, thereby exerting a positive, adaptive influence on their life and wellbeing.

REFERENCES
[1] [n. d.]. Desk Lamps — dyson.com. https://www.dyson.com/lighting/desk-lamps. [Accessed 14-03-2024].
[2] [n. d.]. Home app — apple.com. https://www.apple.com/home-app/. [Accessed 14-03-2024].
[3] [n. d.]. A home that knows how to help. — home.google.com. https://home.google.com/welcome/. [Accessed 14-03-2024].
[4] [n. d.]. Huawei Whole Home Intelligence — consumer.huawei.com. https://consumer.huawei.com/cn/wholehome/. [Accessed 14-03-2024].
[5] Paolo Coppola, Vincenzo Della Mea, Luca Di Gaspero, Raffaella Lomuscio, Danny Mischis, Stefano Mizzaro, Elena Nazzi, Ivan Scagnetto, and Luca Vassena. 2010. AI Techniques in a Context-Aware Ubiquitous Environment. Springer London, London, 157–180. https://doi.org/10.1007/978-1-84882-599-4_8
[6] Daniel Jorge Viegas Gonçalves. 2001. Ubiquitous computing and AI towards an inclusive society. In Proceedings of the 2001 EC/NSF Workshop on Universal Accessibility of Ubiquitous Computing: Providing for the Elderly (Alcácer do Sal, Portugal) (WUAUC’01). Association for Computing Machinery, New York, NY, USA, 37–40. https://doi.org/10.1145/564526.564538
[7] Fatima Hameed Khan, Muhammad Adeel Pasha, and Shahid Masud. 2021. Advancements in Microprocessor Architecture for Ubiquitous AI—An Overview on History, Evolution, and Upcoming Challenges in AI Implementation. Micromachines 12, 6 (2021). https://doi.org/10.3390/mi12060665
[8] licht.de organization. 2012. licht.wissen No. 02 “Good Lighting for a Better Learning Environment“. https://www.licht.de/en/service/publications- and-downloads/lichtwissen-/-series-of-publications.
[9] licht.de organization. 2012. licht.wissen No. 04 “Light as a Factor in Health“. https://www.licht.de/en/service/publications-and-downloads/lichtwissen-/-series-of-publications.
[10] licht.de organization. 2012. licht.wissen No. 04 “Office Lighting: Motivating and Efficient“. https://www.licht.de/en/service/publications-and-downloads/lichtwissen-/-series-of-publications.
[11] licht.de organization. 2016. licht.wissen No. 01 “Lighting with Artificial Light“. https://www.licht.de/en/service/publications-and-downloads/lichtwissen-/-series-of-publications.
[12] licht.de organization. 2018. licht.wissen No. 21 “Guide to Human Centric Lighting (HCL)“. https://www.licht.de/en/service/publications-and-downloads/lichtwissen-/-series-of-publications.
[13] T Moore, DJ Carter, and AI Slater. 2002. A field study of occupant controlled lighting in offices. Lighting Research & Technology 34, 3 (2002), 191–202. https://doi.org/10.1191/1365782802lt047oa arXiv:https://doi.org/10.1191/1365782802lt047oa
[14] Yvonne Rogers. 2006. Moving on from weiser’s vision of calm computing: Engaging ubicomp experiences. In UbiComp 2006: Ubiquitous Computing:8th International Conference, UbiComp 2006 Orange County, CA, USA, September 17-21, 2006 Proceedings 8. Springer, 404–421.
[15] Mark Weiser. 1999. The computer for the 21st century. ACM SIGMOBILE mobile computing and communications review 3, 3 (1999), 3–11.
[16] Mark Weiser and John Seely Brown. 1996. Designing calm technology. PowerGrid Journal 1, 1 (1996), 75–85.
[17] Dingtian Zhang, Jung Wook Park, Yang Zhang, Yuhui Zhao, Yiyang Wang, Yunzhi Li, Tanvi Bhagwat, Wen-Fang Chou, Xiaojia Jia, Bernard Kippelen, Canek Fuentes-Hernandez, Thad Starner, and Gregory D. Abowd. 2020. OptoSense: Towards Ubiquitous Self-Powered Ambient Light Sensing Surfaces. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 4, 3, Article 103 (sep 2020), 27 pages. https://doi.org/10.1145/3411826

-->ILLUMINATIO
-->Intelligent Illuminating Product Design Based on Machine Learning (original version of ILLUMINATIO)