Proof of Concept

With the increasing social pressure, people's moods tend to change unsteadily. When faced with blows, people often feel sad and depressed and cannot control their emotions well. The long-term sadness and depression can affect people's health. In response, the JLU_China team has invented a mood regulator that automatically identifies emotions and releases different substances according to them. It uses AI recognition and heart rate monitoring of the bracelet to identify emotions, translate them into light signals, and regulate E.coli to synthesise odour substances that can regulate emotions. An emotion regulation system combining AI intelligence and synthetic biology is constructed, and the overall device is divided into three processes: recognition, judgement and adjust.

Firstly, when the user's mood changes in waves, the AI recognition and the mood fluctuations detected by the bracelet will be analysed and judged in the background to classify the user's mood into different categories, such as sad mood. Then, this change in emotion captured by the recognition will be successively transformed into different light signals, we set the output of 465nm blue light signal when feeling sad emotion. Finally, E.coli receives the light signal and produces the appropriate chemicals to regulate the emotion and calm the mood.

Recognition process: The device starts with facial recognition and a bracelet monitoring system to determine the user's mood at the moment, with facial recognition using camera capture and convolutional neural network calculation, and heart rate detection using the user wearing a bracelet detector as the basis. We also strictly comply with data protection regulations and user authorisation regulations during the recognition process to protect the user's privacy.

Judgement process: We use the heart rate readings from the bracelet as the threshold for emotion judgement, combined with the emotion facial recognition results for multimodal calculations. The obtained data is processed and analysed, and emotions such as anxiety and sadness are subsequently output as various combinations of red and blue light signals. In the blue light-responding switch, we selected the EL222 protein and LOV structural domain, which upon activation by light will recruit RNA polymerase to initiate downstream gene expression; in the red light-responding switch, the BphP1 protein is used as a photosensitive protein, which upon activation by light initiates downstream gene expression.

Adjust process: The regulation process is divided into two main parts and release links of the red light-responding switch and the blue light-collected system.

1. (1) Blue light-collected system: Upon recognition of the user's sadness, the blue light-collected system is activated and transcribes KdcA and Adh1, which are key enzymes for the synthesis of 2-PE. A large amount of 2-PE molecules are then synthesised inside the E.coli and escape from the cell by simple diffusion due to their high lipid solubility, calming the mood.
2. (2) Red light-responding system: After recognising the user's anxiety, the red light-responding switch is switched on and BphP1 is activated under far-red light conditions at 760nm.
3. (3) Finally, the fragrance is released through the hardware.

To accomplish these three processes, we have set seven engineering goals. So far, we have confirmed that our projects in mood recognition, signal conversion, designing plasmids and engineered bacteria, blue light-controlled system, red light-responding systems, red and blue light suicide systems, detection and release systems have been validated as described in our design page.

To prove our concept, the core modules of our project have now been validated. Based on the experimental results available so far, we have determined that our solution is feasible and can be further optimised and scaled up.

The main components of our project (sadness capture, blue light signal conversion and phenylethyl alcohol odour molecule release regulation) have now been fully validated and the experimental results prove that our project proposal is fully feasible.

1. Mood Recognition System

The first engineering goal facing our project was how to effectively recognise expression interest. For the recognition part, we mainly considered the use of a combination of facial expression recognition and a bracelet heart rate system to accurately monitor the user's emotions. For the facial emotion recognition part, we use a camera to capture the user's face and then combine it with a convolutional neural network to make a judgement; for the bracelet part, we use a PPG based on an LED light source and a detector, which converts the light signal into an electrical signal, extracts the changing AC signal from it and obtains the characteristics of the blood flow and thus the pulse rate data, thus achieving the monitoring of the user's emotion. The aim is to monitor the mood of the user.

1.1 AI Facial Recognition

To accomplish AI facial recognition, we build systems that capture facial convolutional neural networks and make sentiment judgments through software analysis and database comparisons. For the input device, we dynamically capture the user's facial information via a camera to collect the data required for sentiment analysis. For the output device, we wrote a program that analyses the convolutional neural network and compares the analysis results, and outputs the input information and data for comparison to the corresponding real-time emotion of the user. A preliminary schematic construction is shown in the following figure.

1.1.1 Process are Like

After user authorization we call the laptop's own camera to capture the user's expressions The image containing the face is transformed into a 48*48 matrix as input to the convolutional neural network The algorithm convolutional neural network is used to compare the combined scores of the different expressions in the database material (screenshots of the expressions in the dataset are shown below) After the iteration, we add a step: the emoji with the highest score will be displayed and combined with the bracelet results.

1.1.2 Results

This is a graph of the accuracy of the results of our first round of recognition.

As can be seen by analysing the data in the graph above, the accuracy of our first round of model testing rose rapidly in the early stages of training, and began to slow down after the tenth round, with a basic increase stopping after the twentieth round, maintaining it in the 60% judgement accuracy range. Although we achieved an effective expression recognition process during the first iteration, the overall accuracy was effective and the final 68% did not effectively provide a guide to recognition and judgement.

This is a graph of our second round of recognition results accuracy results.

We found that by adding pre-processing of the user's own features, the recognition accuracy of the model improved significantly, adding up to about 11 percentage points from the original 68%. The second round of modified models had a pre-processing process and converged more slowly in the first ten rounds of training, but the recognition accuracy increased significantly afterwards.

In the tests of the emotion recognition system, our system is generally accurate in recognising expressions. However, sometimes people's complex emotions may be subjectively or objectively passively masked. In order to be able to accurately identify the relevant emotions of the user, we used a bracelet worn as a heart rate to capture these physiological signals as a valid witness, thus greatly improving the accuracy of the recognition.

1.2 Heart Rate Detecting by Bracelet

1.2.1 Process

When LED light is directed at the skin, the light reflected back through the skin tissue is accepted by the photosensitive sensor and converted into an electrical signal which is then converted into a digital signal by AD.

When the light signal is converted into an electrical signal, the changing AC signal can be extracted and the characteristics of the blood flow can be obtained, which in turn leads to pulse data.

After getting the user's heart rate data, the bracelet uploads the data to the server's MYSQL database via HTTP GET request and saves it.

1.2.2 Results

This is a graphical representation of the accuracy of our most recognised results.

After several iterations, we added pre-processed extraction of user expressions in the early stages of facial expression recognition, which led to an overall increase of 11 percentage points in the model recognition. We then added a hand ring as a supplement to the physiological signal acquisition, effectively avoiding the shortcomings of facial emotion recognition in certain situations, and combined with the mutual corroboration of the two multiple signals, the model as a whole achieved a high convergence accuracy. It can be said that it is this iterative engineering upgrade that has effectively improved the overall recognition model, and the effective recognition has also laid the pioneering foundation for later transformation into light signals and emotion regulation.

The above experiments successfully demonstrate that when emotions are captured correctly, the emotional signal-electrical signal conversion can be obtained smoothly through AI facial recognition and bracelet heart rate detection.

2. Signal Shift System

(1) blue light-responding switch: On recognizing the user's sadness, we convert the corresponding signal into a 465nm blue light signal, thus system activation.

(2) red light-responding switch: when the user's anxiety is recognized, a red light-responding switch combining 660nm and 760nm far red light will be switched on, so that BphP1 is activated under the far red light condition of 760nm.

(3) When the device monitors that the user is in a happy mood, red and blue light will be irradiated simultaneously, which will produce the toxic protein blrA to lyse the E.coli bacterium.

As shown below this is the pin design of our designed red and blue light source on a microcontroller system with the following drive motor to power the release device.

We use Pulse Width Modulation (PWM) for dimming, by changing the duty cycle of the output voltage to obtain different light intensities to meet the experimental requirements. For the red and blue light we chose 2835 lamp beads as the light source, with wavelengths of 465nm, 660nm and 760nm respectively, operating voltage of 3.1v, single power of 0.5w, and a luminous angle of 120 degrees. The light is controlled by the GPIO port of Arduino Uno R3.

The experimental results of the above signal conversion system show that the electrical signals have been successfully converted into blue and red light signals

3. Blue Light-Controlled System

3.1 Blue Light-Responding Switch

In the blue light starter system, we constructed the pSB1C3-stuffer-pro plasmid (BBa_K4427009), which functions to synthesize EL222 protein upon induction of L-arabinose, and blue light irradiation induces EL222 protein dimerization onto the pBLind to initiate the expression of downstream red fluorescent protein (mRFP1). To validate the above function of our blue light-responding switch pSB1C3-stuffer-pro plasmid (BBa_K4427009), we performed the following validation.

3.1.1 mRFP1 Expression

After determining that the plasmid was correctly constructed, we performed functional validation of the pSB1C3-stuffer-pro plasmid (BBa_K4427009) in a dark room. With reference to the literature we performed two rounds of experiments with the engineered bacteria in cycles of 2 hours of blue light irradiation and 2 hours of darkness under sheltered conditions, and a control experiment with complete protection from light. At the end of the two cycles, the results were observed in the following four aspects.

3.1.2 UV Lamp Fluorescence

The results of using Portable UV analyze to irradiate the EP tubes containing the bacterial solution showed that the experimental group showed significant fluorescence, while the control group showed no fluorescence. This indicates that the engineered bacteria synthesized fluorescent protein under the regulation of blue light.

(The graph on the left shows a comparison of the fluorescent proteins under 365nm UV irradiation, the blue light irradiated group (left) shows a clear fluorescence, the control group (right) does not show fluorescence. (The picture on the right shows the comparison under the natual light, the blue light irradiated group (left) appears light pink, the control group (right) had no red fluorescence.)

3.1.3 laser scanning confocal microscopy

The bacterial microstructure of the above fluids was observed under a laser scanning confocal microscopy.

The results show that our engineered bacteria synthesized red fluorescent protein under the regulation of blue light.

The above results indicate that the blue light system can be successfully activated after receiving blue light irradiation.

3.2 blue light expression system

We constructed the pSB1C3-stuffer-LP plasmid (BBa_K4427002) , which functions in the synthesis of EL222 protein induced by L-arabinose, and blue light irradiation induces EL222 protein dimerisation onto the Pblind Prometer to initiate the expression of downstream ethanol dehydrogenase (Adh1) and alpha-ketoacid decarboxylase (KdcA). Catalytic synthesis of the substrate into 2-PE (2-PE).

To validate the above functions of our blue light expression system pSB1C3-stuffer-LP plasmid (BBa_K4427002), we performed the following validation. With reference to the literature we performed a cycle of blue light irradiation for 2 hours and dark 2 hours under sheltered conditions for two rounds of 8 hours for the engineered bacteria, and a control experiment with complete light protection.

3.2.1 Activity of KdcA and Adh1

We measured the activity of KdcA and Adh1 using pyruvate as the reaction substrate and use a sulotion of Tris-HCL at PH=7 as a buffer to verify the functionality of the pSB1C3-stuffer-LP plasmid (BBa_K4427002) by using a cascade reaction in which KdcA catalyzes the reaction of pyruvate to produce acetaldehyde and Adh1 then acetaldehyde catalyzes ethanol. We added 10, 15, 20, 30, 35 and 40 μL of the dual enzyme enzyme solution to the substrate and performed six sets of control experiments. The experimental results showed that the amount of enzyme solution at 10-40 μL showed an increasing rate of substrate reduction with increasing enzyme concentration, i.e. the higher the enzyme activity. It shows that our plasmid can synthesise the final product 2-PE after the addition of substrate.

The above validation results for the blue light system successfully show that E.coli can express phenylethyl alcohol molecules efficiently after receiving activation from the blue light signal, achieving a successful light signal-biosignal-end product pathway.

3.2.2 Gas chromatogram and mass spectrometry

We purchased a 2-PE standard and compared it with the sample obtained from our experiments by gas chromatogram and mass spectrometry. The results showed that the experimental data of the samples obtained from our experiments and the standards were in general agreement, and the product obtained from our experiments could be identified as the final destination product 2-PE.

See more details in result

4. Red Light-Responding Switch

For verification of the red light signal, we ligated the sfGFP green fluorescent protein downstream of the red light plasmid and then co-transformed the plasmid with a control plasmid into E.coli and tested whether the plasmid could emit green fluorescence under fully shaded transformation conditions, as well as under 760 nm far red light irradiation to confirm the success of the plasmid transformation.

5. Suicide system

We use the suicide gene provided by XMU-China to terminate the user's "emotional physiotherapy". Our design is to produce the toxin through the simultaneous activation of the red light activation system and the blue light activation system. The red and blue light activation systems have been fully validated as described above, so we focus here on validating the toxin function.

We conducted a collaborative experiment with HainanU_China, who performed the validation of the virulence protein. HainanU_China chose the E.coli receptor state, added 1 μL of the virulence gene plasmid to 50 μL of the receptor state, coated the plates after transformation according to the transformation process, added 50 μL of the solution to each plate and incubated it overnight at 37°C, and finally validated it successfully.

The above experimental results show that our suicide system is fully feasible and that E.coli can be lysed at the right time under the right conditions.

6. Detect and release system:

With the user's feelings in mind, we have designed the unit in the form of a bladeless fan fan with a soft wind and a small air supply area, thus making the volatile substances more conducive to blowing to the user. The air blown from the bladeless fan is channeled through the internal air ducts so that there is no violent wind sensation impact in order to make the wind softer and smoother, thus making the user's experience more comfortable and thus more enjoyable. To achieve lightness and portability, we have designed the unit in the shape shown below.

Although the use of essential oils derived from plants for physical and mental health is increasingly seen as an alternative or therapeutic adjunct to improve the mental health of patients[3], given that the effectiveness of the end-products of the experiments on people's mood regulation cannot be verified in humans, we synthesised and compared a large number of literature controls to validate the considerations at the outset of the project. We note that the University of Lyon, France, recruited 19 healthy participants for a human emotion regulation experiment, including odour detection, and that the researchers detected the chemical responsible for the pleasurable emotions, namely 2-PE molecules, with satisfactory results [4]. Similarly, the study by Archana K Singh et al. could confirm the pleasurable sensations produced by 2-PE in humans [5].

Therefore, using the results of this experiment as the basis for our project, we turned to 2-PE, one of the main pharmacological active ingredients in rose oil [6], which was found to have a significant mood-pleasing effect. in which researchers continuously recorded cortical activity, heart rate, skin conductance and respiratory cycles, showed that the difference in absolute power of bands theta, alpha1, alpha2 and beta1 at eight locations increased after inhalation of 2-PE molecules, indicating their powerful pleasurable ability in humans [8].

In conclusion, there is no doubt that the experiment produced a modulating effect of the released 2-PE molecules on sad mood.

7. Overall

In summary, it is shown that the bracelet monitoring recognition and facial recognition system can monitor the user's emotions well and smoothly convert the collected emotional information into red and blue light signals, thus proving that the emotion recognition system and the signal conversion system can work well individually and work well as a whole.

The conversion of electrical signals into red and blue light activates EL222 and BphP1 fluorescent proteins, respectively, as a means of initiating the blue and red light pathways. In the experiments with the blue light-controlled system we have detected the production of phenylethyl alcohol at a concentration of 100 mg/L.

This shows that our engineered bacteria can produce the target product well and that it can be detected by the monitoring system, thus demonstrating that the blue light-responding switch and the monitoring system work well individually and as a whole. The product diffuses from the cell membrane into the device and is delivered to the user by our multi-construction model.

This proves that our release system works well. It can thus be seen that the seven engineering objectives of the whole system work well individually and in concert as a whole, and our experiments have proven to be an overall success.

8. Future

Although we have accomplished our goal of using scent substances to regulate the user's emotions, in the future we would like to further refine our product to achieve the ambitious goal of multi-modulation, considering that music can trigger and influence people's emotions, and that light often appears as a visual language for music to have a more multi-dimensional effect on people. The combination of light and music has a powerful emotional power and can have a diverse impact on people, so we plan to add ambient lighting or background music to the aromatherapy system, using audio-visual regulation of the user's mood.

8.1 Moods Extended

On the basis of the blue light-responding switch for sadness regulation, we have explored more emotions and their corresponding regulation extensibility, such as the red light-responding switch for regulating anxiety, and we will also develop more regulation systems for emotions and light sources on this basis, and we can even achieve the unification of ambient light and regulation system light colours, which is more convenient and environmentally friendly.

8.1.1 Extended Red Light Corresponds to the Mood

In the red light-collected system, we selected the red light-sensitive protein BphP1 from Rhodiola rosea, which, as a widely used photosensitive protein, is activated by far-red light irradiation at 760 nm and binds to the upstream promoter blocker protein PpsR2, enabling the initiation of transcription of downstream MVA pathway genes.

In order to ensure the accurate reliability of the red light-collected system, we have used separate experimental validation. As we have sufficiently demonstrated that the red light-responding switch can work well, we expanded the section mainly to demonstrate whether the red light-controlled system can work well, and plan to transform plasmids for each of the key proteins in the system (MVA, LIS) into E.coli to verify their expression and function.

Using the MVA pathway to synthesise linalool:

We will also explore the possibility of more light regulation systems for more emotions, such as red light, green light, etc., based on the overall closed-loop process of blue light sadness regulation. In this way, we can achieve real-time adjustment functions for more adverse emotions

8.2 Other Ways to Adjust Your Mood

The sense of smell is closely linked to emotion: we have found a very strong neural link between smell and emotion, with 75% of human emotions being produced by smell. The main site of interaction with the olfactory centre is the amygdala. Apart from smell, no other intuitive system can influence the amygdala, the area of the brain that controls human emotions, in such a unique and direct way. All our sensory organs react after thinking, except for odour, to which the brain reacts first and then enters into thinking.

On the basis of phenylethyl alcohol odour modulation, we then consider the role of other odorous substances, such as the sedative effect of linalool, which subsequently enhances the use of other odorous substances as well. In addition, we have taken into account the powerful emotional power of music and light, so we plan to add sound and light to the hardware system to assist in regulating the user's mood. The image below shows the sound and light PCBs we have designed, with the sound signal output and the light control signal output.

8.2.1 Music

Music can help people to shift their attention from negative emotions to something pleasant. Music also has a direct effect on the centres of the human brain, such as the hypothalamus and the limbic system, which are responsible for emotions and regulate them. Each piece of music has its own type of emotion, and we can get better results by matching the emotion to the type of music. Therefore, we expect to add a music playing device to the system, which will play different types of music according to the current mood and regulate the user's mood.

8.2.2 Colorful lights

Research by scientists has found that different light colours do have an effect on certain hormones in the human physiology, thus affecting the mood of the person. We expect to add a part of the colour adjustable light strip on the exterior of the device, which will emit different colours and intensity of light through the user's current mood condition, and with different light movements to achieve the purpose of regulating the user's mood. Different colours and intensities of light have different effects on positive and negative emotions, so we will design different lights according to different emotions in order to make the most of our equipment.

For example, pink light has a certain effect on emotional appeasement and relaxation. Pink light passes through the human eye, travels to the cerebral cortex, then to the hypothalamus, through the pineal gland and pituitary gland, stimulates the adrenal glands, causing the adrenal medulla to secrete less adrenaline, so that the human physiology can relax, the heart's activity gradually becomes slower, the muscles are also relaxed, thus making people feel relaxed and moderate their emotions. The blood pressure is lowered and the pulse rate is soothed by the blue light background. The mood is also relaxed and the state of mind is gradually relaxed under prolonged blue lighting, resulting in drowsiness.

9. Refences

[1] HUANG Rumin. Empight: Interactive Design of household atmosphere Light Effect Driven by music emotion [D]. Zhejiang University,2019.

[2] He Juan. Analysis of the Regulation of Music on Negative Emotion by ECG [D]. Southwest University,2016.

[3] Herz RS. Aromatherapy facts and fictions: a scientific analysis of olfactory effects on mood, physiology and behavior. Int J Neurosci. 2009;119(2):263-90.

[4] Manesse C, Fournel A, Bensafi M, Ferdenzi C. Visual Priming Influences Olfactomotor Response and Perceptual Experience of Smells. Chem Senses. 2020 Apr 17;45(3):211-218.

[5] Singh AK, Touhara K, Okamoto M. Electrophysiological correlates of top-down attentional modulation in olfaction. Sci Rep. 2019 Mar 20;9(1):4953.

[6]Umezu T, Ito H, Nagano K, Yamakoshi M, Oouchi H, Sakaniwa M, Morita M. Anticonflict effects of rose oil and identification of its active constituents. Life Sci. 2002 Nov 22;72(1):91-102.

[7] Ramadan B, Cabeza L, Cramoisy S, Houdayer C, Andrieu P, Millot JL, Haffen E, Risold PY, Peterschmitt Y. Beneficial effects of prolonged 2-phenylethyl alcohol inhalation on chronic distress-induced anxio-depressive-like phenotype in female mice. Biomed Pharmacother. 2022 Jul;151:113100.

[8] Brauchli P, Rüegg PB, Etzweiler F, Zeier H. Electrocortical and autonomic alteration by administration of a pleasant and an unpleasant odor. Chem Senses. 1995 Oct;20(5):505-15.