Target 1

How to recognize emotions effectively? ----------AI facial recognition and bracelet heart rate

First Circle

First Design

The first engineering goal facing our project is: how to effectively identify the expression interest. Because only on the basis of the correct recognition of emotions, we can achieve an effective emotion regulation process.

In this regard, Luo Zijin, who is responsible for the software in our group, decided to use python to call the camera to implement expression capture. In the way of using convolutional neural network contains the picture of the face will be transformed into a 48*48 matrix as the input of convolutional neural network, using the algorithm convolutional neural network to compare the different expressions in the database material in the open source. The screenshots of expressions in the dataset are shown below:

First Design

We build systems that capture facial convolutional neural networks and perform emotion judgment through software analysis and database comparison. For the input device, we dynamically capture the user's facial information through a camera and collect the data required for emotion analysis. For the output device, we wrote a program that can analyze the convolutional neural network and compare the analysis results, and output the input information and data analysis and comparison as the corresponding real-time emotion of the user. The preliminary schematic-construction is shown in the following figure:

As shown above, the image input is mainly processed by the convolutional layer, and then the fully-connected layer performs the expression score classification, and finally outputs the corresponding result

First Test

As shown in the picture, our classmate in the software group, Zijin Luo, tested the facial recognition program we wrote to call the camera to judge its own expression using convolutional neural network. We call the camera that comes with the laptop, and the program will contain the picture of the face will be transformed into a 48*48 matrix as a convolutional neural network input. The algorithm Convolutional Neural Network is used to compare the combined scores of different expressions of the database material, and finally the most similar emotion results are inferred according to the relevant scores. This is the accuracy figure of our first round of recognition results.

First Learn:

As can be seen by analyzing the data in the above figure, the accuracy of our first round of model testing rose rapidly at the beginning of training, and began to slow down after the tenth round, and basically increased to a halt after twenty rounds, maintaining it in the 60% judgment accuracy range. In the first round of iterations, we achieved an effective expression recognition process. However, the overall accuracy is effective, and the final accuracy of 68% does not provide effective recognition and judgment guidance.

Second Circle

Second Design

In the first iteration, we camera to achieve expression capture, the use of algorithms Convolutional Neural Network to compare different expressions of database material in open source although the method can achieve process closure. However, the overall recognition effect was average, and the 62% accuracy did not satisfy the actual usage effect.

In the group meeting of the software group, Wang Zhixin proposed that: based on the traditional machine learning process, it is necessary to have better data sets as samples for giving machine learning. Our initial recognition selection was open source of someone else's black and white processed expression data, but it was obviously a bit harsh to let the machine learn to determine the change of expressions of different students in real time.

After discussion, we decided to use a combination of the user's own facial features capture and database assisted reference, using the user's own features as the key input reference to retest the improved process.

Second Build

(1) First, we call the laptop's own camera to capture the user's expressions (after user authorization) while capturing the user's own key facial features, taking into account the positioning office scenario and the user's hardware burden.

(2) Then, the image containing the face will be transformed into a 48*48 matrix as the input to the Convolutional Neural Network

(3) The algorithm Convolutional Neural Network is used to compare its own feature values with the database samples and calculate the combined scores of different expressions

The expression with the highest score will be displayed

The combination of our user's own facial feature capture, and database assisted reference, uses the user's own features as a key input reference to retest the improved process.

Second Test

We improved the user's own facial feature extraction by adding a pre-processing module for user feature extraction to the above model calculation, and the final recognition accuracy obtained is shown in the following figure:

Second Learn

We found that after adding the pre-processing of the user's own features, the recognition accuracy of the model was greatly improved, adding up to an increase of about 11 percentage points from the original 68% accuracy. Although the second modified model has one more step of the pre-processing process and the effect converges more slowly in the first ten training rounds, the recognition accuracy is significantly increased afterwards.

Third Circle

Third Design

After the second iteration, we developed the recognition program as software and combined it with the light source system afterwards. After a period of practical application, we found that: in the test of the emotion recognition system, our system is basically accurate for the recognition of expressions. But sometimes people's complex emotions may be subjectively or objectively passively disguised. For example, they may pretend to be calm when they are sad, or consciously and deliberately make expressions that are different from their actual emotions. This all brings new challenges to the effective recognition of our system.

To address this issue, we have explored alternative approaches to emotion recognition through literature research. Although emotions such as expressions are subjectively influenced, they are easy to capture and quick to respond, but there is an inherent defect in recognizing specific situations. To address this situation, I started to think about whether there is a more accurate emotional response signal that does not "lie" and is easy to capture. This would complement the failure of facial recognition methods in specific situations.

Third Build:

In response, Zhixin Wang from our group thought that we could use a common device, the smart bracelet, to assist us in judgmental recognition. Using the bracelet to achieve the directional capture of heart rate, especially for the special peak change to match the situation.

We finally designed a judgment method combining facial recognition and heart rate, so as to improve the accuracy of judgment and avoid the possibility of certain mis-touch judgment. So we designed a bracelet user's heart rate for detecting auxiliary input with reference to sports bracelets and other related devices.Eventually we constructed a new comprehensive judgment system combining heart rate recognition of a bracelet and facial recognition as shown in the following figure:

Third Test:

We built a bracelet system to capture heart rate, using PPG (photoplethysmographic, photo volumetric pulse wave tracing method) to record the pulsation state of blood vessels and measure pulse waves by illuminating human blood vessels with LED light sources and using sensors to measure the attenuated light reflected from human blood vessels and tissues.

Firstly, Wang Zhixin of our group chose a sports bracelet for heart rate detection. BLE (Bluetooth Low Energy) communication is used to transmit between the bracelet and the computer. Compared with the classic Bluetooth method, BLE also has the characteristics of fast transmission speed, long transmission distance and large broadcasting capacity. The bracelet measures the user's heart rate and then broadcasts the user's current heart rate outward via BLE. The phone receives the heart rate and uploads it to the cloud for analysis.

Then, Zijin Luo from our group combined the previous facial recognition algorithm with the heart rate condition measured by the bracelet. In order to calculate the combination of both data, we developed the relevant comprehensive judgment software. We also combined gender, age and other key information input to achieve the correction of the heart rate model of the bracelet, and added the protection of user privacy instructions. The overall software implementation process is as follows:

(1)Enter the program and a help window will pop up, prompting the user to enter the range of parameters to be obtained:

(2) When the user enters the parameters correctly, clicking the Start button will bring up the Privacy Statement button, asking the user to confirm permission to use the camera and stating that the information entered will be encrypted and will not be saved:

(3) After the user accepts, the recognition program will start running, the main window will display the heart rate (the program currently uses random numbers for heart rate), while the camera window and the window displaying real-time MAL will pop up, and the MAL window will display the recognized emotions

(4) Overall recognition accuracy effect comparison: we can find that with the addition of the bracelet heart rate sampling as an auxiliary set, the overall effect of the model achieves an effective increase, and the effect is more obvious after 20 rounds of iterations.

Third Learn:

After several rounds of iterations, we added pre-processing extraction of user expressions in the pre-recognition stage of facial expression recognition, which led to an overall increase of 11 percentage points in model recognition.After that, we added the hand ring as a supplement to physiological signal acquisition, which effectively avoids the shortage of facial emotion recognition in some specific situations. We combined the two multiple signals to corroborate each other, so that the model as a whole achieved high convergence accuracy.

It can be said that it is also this iterative engineering upgrade that effectively improved the overall recognition model, and also laid the pilot foundation for later transformation into light signal and emotion regulation, and also let me experience the charm of engineering.

Target 2

How to identify emotional outcomes to odor regulation -------- Light-Responding Switch

First Circle

First Design

The first problem we have to solve in order to construct a light-initiated expression system is how to get a working light source promoter. Only if the promoter is working properly can we achieve our goal of using the light source to initiate gene pathway expression.

In response, we designed a gene expression pathway that uses the blue light photosensitive protein EL222(BBa_K4427003)to regulate the red fluorescent protein mRFP. E. coli BL21 can synthesize the blue light photosensitive protein EL222 upon induction of L-arabinose, and blue light irradiation can alter its LOV structural domain and initiate the expression of the downstream red fluorescent protein mRFP1（ BBa_K4427006）. By examining the final product, red fluorescent protein, we can determine whether the blue light initiation system is working properly.

First Build

To achieve the above functions, we constructed the pSB1C3-stufffer-pro (BBa_K4427009) plasmid using pSB1C3:

①Ara operon(BBa_K4427003):UCL iGEM 2017 employed a constitutive promoter ahead of EL222 to construct the light control system. However, by literature retrieval, we found there was a risk of expression leakage for constitutively expressed EL222, making it difficult to demonstrate the efficacy of blue light induction intuitively (compared to quantified characterization). Thus, ara operon was involved in the pathway in control of the expression EL222.
②EL222(BBa_K4427003):EL222 is a blue light photosensitive protein. Upon irradiation with blue light at a wavelength of 465 nm, the LOV structural domain of EL222 undergoes metatable activation and the metatable EL222 binds to the PBlind promoter to initiate the expression of downstream genes.
③mRFP1( BBa_K4427006):mRFP1 is a red fluorescent protein. The excitation wavelength is 584.9nm and the emission wavelength is 612.

First Test

Firstly, we tested whether our pSB1C3-stufffer-pro plasmid transformed properly in BL21 by CmR resistance screening, plasmid miniprep, XbaI and SpeI double digestion nucleic acid electrophoresis experiments, and the experimental results proved that we obtained the engineered bacterium BL21-Pro.

We then used arabinose to induce the synthesis of EL222 protein by BL21-Pro under light-proof conditions, followed by cyclic experiments with alternating blue light irradiation and dark incubation of the engineered bacteria, and finally UV irradiation, bacterial confocal characterisation and fluorescence spectroscopy to detect the red fluorescent protein mRFP1.

The experimental results (available on the RESULTS page https://2022.igem.wiki/jlu-china/results) that BL21-Pro synthesizes the red fluorescent protein mRFP.

(A1 and A2: Expression control results of blue light-initiated red fluorescent protein mRFP1, A1 is the experimental group with red fluorescence produced under UV, right is the blank control; A2 is the experimental and control PCR tubes without UV irradiation; B: Fluorescence spectroscopy results of red fluorescent protein mRFP1. The excitation wavelength is 612 and the emission wavelength is 589, which is consistent with the characteristics of mRFP1; Right: Confocal characterization of BL21-Pro microstructure, 1000x )

(top left: blue light-initiated expression control results for red fluorescent protein mRFP1; bottom left: red fluorescent protein mRFP1 fluorescence spectroscopy results; right: BL21-Pro Microstructure confocal characterisation,1000x.)

First Learn

After research and analysis of the experimental results within the group, our blue light starter system is able to effectively control the expression of gene pathways. Compared to other light-controlled components, it can be better adapted to our needs. We can replace the red fluorescent protein gene with a gene related to the synthesis of 2-PE on the basis of this pathway.

Second Circle

Second Design

After proving that the blue light promoter works, the second problem we had to solve was how to use blue light to regulate 2-PE expression. We found the biosynthetic pathway of 2-PE.

Based on this pathway, we have improved on the blue-red fluorescent protein expression pathway and designed a heterologous blue-light expression system for overexpression of.

This system allows E. coli to initiate the expression of 3-deoxy-D-arabinoheptulose-7-phosphate synthase and the bifunctional enzymes branching acid metatase (CM)-prephenate dehydratase (PT), KdcA and Adh1, which catalyse the progressive synthesis of 2-PE from glucose via a cascade reaction, and this pathway enhances 2-PE production through overexpression of the enzyme, under the modulation of blue light.

①DS:DS (3-dcoxy-7-phosphoheplulonate synthas) is an important protease that controls the synthesis of aromatic amino acids by regulating enzyme activity through an allosteric mechanism. The synthesis of PEP (phosphoenolpyruvate) to DAHP (3-deoxy-D-arabino-hept-2-ulosonate 7-phosphate) can be catalysed.

②CM-PT:CM-PT (chorismate mutase/prephenate dehydratase) is a key enzyme in the L-Phe anabolic branching pathway, catalyzing the decarboxylation, aromatization and deacylation of prephenic acid (PRE) to produce phenylpyruvate, which is then subjected to a pyridoxal phosphate (PLP)-dependent transamination reaction to produce L-Phe. This is followed by a pyridoxal phosphate (PLP)-dependent transamination reaction to produce L-Phe.

③KdcA(KdcA BBa_K4427005):KdcA (branched chain alpha-keto acid decarboxylase) is a keto acid decarboxylase that converts phenylalanine PPA to PAD, which in turn is converted to 2-PE.

④Adh1( BBa_K4427004):Adh1 (alcohol dehydrogenase) converts the above-mentioned PAD catalyzed by KdcA into the final product 2-PE, a geotactic process that uses the genetically catalyzed product PAD of the former enzyme, which is then converted to the final product 2-PE by Adh1.

Second Build

To achieve the desired function, we integrate this synthesis pathway and constructed the pSB1C3-stuffer-all（BBa_K4427012 ） plasmid. The map is as follows:

Second Test

First of all, the transformation of pSB1C3-stuffer-all into BL21 was examined before the functional validation. Theoretically, bacteria with successfully transformed plasmids should be chloramphenicol-resistant due to the function of CmR. However, only a small number of colonies were actually observed on the culture media with chloramphenicol after several repeated experiments. Amplification and plasmid extraction was performed on the "fortunate" minority, but only to find the result disappointing—no qualified plasmids were obtained.

Second Learn

A group discussion was held to look for possible explanations for the frequent failure of plasmid transformation. We speculated the main reason to be the extra size of our plasmid, which contained a whole lot of genes to be expressed, making it difficult for the engineered bacteria to absorb or process them.

To find a solution, we reviewed a large amount of literature and found that E. coli itself contains the gene pathway to synthesize PPA, the precursor of 2-PE. So, all we needed to do was to introduce plasmids merely containing two key enzyme genes, KdcA and Adh1 to enable the synthesisis of 2-PE in the engineered bacteria by shortening the overall length of the exogenous genes.

Third Circle

Third Design

Based on the assumptions above, we designed a new blue-light expression system, the EL222-KdcA-Adh1 expression pathway.

In this system, BL21 synthesized protein EL222 in response to L-arabinose induction. Upon blue light irradiation, EL222 initiated the expression of downstream genes KdcA and Adh1 to synthesize 2-PE from its precursor PPA previously produced by E. coli itself.

Third Build

The shortened plasmid pSB1C3-stuffer-LP (BBa_K4427002) was then constructed by omitting the aroF gene for DS synthesis and the pheA (fbr) gene for CM-PT synthesis. The map of the updated plasmid is as follows:

Third Test

In order to test whether the newly designed plasmid could be transformed properly, the same screening process was carried out by culturing the post-transformation BL21 on the medium with chloramphenicol. As expected, the bacteria managed to survive, demonstrating our engineering success.

Next, to verify the synthesis 2-PE, we performed a series of experiments on the engineered bacteria, which included 1) plasmid extraction with electrophoresis to validate the transformation completion, 2) double-digestion with electrophoresis to validate the correct insertion of the three genes of interest, and 3) SDS-PAGE and measurement of enzyme activity to validate the capability of the engineered bacteria to synthesize active KdcA and Adh1.

(top left: UV absorption spectrum of coenzyme NADH in the KdcA cascade reaction; top right: concentration-dependent enzymatic activity of Adh1 enzyme; bottom left: absorption spectrum of Adh1 coenzyme NADH; bottom right: cascade catalytic activity of a blue light-initiated bacterial solution containing a dual enzyme plasmid of KdcA and Adh1)

Results above (see details on the RESULTS page https://2022.igem.wiki/jlu-china/results) demonstrated that the engineered bacteria could produce 2-PE. Quantitative assays were further conducted on our samples and gas chromatography was performed with standard 2-PE. The results confirmed that our samples were 2-PE with yields of 90-100 mg/L.

(top: mass spectra of extracted 2-PE samples in bacterial broth; bottom: gas chromatograms of extracted 2-PE samples in bacterial broth)

Third Learn

On reflection and reviewing, we learnt that the BUCT team were also synthesizing 2-PE. We reached out to team BUCT and found that our designs differed in the expression pathway. The synthetic strategies of the two teams are shown respectively as follows.

1.JLU_China:

2.BUCT:

Our engineered bacteria are capable of synthesising 2-PE in yields of 90-100 mg/L, compared to 20-60 mg/L for BUCT's method of synthesis.

In order to raise our yields, we held several discussions with BUCT. Combined with additional literature research, we learned that tyrB encodes an aromatic amino acid transaminase that plays a key role in the synthesis of aromatic amino acids and 2-PE. Lack of the tyrB gene might restrain the intracellular movement of phenylethylaldehyde, leading to its massive accumulation inside the cell, thus inhibiting the cell growth.

We therefore assumed the addition of an exogenous tyrB gene to the engineered bacteria could possibly reduce this effect and might increase the expression of 2-PE.

Fouth Circle

Forth Design

In order to obtain relatively higher 2-PE expression, we designed a fourth-generation pathway of the blue light expression system.

TryB(tyrosine aminotransferase):Deletion of the tyrB gene resulted in unbalanced flux, large amounts ofaccumulated phenylacetaldehyde and repressed cell growth.

In comparison to the original blue-light expression system, the expression of 2-PE was increased by reducing the inhibition of cell growth by aldehyde.

Forth Build

We modified the pSB1C3-stuffer-LP (BBa_K4427002) plasmid by adding the TyrB gene to construct the pSB1C3-stuffer-LP2 ( BBa_K4427021) plasmid. The improved plasmid was ligated as follows:

Constrained by the time limit, we did not perform a complete test of the fourth-generation blue-light expression system. If possible, we'll validate this system and explore more strategies to boost the production of phenylethanol in the future.

Target 3

How to ensure biosafety when using products - Red and blue light suicide systems

First Circle

First Design

In order to ensure the safety of our device in real-life application, we designed the red- and blue-light suicide system together with Hainan University on the basis of the lethal gene ( BBa_K3739028) provided by Xiamen University. When irradiated with both red and blue light, the engineered bacteria would initiate the expression of toxic proteins.

Considering the experimental time, we only designed the blue light-induced suicide system by replacing the mRFP1 gene in the pSB1C3-stuffer-pro plasmid with a virulence protein gene, and then performed the verification experiment.

Due to the specificity of the pSB1C3-stufffer-pro plasmid, the experiments could not be carried out with suitable enzymatic sites found on both sides of the fluorescent protein, so the experimental steps were adjusted and modified, and four methods were found for the extraction and ligation of the plasmid and the toxic protein sequence.

First Build

We constructed the pSB1C3-stuffer-BDH（BBa_K4427009）plasmid by replacing the red fluorescent protein gene in the p1C3-stufffer-pro (BBa_K4427015) plasmid with a virulence protein gene.

After thorough communication, the agreed experimental procedure was to amplify the pro plasmid-containing punctures and extract the pro plasmid, followed by enzymatic ligation and verification of the ligation results, followed by blue light induction of expression after transformation to obtain the experimental results.

Due to the specificity of the pro plasmid, it was not possible to use the appropriate enzymatic sites on both sides of the fluorescent protein for the experiments, so the experimental procedure was modified and four methods were found for the extraction and ligation of the plasmid and the virulent protein sequence.

Option 1: The pro plasmid vector is linearised by reverse pcr, with primers for both NdeⅠ and PacⅠ at both ends for pcr. The virulent gene fragment is amplified by pcr and then double ligated and transformed. The disadvantage is that the reverse pcr length of the pro plasmid is too long, over 4000bp.

Option 2: The vector is linearised in such a way that the pro plasmid fluorescent protein sequence is disrupted and some fluorescent protein sequence remains. Blra toxicity gene fragment was amplified by the PCR method. The target plasmid was then obtained by double digestion ligation method.

Option 3: The vector is linearised in such a way that the fluorescent protein sequence is destroyed and some of the fluorescent protein sequence remains. and enzymatic sites are not common, amplification efficiency needs to be considered. blra toxicity gene fragment amplified by the PCR method. A homologous recombination kit is then used and a one-step cloning is performed.

Option 4: The pro plasmid vector is linearised by reverse pcr without enzymatic sites and the virulence gene fragment is amplified by pcr and then cloned in one step using a homologous recombination kit. The disadvantage is that the reverse pcr length of the pro plasmid is too long, over 4000bp.

(top left: amplification of puncturing bacteria; top right: pro plasmid attempted pcr pre-amplification , 1 and 5 are fragments obtained from method one and method four respectively, 2 and 6 are dilutions of the template 10-fold, 3 and 7 are dilutions of 20-fold, 4 and 9 are dilutions of 100-fold, 8 is added incorrectly and 9 is a make-up of 8. (Below: amplification of the blrA toxicity gene fragment)

(From left to right, top to bottom, Blra gum recovery concentrations for scenarios I, II, III and IV)

First Test

After obtaining the recombinant plasmid using the above method, we carried out transformation experiments using BL21 to obtain BL21-BDH and carried out control experiments with blue light irradiation and dark culture for BL21-BDH. The experimental results showed that no colonies grew in either the control or experimental groups, and the pro plasmid was difficult to reconstitute by conventional methods.

First Learn

After conducting several experiments, none of the expected results were achieved. After communication with the Hainan University team, an analysis of possible causes was carried out.

1. Not completely protected from light and toxic genes leaked into natural light.
2. Transformation failure due to inaccurate control of heating temperature and heating duration.
3. The plasmid base number was too large and spontaneous spitting occurred, so we had to keep changing the sensory state to try.
4. Due to reverse pcr, the pro plasmid is too long and there are various factors that affect the trailing band and stray band.
5. Enzyme are not common.

Second Circle

Second Design

The design has been continuously improved so that all problems except for the difficulty of constructing the pro plasmid can be effectively solved. As the pro plasmid is difficult to reconstitute by conventional methods and we have previously validated the function of the pro plasmid. Given the limited time available, we decided to validate the toxin alone first. If the virulence protein works properly, then the lethal task can be accomplished by ligating the blue light initiation system (BBa_K4427009) to the virulence protein.

Second Build

In order to verify whether the virulence gene protein expressed normally, we designed a control experiment together with HainanU_China, expecting bacteria that expressed the virulence protein not to grow on the plate, while the bacteria without this part to grow as the control group to better prove the detrimental function of the virulence gene that put our bacteria to death.

Second Test

HainanU_China added 1 μL of the virulence gene plasmid to 50 μL of competent E. Coli cells, spread the plates after transformation, added 50 μL of the solution to each plate, and had them incubated overnight at 37°C. The results are shown below, which proved their engineering success.

(Left: engineered bacterium introducing a virulent protein pellet; right: engineered bacterium introducing a plasmid but without a virulent protein gene.)