Optimization of Multi-Component Olfactory Display Using Inkjet Devices

Hiroya HACHIYAMA, Takamichi NAKAMOTO

Summary :

Devices presenting audiovisual information are widespread, but few ones presenting olfactory information. We have developed a device called an olfactory display that presents odors to users by mixing multiple fragrances. Previously developed olfactory displays had the problem that the ejection volume of liquid perfume droplets was large and the dynamic range of the blending ratio was small. In this study, we used an inkjet device that ejects small droplets in order to expand the dynamic range of blending ratios to present a variety of scents. By finely controlling the back pressure using an electro-osmotic pump (EO pump) and adjusting the timing of EO pump and inkjet device, we succeeded in stabilizing the ejection of the inkjet device and we can have large dynamic range.

• Publication: IEICE TRANSACTIONS on Fundamentals Vol.E107-A No.8 pp.1338-1344
• Publication Date: 2024/08/01
• Publicized: 2023/12/28
• Online ISSN: 1745-1337
• DOI: 10.1587/transfun.2023EAP1051
• Type of Manuscript: PAPER
• Category: Multimedia Environment Technology

1.  Introduction

Due to the recent development and spread of VR technology, VR experiences have become familiar to many people. Although it has become possible to easily experience VR at home, most of them are only visual and auditory experiences. In order to increase the reality of the experience, it is effective to add the presentation of odors using an olfactory display [1]. Regarding olfactory display devices, a scent projector [2], aroma card [3], MEMS device [4], wearable device [5], smelling screen [6] have been proposed. Regarding olfactory contents, Movie with scents [7], game with scents [8], [9], cooking simulator [10], disaster simulator [11] have been proposed. However, the more types of odors that can be presented by the olfactory display, the more scenes with scents can be represented.

In order to present many odors, it is necessary to prepare many perfumes. However, we can expect to efficiently create a variety of odors by blending multiple odors to create new one.

Although we proposed odor components to reproduce a variety of odors [12], a device to present smell is indispensable. Therefore, we have attempted to develop a multi-component olfactory display [13]. One method of presenting odors on an olfactory display is to eject and vaporize a liquid perfume. We have proposed multi-component olfactory displays using various droplet ejection devices and vaporizers [14], [15]. However, the dynamic range of the blending ratio was insufficient. There is the upper limit of ejection volume because the switching time of the previous device, i.e., solenoid valve, limits the increase in its drive frequency. Thus, the dynamic range is insufficient. On the other hand, the drive frequency of the inkjet device can be raised much higher than the solenoid valve, whereas the lowest frequencies in both devices are similar. In order to reproduce various odors with an olfactory display, it is necessary to blend multiple odor components (sources of odors) in a wide range of ratios.

Therefore, we focus on inkjet device since its droplet is tiny and the dynamic range of blending ratio can be expanded. Although we have developed olfactory display using inkjet devices [16], its stability was insufficient because it lacked the method to supply perfume to inkjet device. More sophisticated method is required to raise the stability. Although other olfactory display using inkjet devices have been studied [17], there have been no research on blending odors. Since scent pulse was very short in that paper, it is necessary to check its blending capability. Moreover, the detailed technical information was not enough to replicate it.

Since the inkjet device does not have a valve, its back pressure should be finely adjusted. Moreover, simple and tiny device is indispensable to control the backpressure of the inkjet device. In this study, we adopted EO pump to control the back pressure of inkjet device and timing of EO pump and inkjet device precisely so that the ejection stability could be improved. Then, we evaluated the improvement in ejection stability by comparing the concentration measurements using a gas concentration sensor called PID (Photo Ionization Detector).

2.  Overview of Olfactory Display Using Inkjet Device

The designed olfactory display is shown in Fig. 1. Liquid perfume is supplied to the inkjet device from the EO pump, and droplets are ejected toward the micro-ceramic heater. An EO pump is a small, low speed pump with a built-in liquid reservoir. The droplets are vaporized by the heater and delivered to the user’s nose by a fan. A 60 V pulse is used to drive the inkjet device, and one droplet is ejected with one pulse. In other words, by adjusting the frequency, the odor concentration can be controlled. Digital-to-Analog converter (DAC) is needed to adjust EO pump voltage since back pressure of inkjet device is influenced by its drive frequency. FPGA makes control signals for DAC and inkjet devices. The parameters of the pulse signal input to the inkjet device and the applied voltage of the EO pump can be controlled from the GUI on the PC. The current system can blend up to four perfumes at specified ratio.

Fig. 1  Structure of olfactory display.

3.  Control Design of Olfactory Display

3.1  Inkjet Device

The inkjet device (MicroFab, MJ-AB-01) has a structure in which a piezoelectric material surrounds a glass tube. When a voltage is momentarily applied to this piezoelectric material, pressure is applied to the glass tube, pushing out the liquid at the tip and ejecting one droplet. It is necessary to generate a continuous pulse signal with an arbitrary frequency to control the odor concentration. In this study, FPGA generates pulse stream when pulse width, its period and total number of pulses are specified. The pulse width is fixed at 30 \(\mu\)s, and the pulse period and the total number of pulses can be set to arbitrary values using the GUI introduced in Sect. 3.4. As shown in Fig. 1, the pulse parameters input to the GUI are sent to the FPGA via USB serial communication, and the FPGA generates pulse signals. Since FPGA outputs 3.3 V pulse, a 60 V pulse signal to drive the inkjet device is made through power MOSFET circuit.

3.2  Electro-Osmotic Pump

The EO pump (Takasago Electric, EBP-RF1R) generates a flow in the liquid in the built-in reservoir by applying a voltage, and the flow rate can be controlled by the applied voltage. Since the ejection amount of the inkjet device changes depending on the drive frequency, it is necessary to adjust the amount of liquid supplied by the EO pump according to the frequency. Therefore, as shown in Fig. 1, it is necessary to connect one EO pump to one inkjet device and set appropriate EO pump voltage for each channel. At initial stage, DC constant voltage was supplied to an EO pump. However, a multi-channel DAC was adopted now since each EO pump needs different voltage according to inkjet drive frequency. EO pumps consumes very little power, so it can be driven directly from the DAC. A multi-channel DAC (ANALOG DEVICES, AD5535) is a device that converts a digital signal into an analog voltage. By inputting 19-bit serial data called a DAC code, it can be output any voltage from 0 V to 200 V. Similar to the inkjet device, the voltage value input to any channel through the GUI on the PC is sent to FPGA, which sends DAC code to D/A converter. Then, the DC voltage at the specified channel is generated.

3.3  Heater and Fan

Both the micro-ceramic heater used as the vaporizer and the DC fan that generates the airflow to deliver the odor to the user are supplied with current and voltage from a DC power supply. A micro-ceramic heater (SAKAGUCHI E.H VOC CORP., MS-3) can raise the surface temperature by passing an electric current through it. By flowing a 0.2 A current, it is possible to keep the temperature between 200 and 300 Celsius to evaporate liquid perfume. The DC fan (SHICOH, F3010ES-12UCV) can adjust the air flow speed by the input voltage value.

In this study, both are driven by DC power supplies, but we plan to control the fan from an FPGA in order to synchronize it with the presentation of odors in the future. Since the micro-ceramic heater takes time to heat up, it is driven first.

3.4  MATLAB GUI

We created a GUI using MATLAB to easily control the inkjet device and the EO pump on a PC. Figure 2 shows the GUI. The channel to be used can be selected by clicking the check boxes from 1ch to 4ch. A user is requested to input the inkjet device drive frequency, the applied voltage of the EO pump, and the presentation time of the selected channel, and press “Write” to send each parameter to the FPGA.

Fig. 2  GUI to control each device.

When the user presses “Start”, the pulse for driving the inkjet device is generated with the sent parameters, and the DAC code for the voltage applied to the EO pump is sent to the D/A converter, thereby starting the ejection of the liquid perfume.

3.5  Jig

We made a jig with a 3D printer to fix each device such as inkjet devices, EO pumps, and a micro-ceramic heater. Figure 3 shows the jig for supporting these devices. The side views and geometries of EO pump jig and inkjet device jig above it are illustrated in Figs. 4(a) and (b), respectively.

Fig. 3  Jig for electro-osmotic pumps, inkjet devices and heater.

Fig. 4  Geometry of jigs for (a) electro-osmotic pumps and (b) inkjet devices.

4.  Experimental Result

4.1  Ejection Stabilization of Inkjet Device

4.1.1  Voltage Measurement of an EO Pump

Since the inkjet device does not have a valve at the tip, fine control of back pressure by an EO pump is essential. Figure 5(b) shows an ideal state at an appropriate back pressure. However, it is not easy to maintain this ideal state. As shown in Fig. 5(a), if the back pressure is lower than one required at specified driving frequency of the inkjet device, air bubbles enter the tip. If the back pressure is higher than required one as shown in Fig. 5(c), the liquid accumulates at the tip. In other words, it is necessary to investigate the optimum voltage value of the EO pump for each driving frequency of the inkjet device.

Fig. 5  State of tip of inkjet device due to relationship between back pressure by EO pump and drive frequency of inkjet device.

Several representative drive frequencies for inkjet device were selected, and the range of EO pump voltage that enables stable ejection at each frequency was investigated every 5 V from 1 V to 200 V. In addition, it was judged to be “ejectable” when the ejection was completed three times for 30 seconds without stopping even once. The results are shown in Table 1. The range of EO pump voltage that can be discharged for each frequency on the horizontal axis is shown in blue. For example, when the inkjet device is driven at 500 Hz, stable ejection is possible by setting EO pump voltage between 30 V and 100 V.

Table 1  Range of EO pump voltage that can be ejected (horizontal axis is inkjet drive frequency (Hz) and vertical axis is voltage applied to EO pump.

4.1.2  Adjustment of Driving Timing of EO Pump

In the previously developed prototype, the voltage of the EO pump was manually supplied from a DC power source. As a result, ejection became unstable when there was a discrepancy between the driving timing of the inkjet device and the EO pump. When the EO pump is driven first, the liquid accumulates at the tip of the inkjet device before the ejection starts. In this study, the EO pump and inkjet device become driven at the same time. Its timing can be controlled by the computer, whereas the timing could not be controlled by the previous manual operation.

4.2  Inkjet Device Ejection Volume Measurement

Although we previously thought that the ejection volume per droplet was constant and the blending ratio was determined by the frequency ratio, it is not true. The ejection volume per droplet changes drastically under constant back pressure condition. In this study, the ejection volume per droplet was measured at each frequency in order to determine the blending ratio based on the ejection volume.

Ethanol was ejected from the inkjet device into the vial, and the change in mass before and after ejection was measured with an electronic balance to calculate the ejection volume per droplet. Table 2 shows the ejection volume of one droplet measured at each representative frequency. Each value shows the average value of three measurements. It can be seen that in the range from 5 Hz to 1000 Hz, the ejection volume is stable around 200 pL regardless of the frequency. It is thought that fine adjustment of the back pressure enables the stable ejection volume over wide range of drive frequency. Since the ejection volume is almost constant, the blending ratio is identical to frequency ratio, and by using frequency from 5 Hz to 1000 Hz, a maximum blending ratio is 200.

Table 2  Inkjet device frequency and average ejection volume per one droplet.

4.3  Ejection Stability Evaluation by Random Sequence

In order to confirm that stable ejection can be achieved under various ejection conditions (inkjet frequency, ejection time, ejection interval time), the ejection conditions were randomized and ethanol was ejected continuously.

For a total of 12 ejection patterns with 4 inkjet frequencies (50 Hz, 100 Hz, 500 Hz, and 1000 Hz) and 3 types of ejection time (10 s, 20 s, and 30 s), the order was randomized and each pattern occurred 3 times. The ejected ethanol was vaporized on the heater, and the concentration was measured by a gas monitor called PID (Photo Ionization Detector, Honeywell, ppbRAE3000+). The PID detector was placed about 5 cm away from the heater. In order to confirm the improvement in ejection stability due to the driving timing adjusted in Sect. 4.1.2, we compared the case when the EO pump was driven 1 second earlier and the case when the inkjet device and the EO pump were driven simultaneously.

Figure 6 shows the PID response when the EO pump was driven 1 second earlier, and Fig. 7 shows the PID response when the inkjet device and EO pump were driven simultaneously. Only the first 10 responses out of 36 consecutive ones are shown for simplicity.

Fig. 6  PID response when EO pump was driven 1 second ahead of inkjet device. Graph shows only first 10 out of 36 consecutive measurements.

Fig. 7  PID response when inkjet device and EO pump were driven simultaneously. Graph shows only first 10 out of 36 measurements.

Figures 8 and 9 show the PID response at 100 Hz, 20 seconds when EO pump was driven 1 s earlier and when both devices were simultaneously driven, respectively. Vertical lines in the figures mean the start and the sop of the ejection. When the EO pump was driven 1 second earlier, it was confirmed that the peak of the PID response was unstable and the response did not decrease after the end of the ejection (smell remained).

Fig. 8  PID response when EO pump was driven 1 s ahead (inkjet drive frequency: 100 Hz, duration: 20 s).

Fig. 9  PID response when inkjet device and EO pump were driven simultaneously.

This is thought to be due to the fact that liquid accumulates at the tip (state shown in Fig. 5(c)) of the inkjet device before the start of ejection, leading to unstable ejection and residual odor. On the other hand, when the inkjet device and the EO pump were driven simultaneously, the response peak was stable, and no residual odor was observed. The ejection stability was improved by adjusting the drive timing, and stable ejection was achieved then under various ejection conditions.

The time lag of a few seconds occurred at the start and the stop of the ejection. It is acceptable since PID response time is 3 s [18].

4.4  Ejection Experiment of 4 Components

Ejecting from multiple channels at the same time was performed to confirm whether the specified ratio could be achieved. One or more channels were selected from 4 channels (15 ways of selection), and one measurement was performed for each selection. The driving frequency of the inkjet device was 1000 Hz. The ejection time was 30 seconds, and the ejection interval time was 30 seconds. Then, 15 consecutive ejections were totally performed. The ejected ethanol was vaporized with a heater and the concentration was measured by PID.

The results of 15 consecutive measurements are shown in Fig. 10. The number above each response indicates the number of channels driven simultaneously, and it can be seen that the number of channels driven and the peak value of the PID response are almost proportional. From this result, it is shown that blending is possible by the proposed olfactory display.

Fig. 10  PID response in 15 consecutive measurements. Number above each response indicates number of channels driven.

5.  Conclusion

In this study, we combined an inkjet device and an EO pump, synchronized the ejection timing of each device, and achieved stable ejection by finely adjusting the applied voltage of the EO pump according to the frequency of the inkjet device. Furthermore, we constructed a 4-component olfactory display and demonstrated the feasibility of blending by measuring the PID response when using multiple channels simultaneously.

In the future, we will conduct odor blending using actual perfumes and odor components, and evaluate the accuracy of odor blending using the olfactory display through sensory tests although its fundamental blending capability was confirmed here. Then, we aim to create an olfactory display that can reproduce various odors by further increasing the number of components.

Acknowledgments

This work was partially supported by JST Mirai project No.2269685.

References

[1] Y. Yanagida, “A survey of olfactory displays: Making and delivering scents,” SENSORS, 2012 IEEE, pp.1-4, 2012. doi: 10.1109/ICSENS.2012.6411380

[2] F. Nakaizumi, H. Noma, K. Hosaka, and Y. Yanagida, “SpotScents: A novel method of natural scent delivery using multiple scent projectors,” IEEE Virtual Reality Conference (VR 2006), pp.207-214, 2006. doi: 10.1109/VR.2006.122

[3] D.W. Kim, Y.H. Cho, K. Nishimoto, Y. Kawakami, S. Kunifuji, and H. Ando, “Development of aroma-card based soundless olfactory display,” 2009 16th IEEE International Conference on Electronics, Circuits and System-(ICECS 2009), pp.703-706, 2009. doi: 10.1109/ICECS.2009.5410784

[4] H. Kim, J. Park, K. Noh, C.J. Gardner, S.D. Kong, J. Kim, and S. Jin, “An X-Y addressable matrix odor-releasing system using an onff switchable device,” Angewandte Chemie International Edition, vol.50, no.30, pp.6771-6775, 2011.

[5] J. Amores and P. Maes, “Olfactory interfaces for unconscious influence of mood and cognitive performance,” Proc. 2017 CHI Conference on Human Factors in Computing Systems, pp.28-34, 2017. doi: 10.1145/3025453.3026004

[6] H. Matsukura, T. Yoneda, and H. Ishida, “Smelling screen: Development and evaluation of an olfactory display system for presenting a virtual odor source,” IEEE Trans. Vis. Comput. Graphics, vol.19, no.4, pp.606-615, 2013. doi: 10.1109/TVCG.2013.40

[7] T. Nakamoto and K. Yoshikawa, “Movie with scents generated by olfactory display using solenoid valves,” IEICE Trans. Fundamentals, vol.E89-A, no.11, pp.3327-3332, Nov. 2006.

[8] A. Mochizuki, T. Amada, S. Sawa, T. Takeda, S. Motoyashiki, K. Kohyama, M. Imura, and K. Chihara, “Fragra: A visual-olfactory VR game,” ACM SIGGRAPH 2004 Sketches, p.123, 2004. doi: 10.1145/1186223.1186377

[9] N. Ranasinghe, K.C.R. Koh, N.T.N. Tram, Y. Liangkun, K. Shamaiah, S.G. Choo, D. Tolley, S. Karwita, B. Chew, D. Chua, and E.Y.-L. Do, “Tainted: An olfaction-enhanced game narrative for smelling virtual ghosts,” International Journal of Human-Computer Studies, vol.125, pp.p7-18, 2019.

[10] T. Nakamoto, S. Otaguro, M. Kinoshita, M. Nagahama, K. Ohinishi, and T. Ishida, “Cooking up an interactive olfactory game display,” IEEE Comput. Graph. Appl., vol.28, no.1, pp.75-78, 2008.

[11] T. Nakamoto, T. Hirasawa, and Y. Hanyu, “Virtual environment with smell using wearable olfactory display and computational fluid dynamics simulation,” 2020 IEEE Conference on Virtual Reality and 3D User Interfaces, pp.713-720, 2020. doi: 10.1109/VR46266.2020.00094

[12] T. Nakamoto, M. Ohno, and Y. Nihei, “Odor approximation using mass spectrometry,” IEEE Sensors J., vol.12, no.11, pp.3225-3231, 2012.

[13] M. Iseki, D. Prasetyawan, Y. Yokoshiki, and T. Nakamoto, “A study of odor reproduction using multi-component olfactory display,” Electrical Engineering in Japan, vol.215, no.3, e23392, 2022.

[14] Y. Ariyakul and T. Nakamoto, “Fundamental study of olfactory display using extremely small liquid pump and SAW atomizer,” The Virtual Reality Society of Japan, vol.15, no.4, pp.589-594, 2010.

[15] T. Nakamoto, S. Ito, S. Kato, and G.P. Qi, “Multicomponent olfactory display using solenoid valves and SAW atomizer and its blending-capability evaluation,” IEEE Sensors J., vol.18, no.13, pp.5213-5219, 2018.

[16] T. Nakamoto, H. Takigawa, and T. Yamanaka, “Fundamental study of odor recorder using inkjet devices for low-volatile scents,” IEICE Trans. Electronics, vol.E87-C, no.12, pp.2081-2086, 2004.

[17] S. Sugimoto, D. Noguchi, Y. Bannnai, and K. Okada, “Ink jet olfactory display enabling instantaneous switches of scents,” Proc. 18th ACM International Conference on Multimedia, pp.301-310, 2010. doi: 10.1145/1873951.1873994

[18] Data sheet of ppbRAE 3000+, Honeywell, 2018.

Authors

Hiroya HACHIYAMA
  Tokyo Institute of Technology

Takamichi NAKAMOTO
  Tokyo Institute of Technology

Keyword

olfactory display,  virtual reality,  inkjet device,  electro-osmotic pump