SEARCH WITHIN CONTENT
Citation Information : International Journal on Smart Sensing and Intelligent Systems. Volume 11, Issue 1, Pages 1-8, DOI: https://doi.org/10.21307/ijssis-2018-008
License : (BY-NC-ND 4.0)
Published Online: 17-September-2018
Digital cameras are photodetector arrays that respond to particular bands of the electromagnetic spectrum, and specifically those in the visible spectrum between 380 and 780 nm are the most commonly used. The main cameras function is store the image from an aesthetic point of view, when these are used in home applications. For this reason, several processes are used in image reconstruction to improve the visual appearance, even from the manufacturing process of sensor decreasing the possibility to obtain optical measurements with this cameras. However, digital cameras can also be used as optical measuring instruments knowing their features in depth. Therefore, in this paper is proposed a calibration method for monochrome digital camera using a monochromator and an automatic rotation system. In this way, the uniformity of the pixels response is verified, bad pixels are detected, and the sensitivity curve is obtained. The results show the advantages of the method to find regions of pixels turned off. Also, the value of sensitivity as a function of wavelength is obtained to verify the regions with significant changes of uniformity. This work aims to the prototyping of a color measurement system and it is very important a comprehensive understanding of the camera.
Digital cameras are developed using photosensors array. This array is built with individual sensors which can be charge-coupled device or complementary metal-oxide-semiconductor. The array size has been increased and the total sensor area has been reduced significantly in recent years in order to improve the resolution. Furthermore, with the objective of achieving a better image reproduction the color filter mosaic and processing techniques are developed. In general, the largest application segment for cameras is the reproduction of scenes from an aesthetic point of view, causing that their development is oriented to improve the appearance, but not the quality of the measures in the image. Technological development has allowed the access to image acquisition devices of good quality and reasonable cost, encouraging the development of systems based on image processing and analysis. It is no secret that image analysis has become an important tool in a wide range of areas, ranging from everyday situations to specific applications in the industry, medicine, biology, chemistry, among others. In this sense, digital cameras have facilitated the image recording and improved the quality of them.
In particular, the use of digital cameras for color measurement has proved to be useful in several applications. For example, some researches areas are food quality analysis (Popov-Raljić and Laličić-Petronijević, 2009; Amensour et al., 2010; Xiao-bo et al., 2010; Lee et al., 2011; Wu and Sun, 2013), UVI indication (Meng et al., 2016), determination of glucose in urine (Jia et al., 2015), determination of plant nutrients in soil (Moonrungsee et al., 2015), fish spoilage monitoring (Morsy et al., 2016), and other applications in analytical chemistry (Capitan-Vallvey et al., 2015; Masawat et al., 2015). However, color distortion is a common problem, even the same camera captures different colors of the same scene in different times. Although the scene also depends capture conditions of the image. Indeed, in a monochromatic scene the pixels within an image could vary the measured intensity. This happens because each camera light sensors (photosensors or photodetectors) produces different responses to the same stimulus, or because of defective regions in the array.
A strategy used for color calibration is to create a database with color images and their respective labels. New images are acquired and compared with the stored images in the database in order to determine its category (Masawat et al., 2015). Other proposals employ Color Checkers, color arrays, and color-encode fringe patterns, to establish the relation between the measured color and the real color, and generate a way to correct the measured color if necessary (Shrestha et al., 2011; Moonrungsee et al., 2015; Gong et al., 2016; Li and Duan, 2016). Usually, the goal is to obtain a reliable measure, nevertheless, these approaches require a lot of experimental work and the results may be biased because it is not feasible to consider all capture conditions (e.g. all possible light sources).
The objective of this work is to find the characteristic curve of a digital camera, find defective pixels, and evaluate the uniformity of the response starting from a broadband light source. The authors have developed similar works (Botero V. et al., 2016), but only to estimate the sensitivity curve in individual photodetectors. The generated light is introduced in a monochromator to ensure a specific wavelength. After this, the light is divided to feed a power meter and the photodetector surface simultaneously. The wavelength of the monochromator is varied in the range of visible light (380–780 nm) using 20 nm increments to generate curves of 21 points for each photodetector.
EQ-99 is a laser-driven light source UV–VIS–NIR manufactured by ENER-GETIQ. This broadband source is specially designed for high brightness and high stability. The spectral output ranges from 170 to 2,100 nm, with a numerical aperture up to 0.47 and a typical bulb life longer than 5,000 hr. For the method presented in this paper it is crucial that the light source shows a flat spectral response, in such a way that the power is uniformly distributed in all the components of the visible spectrum, to ensure the presence of all the monochrome bands in the spectrum.
It is very important to use in the process a source capable of producing a continuous spectrum in the VIS with a output power close to uniform.
Mini-Chrom is a monochromator that uses a dial to select the output wavelength. A screw bar mechanism accurately guides the rotation of a diffraction grating, which positions the selected wavelength in the output. The wavelength is read directly in nanometers (nm) by a four-digit counter in all models. The operating range is 200 to 800 nm. SMA connectors will be adapted at the input and output for connecting plastic optical fibers, it is also possible to adapt a slit to the output. In the method presented in this work the output is open, and the camera surface is illuminated with the light coming out of the slit.
AQ6373 is a Spectrum Analyzer that provides an accurate high-speed analysis for a wavelength range between 350 and 1,200 nm. This optical spectrum analyzer (OSA) is well suited for general purposes. It also allows USB storage to save data in .csv format for further analysis. The resolution can be varied from 0.02 to 10 nm, or 0.01 nm for 400 to 470 nm range. Additionally, the AQ6373 has an optical alignment source.
FL3-U3-13Y3M-C (Fig. 1) is a monochromatic camera developed by FLIR with excellent performance. One of the main advantages of this type of cameras compared to those with domestic use is the availability of data in raw format. This allows one to directly obtain the response of the sensors without processing, and this is a necessary characteristic to use the camera as a measuring element, and all control parameters are known and operated manually.
In Figure 2 the explanatory diagram of the proposed experiment is presented and its parts are described below:
The initial response value to obtain the saturation parameters is adjusted with light emitted at 525 nm.
The system input is a white light source EQ-99 (polychromatic) with UV–VIS–NIR emission (1) coupled to the monochromator (2) via optical fiber.
The white light enters to the mechanical UV–VIS–NIR monochromator (2) (Mini-Chrom) that selects the desired wavelength. The adjustment of the wavelength is carried out manually with a mechanical element in increments of 20 nm in the visible spectrum (380–780 nm). The monochromator output light is a 300 μm slit.
The output of the monochromator illuminates an area. The camera (3) is positioned in such a way that the surface of the sensor is completely covered by monochrome light. The power of the light is also measured (4) using the spectrometer in the same plane where the sensor is located. The measure is taken in one point using two optical fibers. One of these is connected to a fiber optical power meter and the other one to the OSA.
The camera (3) is mechanically coupled to a digital servomotor that moves it automatically on five different angles for each wavelength. In order to determine the efficiency of the sensor, the power is taken in the measurement plane at 525 nm, because at this wavelength, the maximum sensitivity of the camera is determined. The camera is moved near or far as appropriate to match the power at 525 nm for each wavelength. This task must be performed due to the uncertainty in the elements used to process the light.
The values obtained in the previous steps will allow to know the photodetector efficiency on certain wavelength, the uniformity in the pixels response, and detect the defective pixels in the array.
The compressed image storage formats make undetectable the defects on the sensors, because the processing operations repair or camouflage the pixels with information from the region. For this reason, the images used in this experiment are acquired in raw format due to the camera is used as a measurement element.
The method proposed in this work provides information on the operation of the sensor. The quality of the parameters must be adjusted for each experiment where the camera is used as a measurement element. For example, the number of defective pixels tolerated is a dependent value in the application. The images obtained using the experiment described in the previous subsection can be calibrated with the three steps process described below:
First, the mean and standard deviation are calculated for each of 105 images acquired in the experiment . The average value of the images reflects the power absorbed by the photodetectors array; however, the values in defective pixels are deviated considerably of the average.
In this step, pixels with two deviations above or below of the mean are selected as defective pixels. Defective pixels must be aligned between images with different angles to decrease the error in the experiment. These pixels do not provide information to the image, their response does not change to different stimulus, or it is substantially different to the rest of pixels.
It is desirable to know the uniformity of the pixel response, that is, how similar is the response of the pixels if they are stimulated with the same power. Several imperfections in the camera are impossible to eliminate of the experiment, which affects the uniformity of the power that is arriving on the sensor.
The rotation is again used as a tool to calculate the uniformity. It is possible to know an approximated response of a pixel by calculating the average of five rotated images for each wavelength. In this manner, each pixel is exposed to five different points of the projection plane of the monochromator. The result is a averaged image which is used to calculate the range of sensor response for each wavelength. The source which the pixels are exposed comes from monochromator and it is assumed uniform.
The sensitivity curve of the camera is obtained. For this, the axial displacement of the system was used to ensure that the detector surface would always be exposed to the same power (525 nm). In view of this, the mean value of each of the 105 images was used to delete the defective pixels. The values are represented in a Cartesian system where the x-axis is the wavelength and the y-axis is the mean value of each image. With these points, a curve that represents the experimental dispersion is calculated, and the sensitivity curve of the camera is obtained. A interpolated curve allows one to estimate the sensor response at any wavelength.
In order to determine defects on the sensor surface, the mean and standard deviation of the response values for each image are calculated. Then, the pixels with two deviations above or below of the mean are selected. The results are shown in Figure 3, where P1, P2, and P3 are rotation positions, the images are shown for two wavelengths 460 and 740 nm. The white regions in the figure highlight the defective pixels, 227 defective pixels were detected. The purpose of camera rotation is that defective pixels appear in the same region facilitating their localization, and other regions produced by other different phenomena are discarded. The set of highlighted pixels must be excluded when the sensor is used as a measurement element. It is important to note that the images have artificial color to facilitate their interpretation.
Figure 4 shows the raw response of the photodetector for different rotation positions (P1 and P5) and different wavelengths (480 and 640). These rotation positions were located equidistantly over 300 grades. The number of rotations positions can be increased up to the encoder resolution of the servomotor. The 10 bits encoder allows 1024 rotation positions between 30 and 330 grades, i.e., it is possible to acquire images each 0.29 grades, but this increase the time of recording.
In total the five images of each wavelength are averaged to calculate the resulting image Pm. It can be seen that uniformity improves. On this image, the response range for each wavelength is estimated.
Measurements of spectra were taken every 20 nm in the range between 380 and 780 nm having 21 spectra in total. The uncertainty due to the orthogonality of the sensor plane when is illuminated, and nonuniformity of light of the monochromator output is reduced repeating the measurement five times in different angles. As shown in Figure 5, the spectra are narrow allowing improve the estimation of the generated response model. In addition, as mentioned in the description of the experiment the camera is moved near or far of the monochromator as appropriate to match with the power at 525 nm for each wavelength with the help of the spectrometer.
The mean of each image in each rotation position is obtained in this step and the mean for each evaluated wavelength of each rotated images is represented by the red dots, see Figure 6.
Eq. (1) presents the estimated sensitivity function of the camera (blue line in Fig. 6), where the parameter k must be adjusted with the power. k is a scale factor so that pixels respond proportionally, for this reason, this must be adjusted whenever the power changes to maintain the desired proportional response.
Finally, in Table 1, the average value of each image for each position (P1, P2, P3, P4, P5) and each of the 21 sampled wavelengths is presented. The value is normalized from 0 to 100%, the last column shows the response range of the camera (margin).
The proposed method shows the detection of defective pixels (find regions of pixels off) using the experimental design, estimates the sensitivity values as a function of the wavelength, and verify the uniformity of the pixels response in a photodetector array.
The results show that it is necessary to develop the process described in this work because it is common for sensor arrays to be found defective pixels and as it is known their sensitivity is far from uniform, the sensitivity curve is fundamental if wants to use the camera as a measuring element.
As mentioned, the aim of this work is to acquire information that allows the camera to be used as a measurement element. With the information obtained, it is expected that the camera can be used in color measurement systems, particularly when hyperspectral sources are used and the camera is used to measure reflectance.
This work is part of the research project “Design and implementation of an intelligent control system with balanced natural light to reduce energy consumption in buildings” with ID P12204, of the Automática, Electrónica y Ciencias Computacionales Group COL0053581. Instituto Tecnológico Metropolitano, Medellín-Colombia.