1. INTRODUCTION

The transition from juvenile freshwater fish (parr) to seawater adapted fish (smolt) is an important event in the life cycle of Atlantic salmon (Salmo salar). This process is commonly referred to as smoltification and involves changes in physiology, behavior, and morphology [1]. In salmon farming, this transition is controlled using, e.g., lights or functional feed [2,3]. Smoltification control is important for smolt producers to meet deliveries and to ensure a continuous and predictable supply of fish to sea farms. A vital aspect in maintaining smoltification control is the ability to verify the completion of this transition, since incomplete seawater adaptation means the fish cannot osmoregulate in seawater [4]. Incomplete smoltification may therefore result in poor animal welfare and increased mortality. Animal welfare is of increasing importance in Atlantic salmon farming, as the industry is under pressure to improve production and farming operations due to ethical and consumer concerns, and improving smoltification control may be part of the solution [5,6].

Smolt are produced in either water flow-through or recirculation sites. Conventional smoltification assessments involve measurement of smoltification indicators such as chloride content in blood samples after exposing fish to saline water, or the presence of ion-transporting enzymes through analysis of tissue samples from gills [7]. Such tests are performed in combination with an evaluation of fish morphology and skin features such as color, texture, and light reflective properties. Challenges associated with these methods include that only few individuals are tested (typically 10–20) and that these may not be a representative sample for a population of several hundreds of thousands of fish. Some conventional test methods are also time-consuming. For example, chloride determination in blood samples requires that the fish be exposed to saline water for up to 72 h before euthanization, blood sampling, and -analysis [8]. Supplementary evaluations of, e.g., skin color, texture, and reflective properties are less time-consuming but manual, and therefore depend on the subjective assessment skills, knowledge, and experience of the evaluator. Additional approaches facilitating automated and objective evaluation of the smoltification status in smolt production are therefore highly desired and requested by the industry.

Hyperspectral imaging is an optical, remote-sensing technique that records the light intensity at different wavelengths (i.e., typically ${\gt}100$ color bands) reflected from an object using a hyperspectral imager (HSI). The recorded data are commonly referred to as the object’s spectral characteristic and are largely determined by the object’s surface material composition [9]. This sensing technique is employed in several other areas, including determination of the composition of celestial bodies [10], managing terrestrial crops [11], mapping kelp forests [12], sorting fish viscera [13], and quality assessment of salmon fillets [14]. In recent years, advances in hardware and signal processing have enabled analysis of biological samples [15,16], as well as design of handheld versions of multispectral imagers [17]. Such advances are interesting within the context of smolt production because simple and reliable field-applicable solutions that can be used by production site personnel should be developed.

Because smoltification causes changes in both color and reflective properties of the fish, it is hypothesized that data collected from fish using an HSI could be used to characterize the two biological stages, thereby laying the foundation for automated, real-time verification of smoltification. Furthermore, it is possible that such data sets contain a small subset of wavelengths that can be used in future, cost-effective technologies for scanning fish, thereby supplementing, and possibly replacing, current methods. Such potential future solutions can then be developed and integrated as part of existing and future production sites, enabling farmers to continuously monitor the smoltification process for a larger portion of the fish population. Moreover, unlike conventional methods, HSI-based testing would not require handling or premature exposure to saline water (i.e., during chloride testing), thus potentially reducing the welfare challenges and stress suffered by the fish during testing. It would also eliminate the need to euthanize fish as part of the testing regime.

The objective of this study was therefore to investigate whether smoltification in farmed salmon can be detected using hyperspectral imaging, and to determine whether the spectral characteristics for parr and smolt contain wavelengths uniquely distinguishing the two stages.

2. MATERIALS AND METHODS

A. Ethical Statement

Since the purpose of the study was to map the connection between physiological changes and spectral characteristics of live fish, replacing the fish with alternative models was not feasible. The number of fish used in this study was kept to a minimum by using animals sampled as part of the normal smoltification testing regime at the different production sites. The study was authorized by the Norwegian Food Safety Authority (permit number 18/27599).

B. Study Design and Timeline

To evaluate the robustness and representativeness of a hyperspectral-based approach, emphasis was placed on collecting diverse data with expected variations in, e.g., fish color, patterning, size, and shape from different production sites. The study was conducted in cooperation with three different production sites where data were collected weekly in synchronization with the sites’ respective production and testing schedules. All three sites produced fish of the Aquagen genetic strain. Site 1 was a water flow-through production site operated by Mowi ASA. Site 2 was a water recirculation production site operated by Lerøy Midt AS. Site 3 was a water flow-through production site operated by Måsøval Fiskeoppdrett AS. A summary of the three sites’ conditions and smoltification control strategy as well as the data collection period and the number of fish used (N) is shown in Table 1.

Table 1. Summary of Conditions and Principal Data for the Three Smolt Production Sites^a

View Table | View all tables in this article

C. Sensing Equipment

The sensing equipment consisted of an FX10, sCMOS-V10E push-broom HSI with 400–1000 nm bandwidth, 2.9 nm spectral resolution, 30 µm slot, a maximum frame rate of 100 fps and equipped with a 23 mm/f.2.4 (OLE23) lens (SPECIM, Elektroniikkatie 13, Oulu Finland). The imager was mounted to a frame with a moving tray (SPECIM LabScanner, ${20} \times 40\;{\rm cm}$, 0.1–99 mm/s) including a motor control unit synchronizing the imager to the moving tray (Fig. 1.) to avoid over- or undersampling. The table’s step length corresponded to the camera’s spatial resolution in the scanning direction, while the scanning speed was automatically determined based on the exposure settings.

 

Fig. 1. Hyperspectral camera and moving tray.

Download Full Size | PDF

Because data were collected in the field over long periods of time, exposure settings had to be adjusted depending on local conditions and the state of the fish. The effect of the former was minimized during data collection by mounting the equipment in rooms without windows and turning of all lights where possible, and/or by covering windows with opaque fabric where necessary. Because the parr–smolt transition involves the fish becoming increasingly more reflective (i.e., mirror-like), the latter was addressed by adjusting the shutter speed as required to keep the exposure within the sensor’s dynamic range to the largest extent possible. To make all data sets comparable despite differences in ambient lighting conditions and exposure settings, all data sets were normalized for comparison using white and dark reference images recorded for each data set in combination with the method described below.

For Site 1, the standard halogen lights supplied by SPECIM as part of the LabScanner system were used to illuminate specimens during scanning. To reduce the risk of heat radiating from the halogen lights affecting the results, the lights were changed to a broad-spectrum, programmable LED array (CREE Lumia 5.2) before starting sampling at Sites 2 and 3. Spectra for both light sources are shown in Fig. 2.

 

Fig. 2. Spectra for halogen and LED light sources.

Download Full Size | PDF

In addition, crossed polarizers were used for Sites 2 and 3 to reduce specular reflections. The impact of these changes is evaluated in the discussion. For data collection, a Shuttle SH110G barebone computer with Intel i7 processor, 16 GB D4 2400 memory and solid-state SATA HD for data storage was used. A Karbon CL2 frame grabber from BitFlow (up to 128 bits input at 85 MHz) was installed in the PC to grab frames from the CCD camera stream.

D. Experimental Procedures

For each site, fish were randomly collected from the same tank per sampling event throughout the data collection period (Table 1). A sampling event involved collecting a number of fish and recording (hyperspectral) data from all these fish on the same day. For Site 1, 20 fish were collected per sampling event. For Sites 2 and 3, 10 fish were collected per sampling event. For all sites, a knotless dip net was used and the fish immediately transferred to a transportation bucket containing water from the same tank and transported a short distance (${\lt}{50}\;{\rm m}$ for all sites) to the HSI.

Individual fish were subsequently transferred from the transportation bucket to a new bucket containing a knock-out anesthetic solution (80–100 mg/L Tricaine mesylate). When the fish was judged to have reached level III anesthesia [18], it was carefully placed on the moving table below the HSI and scanned. After scanning, the fish was passed on to the production site’s personnel for blood or gill tissue sampling.

Chloride testing involved exposing fish to saline water (35 ppt for 72 hr in Site 1 and 32 ppt for 48 h in Site 2) prior to each sampling event. During each sampling event, blood samples were collected from the fish exposed to saline water (test group) as well as from a reference group containing an equal number of fish that were not exposed to saline water (reference group). The measured chloride values were used for classification, as described in the Data Processing and Analysis section. Gill tissue sampling involved dissecting a piece of gill tissue for analysis by a third party. The analysis quantified the prevalence of freshwater and seawater ATPase [19], i.e., the presence of chloride-excreting cells in the tissue sample. The difference between these two was used to classify the parr and smolt stages, as described in the Data Processing and Analysis section. Following testing, the fish were euthanized.

E. Data Processing and Analysis

The raw data obtained from the HSI were multidimensional images of individual fish, including their background. Each layer of this multidimensional image represented a single gray-scale image corresponding to the intensity of the reflectance measurement at a specific wavelength. When stacked, all the layers and reflectance measurements represented a 3D cube (hyperspectral cube) with two spatial dimensions ($ x $ and $ y $) and one dimension for the wavelength ($ z $). A representative example of a hypercube is given in Fig. 3.

 

Fig. 3. Illustration of the hyperspectral measurements as a 3D image cube, i.e., a hyperspectral cube. The cube to the right shows a cross section through the fish lateral axis.

Download Full Size | PDF

These hyperspectral cubes were stored in the ENVI image format (a flat-binary raster file with an accompanying ASCII header file). OpenCV, Scientific Python, and C$++$ were used for processing and analysis.

A step-wise procedure was used to process and analyze the data so the low-dimensional spectral characteristics allowing observation of the smoltification process could be determined, and classification of parr and smolt made possible.

Step 1 involved labeling each salmon and its corresponding data set as either “parr” or “smolt” based on the traditional smolt testing results to obtain a baseline for classification. For fish having been subjected to chloride testing, the difference between the chloride value from individual fish in a test group and the average chloride value from the corresponding reference group was used. If the difference was lower than the industry standard of 20 mMol/L, the fish was deemed able to osmoregulate (i.e., excrete ions from its body) in seawater and thus, was labeled smolt. Conversely, if the difference was above 20 mMol/L, the fish was deemed unable to osmoregulate in seawater and thus, was labeled parr. For fish subjected to gill tissue sampling, the third-party analysis provided numbers for freshwater and seawater ATPase components, respectively. Fish with a larger freshwater component were labeled parr, while fish with a larger seawater component were labeled smolt.

Step 2 consisted of black-and-white calibration/compensation to remove the impacts of systematic variations due to, e.g., lighting differences. The reflectance measurements were first normalized using a technique described by Akbari and Kosugi [20]. Every time a hyperspectral image of a fish was recorded, two additional images were acquired: the white and dark references. The white reference was an image of a white reference tile placed on the moving table below the HSI, while the dark reference was an image acquired by the system with the shutter closed (i.e., complete absence of light). The white reference represented the systematic offset in reflectance as a function of wavelength caused by the light source and ambient light. The dark reference represented the spectral noise, i.e., the noise contribution caused by the imager itself. Using these references, each pixel in the raw hyperspectral cube was normalized using Eq. (1),

Step 3 involved computing the mean spectra $\bar R(\lambda)$ (i.e., mean reflectance as a function of the wavelength $\lambda$) over an area, $\Omega$, containing only fish using Eq. (2),

 

Fig. 4. Example images of parr and smolt from the three sites. The fish including background plate is shown in the left column, and the corresponding computed binary mask (right) is used to separate the fish from the background.

Download Full Size | PDF

In Step 4, a machine-learning classifier able to search for a low-dimensional spectral characteristic to distinguish between the two fish classes (i.e., parr and smolt) was implemented. A dimension of three wavelengths was chosen to aid visualization and improve understanding of the observable changes during the parr–smolt transition. The low-dimensional spectral characteristic was derived by first selecting a set of three wavelengths $({{\lambda _1},{\lambda _2},{\lambda _3}})$ and then extracting the corresponding reflectance tuple $({\bar R({\lambda _1}),\bar R({{\lambda _2}}),\bar R({{\lambda _3}})})$ for each fish. The reflectance tuple then represented a 3D point describing the mean reflectance for the three wavelengths across fish pixels. A labeled point cloud could then be obtained by attributing the label of the fish (parr or smolt) to their corresponding 3D reflectance point. The resulting data set was then used to train a classifier and evaluate its performance in accurately predicting fish as parr or smolt. For this, a support vector machine classifier was used [25]. Using a step size of 10 nm, an iterative search for the best classification accuracy over the three wavelengths $({{\lambda _1},{\lambda _2},{\lambda _3}})$ within the range of 400 to 700 nm was performed. Using this approach, optimal wavelengths $({\lambda _1^*,\lambda _2^*,\lambda _3^*})$ resulting in the best classification accuracy were determined for all sites. Using the optimal wavelengths, color-coded plots were created to visualize the transition from parr to smolt, as shown in the Results section.

Table 2. Summarized Results from Individual Sites

View Table | View all tables in this article

In Step 5, the performance of the classifier was evaluated by removing one data point from each training data set and using this point for validation, thereby testing whether the machine-learning approach was able to classify the salmon correctly or not. By doing this “leave one out” cross validation for all fish, a metric for the expected performance of the approach was obtained. The criteria accuracy, sensitivity, and specificity were computed for the outcomes of the classification approach, i.e., the true-positive (TP), true-negative (TN), false-negative (FN), and false-positive (FP) numbers. Note that smolt was defined as the positive class and parr as the negative class. The accuracy, sensitivity, and specificity were computed as given in Eqs. (3), (4), and (5),

The sensitivity represented the probability that a smolt was correctly identified as a smolt. Similarly, the specificity represented the probability that a parr was correctly classified as a parr, while the accuracy represented the proportion of overall correctly classified results and was used to determine the wavelengths that led to the best performance.

3. RESULTS

A. Individual Sites

Table 2 summarizes the results from each individual site, where $ N_p $ and $ N_s $ denote the number of labeled parr and smolt used in the analysis, respectively. The tuple $({\lambda _1^*,\lambda _2^*,\lambda _3^*})$ denotes the three wavelengths giving optimal separation between the parr and smolt classes.

Figures 5, 7, and 9 illustrate typical spectra for parr and smolt for each site. Figures 6, 8, and 10 illustrate the time development of the parr–smolt transition by a 3D plot rotated to show the best possible separation of parr and smolt for each site.

 

Fig. 5. Typical spectra for parr and smolt, respectively, for Site 1. The mean spectra were derived from the 10 first (parr) and 10 last (smolt) in the data set.

Download Full Size | PDF

 

Fig. 6. 3D plot of data from Site 1 during 11 measurement dates (11 colors). Each point represents one fish, and the color represents the measurement date. Salmon classified by traditional methods as parr are represented by circles, while smolts are represented by triangles. Note that the reflectance values $\bar R(\lambda)$ are normalized to [0,1].

Download Full Size | PDF

 

Fig. 7. Typical spectra for parr and smolt, respectively, for Site 2. The mean spectra were derived from the 10 first (parr) and 10 last (smolt) in the data set.

Download Full Size | PDF

 

Fig. 8. 3D plot of data from Site 2 during eight measurement dates (eight colors). Each point represents one fish, and the color represents the measurement date. Salmon classified by traditional methods as parr are represented by circles, while smolts are represented by triangles. Note that the reflectance values $\bar R(\lambda)$ are normalized to [0,1].

Download Full Size | PDF

 

Fig. 9. Typical spectra for parr and smolt, respectively, for Site 3. The mean spectra were derived from the 10 first (parr) and 10 last (smolt) in the data set.

Download Full Size | PDF

 

Fig. 10. 3D plot for data from Site 3 during five measurement dates (five colors). Each point represents one fish, and the color represents the measurement date. Salmon classified by traditional methods as parr are represented by circles, while smolts are represented by triangles. Note that the reflectance values $\bar R(\lambda)$ are normalized to [0,1].

Download Full Size | PDF

B. Overall Assessment

Table 3 summarizes the results from an overall assessment, including all labeled fish from all sites. In Table 3, $ N_p $ and $ N_s $ denote the number of labeled parr and smolt used in the analysis, respectively. The tuple $({\lambda _1^*,\lambda _2^*,\lambda _3^*})$ denotes the three wavelengths giving optimal separation between the parr and smolt classes.

Table 3. Results from Data Combined for All Sites

View Table | View all tables in this article

Figure 11 illustrates the time development of the parr–smolt transition by a 3D plot rotated to show the best possible separation of parr and smolt.

 

Fig. 11. 3D plot of all data collected at the 3 sites containing 24 measurement dates (24 colors). Each point represents one fish, and the color represents the measurement date. Salmon classified by traditional methods as parr are represented by circles, while smolts are represented by triangles. Note that the reflectance values $\bar R(\lambda)$ are normalized to [0,1].

Download Full Size | PDF

4. DISCUSSION AND CONCLUSION

This study has shown that optical measurements have the potential to be used for identification of smoltification status of Atlantic salmon. It may thereby serve as an additional or new tool for smoltification assessment to improve animal welfare and reduce mortality in salmon production.

In this study, a total of 314 fish from 3 different production sites were measured and labeled based on common traditional smoltification evaluations. By sampling production sites using flow-through or water recirculation, the project aimed to capture a range of representative fish traits found in smolt production. Because data were collected in the field, several factors may have an impact on the results.

The clustering of the results from the three different sites implies that the classifier was able to distinguish between parr and smolt using the optimum wavelengths for each site (Figs. 6, 8, and 10). When all data are combined (Fig. 11), we see site-specific clustering likely caused by differences in, e.g., average water temperature and dissolved oxygen, water opacity, and color, as well as lighting and feeding regimes. Despite this, an overall classification accuracy of 90% was achieved. It is important to note that the clusters for parr and smolt seen in Fig. 11 are segregated, indicating that the two biological stages can be identified for the range in fish traits and measurements conditions in these data sets.

For Site 1, parr cluster well for the first six sampling events (Fig. 6). However, for the remaining sampling events, the distinction from smolt became less clear. A similar trend was observed for Site 2 (Fig. 8), where good separation is evident prior to 14 February 2019, while both classes are present in the same cluster following this date. Classification accuracy depends on the smoltification tests provided by the sites. For chloride testing, the industry standard for the relative chloride difference between the reference and test groups is 20 mMol/L. As such, chloride testing does not evaluate smoltification based on an absolute value for chloride content in blood. Consequently, it is possible that fish with a low chloride difference could have been mislabeled as parr, resulting in the mixed clusters. Interestingly, this is not the case for Site 3, where tissue sampling was employed (Fig. 10), thus indicating that tissue sampling may be a more accurate method for smoltification control, and that the industry standard of 20 mMol/L chloride difference may be conservative (i.e., some fish classified as parr are actually smolt).

The accuracy of the results may also have been affected by imperfect masking/background removal. Such imperfections were likely caused by differences in color and patterning of the fish in combination with differences in water spill on the moving tray between data sets. The setup also used a fixed focus distance and because a variation in condition factor [4] between fish (i.e., thickness in this case) must be expected, focus may not have been perfect in all data sets due to limited depth of field. These factors all contribute to different boundary conditions (Fig. 4), allowing a varying amount of background pixels to propagate into the analysis which, in turn, may have affected classification accuracy.

Also, due to the mirror-like, curving surface of the fish, specular reflections may have saturated pixels in different regions of different fish, thereby camouflaging areas that may contain information unique for parr and smolt, respectively. This effect was minimized by adjusting the angle of incidence for the light source in Site 1, and by adding crossed polarizers to the setup for Sites 2 and 3. These polarizers were linear in the wavelength area of 400–700 nm, thereby only affecting intensity and not the spectral characteristic. Hence, data could be compared across sites despite the addition of crossed polarizers for Sites 2 and 3. When looking at the data, specular reflections occurred as a line along the center of the fish. This is an area usually containing repeating patterns on the fish; hence, the spectral information lost due to specular reflections was expected to be captured from neighboring areas without reflections. Thus, it is expected that the small amount of specular reflections has had minimal impact on the results.

The default light source for the HSI consisted of light fixtures using halogen bulbs. During data collection on Site 1, concern related to the stability of the light source in the IR area arose, as a rapid time-varying change in the measured spectral characteristic above 700 nm was observed. This instability may have originated from variations in the air flow and temperature around the halogen bulbs caused by the integrated lighting fixture cooling fans. The halogen bulbs also radiated heat onto the fish being measured. To eliminate the risk of heat affecting the surface of the fish, lights were changed to programmable LEDs, as described in the Materials and Methods section. To ensure that data could be compared between all sites, data for wavelengths ${\gt} 700 \;{\rm nm}$ were omitted from the analysis.

The number of different sites (3) and the number of sampled, and also labeled, fish (314) is relatively low. The number of sites and fish was constrained by time, resources, and accessible production sites, as well as their production cycles and test regimes. The site numbering corresponds to the site order in which data were collected. Due to an initial uncertainty related to how fast the parr–smolt transition occurs, data were collected for a longer period before the transition was expected to occur in Site 1 compared to Sites 2 and 3. Based on the results from Site 1 and dialogue with the producers, it was concluded that the parr–smolt transition was of limited duration, and that collecting data for a long time prior to the expected transition would not add crucial information to detect the transition because the number of fish measured around the time of transition was not so different. Therefore, the impact on the results is expected to be minimal. In addition, the reduction of the time span where fish were monitored helped to remain compliant with ethically responsible animal experimentation (3 R [26]), as it reduced the number of animals used in the experiment.

5. FURTHER WORK AND PROSPECTS

In further work, additional measurements, possibly at different times of the year, would be beneficial to get an insight into the consistency of the results over time. During future data collection, emphasis should be placed on using tissue sampling over chloride difference as a baseline for classification, as the former seems to provide the better accuracy of the two.

The results obtained by using the average spectra of the whole fish has the potential to be improved by optimizing the region used for analysis. In addition, alternative techniques for dimension reduction of the full hyperspectral data to a few characteristic wavelengths should be investigated. Furthermore, it could be considered to investigate which external factors (i.e., environmental conditions, feed composition, or others) have an impact on salmon in general and specifically during the parr–smolt transition using an HSI in more controlled laboratory conditions.

Although the results from this study have the potential to increase the ability of the industry to better monitor and detect smoltification, the issue of desmoltification remains. Desmoltification may occur if a smoltified fish is kept too long in fresh water after the metamorphosis and thus loses its osmoregulatory capacity. This reversal is not accompanied by changes in morphology and camouflage and is therefore more difficult to assess using conventional methods. Investigating whether an his-based approach could be employed to determine if desmoltification has occurred should therefore be subject to further work. Future work should also consider a search for spectral characteristics that are specific for certain body parts of the salmon and may be even more discriminative than the low-dimensional characteristic considered in this work.

Finally, we have shown that three wavelengths are sufficient to identify parr and smolt. This enables development of low-cost instruments that can potentially be used in production tanks or integrated in existing sorting and vaccination systems to support smoltification control in the future.