Difference between revisions of "ISSS608 2017-18 T3 Chan En Ying Grace Which song belongs to thee"
Line 45: | Line 45: | ||
Using the training dataset (2081 birds), we will pick 5 birds per species and then obtain a representative one by choosing the pattern that is most common for that species. We will do this in the following 3 steps. | Using the training dataset (2081 birds), we will pick 5 birds per species and then obtain a representative one by choosing the pattern that is most common for that species. We will do this in the following 3 steps. | ||
− | ====Approach: Envelope Plot, Oscillogram Plot | + | ====Approach: Envelope Plot, Oscillogram Plot, Trellis Plot & Audio Classification==== |
1. First, we will plot the envelope of each sound wave of the bird species, for a quick visualization across each of the 19 bird species. | 1. First, we will plot the envelope of each sound wave of the bird species, for a quick visualization across each of the 19 bird species. | ||
− | <br> 2. Second, we plot the | + | <br> 2. Second, we plot the oscillogram outlined by the envelope to view the amplitude pattern in greater detail, for each of the 19 species. |
<br> 3. Thirdly, we will use ggplot to obtain a trellis plot of the distribution of each parameter of the audio file (e.g. amplitude, HNR, entropy, spectrogram slope, pitch frequency) for each species to obtain the mean by parameter. | <br> 3. Thirdly, we will use ggplot to obtain a trellis plot of the distribution of each parameter of the audio file (e.g. amplitude, HNR, entropy, spectrogram slope, pitch frequency) for each species to obtain the mean by parameter. | ||
+ | <br> 4. Lastly, we will attempt audio classification using Random Forest and Decision Tree algorithms to predict the bird species. | ||
− | With | + | With the first 3 complementary techniques, we will then plot the same for the 15 testing birds against the training data. Thereafter, we will compare the three visualisations against the 19 species of the training data. This will help us identify the bird species of the 15 testing birds of Kasios. |
+ | |||
+ | Note: We also attempted the 4th technique to ascertain whether our visualisations were accurate, though our findings later show that the classification model produced 50% misclassification rate and hence needs to be better refined. | ||
<hr/> | <hr/> |
Revision as of 23:10, 23 June 2018
|
|
|
|
|
Contents
- 1 Audio Visualisation: "Which song belongs to thee, Rose Pipit?"
- 2 Concluding Hypothesis: Pipits NOT being found across the Preserve, based on Kasios Birds
Audio Visualisation: "Which song belongs to thee, Rose Pipit?"
1. Location Matters
Firstly, it is important to note that the 15 testing data are nowhere near the alleged dumping site, except for the four birds. It is unclear whether these 4 birds belong to Rose Pipits, as they could also belong to the Ordinary Snapes or Lesse Birchbeere clusters, which was earlier proven that their home range also exist near the dumping site.
2. Data Preparation
Next, we convert all the MP3 files into .wav format. Wav format is preferred for many of the R packages that deal with audio files. We’ll select and work with the left channel of each file, then convert our sound array to floating point values ranging from -1 to 1.
Now, we are ready to plot the oscillogram to visualize the relationship between Amplitude and Time (seconds), for each bird species. We will also outline the envelope in a different colour, for easier visualization.
Using the training dataset (2081 birds), we will pick 5 birds per species and then obtain a representative one by choosing the pattern that is most common for that species. We will do this in the following 3 steps.
Approach: Envelope Plot, Oscillogram Plot, Trellis Plot & Audio Classification
1. First, we will plot the envelope of each sound wave of the bird species, for a quick visualization across each of the 19 bird species.
2. Second, we plot the oscillogram outlined by the envelope to view the amplitude pattern in greater detail, for each of the 19 species.
3. Thirdly, we will use ggplot to obtain a trellis plot of the distribution of each parameter of the audio file (e.g. amplitude, HNR, entropy, spectrogram slope, pitch frequency) for each species to obtain the mean by parameter.
4. Lastly, we will attempt audio classification using Random Forest and Decision Tree algorithms to predict the bird species.
With the first 3 complementary techniques, we will then plot the same for the 15 testing birds against the training data. Thereafter, we will compare the three visualisations against the 19 species of the training data. This will help us identify the bird species of the 15 testing birds of Kasios.
Note: We also attempted the 4th technique to ascertain whether our visualisations were accurate, though our findings later show that the classification model produced 50% misclassification rate and hence needs to be better refined.
3. Envelope Plot
Amplitude envelope refers to the changes in the amplitude of a sound over time. It is an important property of sound, because it is what allows us to effortlessly identify sounds, and uniquely distinguish them from other sounds. We will thus use envelope plot to distinguish bird calls/songs.
Training Birds (19 Species)
First, we will plot the envelope of each sound wave of the bird species, for a quick visualization across each of the 19 bird species.
Testing Birds (15 Birds Identified By Kasios)
Next, we will plot the envelope of each sound wave for the 15 test birds.
Predicted Results, Based on Visualisation
By visualizing the envelope of the amplitude waves of both the training and testing data, the last column shows the predicted species for each of the 15 test birds.
4. Oscillogram Comparison
For confirmation, let us also look at the oscillogram which gives us not only the envelope but also the waves. Oscillogram is the plot of relative amplitude vs. time in seconds (while a Spectrogram is the plot of frequency in kHz vs. time in seconds).
Training Birds
The oscillograms of each of the 19 species of the training birds are as follows:
No. |
Species Name |
Oscillogram |
1. |
Bent Beat Riffraff |
|
2. |
Blue Collared Zipper |
|
3. |
Bombadil |
|
4. |
Broad Winged Jojo |
|
5. |
Canadian Cootamum |
|
6. |
Carries Champagne Pipit |
|
7. |
Darkwing Sparrow |
|
8. |
Eastern Corn Skeet |
|
9. |
Green Tipped Scarlet Pipit |
|
10. |
Lesser Birchbeere |
|
11. |
Orange Pine Plover |
|
12. |
Ordinary Snape |
|
13. |
Pinkfinch |
|
14. |
Purple Tooting Tout |
|
15. |
Qax |
|
16. |
Queenscoat |
|
17. |
Rose-Crested Blue Pipit |
|
18. |
Scrawny Jay |
|
19. |
Vermillion Trillian |
Testing Birds
The oscillograms of each of the 15 test birds are as follows.
The predicted species is indicated in the last column, after visualising and comparing the similarity of the amplitude plots. Our results show that the predicted species based on oscillogram visualisation, matches the predicted species based on envelope plot visualisation. This is not a surprise because the envelope is obtained from the oscillogram.
We plot both, because the envelope gives a quick comparison while the oscillogram provides a more indepth visualisation.
ID |
Oscillogram |
Predicted Species |
Same as earlier predicted by envelope? |
1 |
Eastern Corn Skeet |
Yes. Though, this is quite close to the Rose-Crested Pipit. However, the Pipit produces more ‘chirps’ per 100 sec, as compared to the Skeet. | |
2 |
Rose-Crested Pipit |
Yes. | |
3 |
Queenscoat |
Yes. | |
4 |
Bombadil |
Yes. | |
5 |
Canadian Cootamum |
Yes. | |
6 |
Qax |
Yes. | |
7 |
Canadian Cootamum |
Yes. | |
8 |
Green-Tipped Scarlet Pipit |
Yes. | |
9 |
Rose-Crested Blue Pipit |
Yes. | |
10 |
Qax |
Yes. | |
11 |
Scrawny Jay |
Yes. | |
12 |
Qax |
Yes. | |
13 |
Qax |
Yes.
| |
14 |
Bombadil |
Yes.
| |
15 |
Pinkfinch |
Yes. |
5. Trellis Plot (Distribution Analysis)
A caveat to the previous analysis is that we did not make use of all the training birds in the visualisation. Rather, we randomly selected 5 birds per species to visualise, and then chose 1 to represent the entire species. Thus, we now make use of all the training birds by plotting the distributions across the parameters.
Selected 7 Parameters
To make the analysis more rigorous, we will use analyzeFolder() to obtain the individual values of each parameters of the audiofile. AnalyseFolder() provides an acoustic analysis of all .wav files in a folder.
There is a total of 15 parameters, out of which, 7 are chosen as these 7 parameters have greater distinction between the species. The 7 parameters are: dom_median, HNR_median,mean, Freq_median, peakFreq_median, pitch_median, pitchAutocor_median, pitchSpec_median.
Distributions
The trellis plot of the 7 parameters of the training birds is shown below, where the mean is indicated by the black solid line.
Next, we will plot each of the 15 testing birds from Kasios onto this plot, in blue dotted line. We will then select the closest species for each parameter. The species with the most parameters selected will be assigned as the predicted species.
Given that Test Bird 2 and Test Bird 9 were predicted to be Rose-Crested Blue Pipits, we will focus on these two birds for visualisation.
Test Bird 2 Against Training Birds
The following shows the trellis plot of Test Bird 2 (in blue dotted line) against the 2081 Training Birds (black solid line).
The ticks in green represent the top 3 (or more if there is a tie) closest training species to the testing species, by parameter. The species with the highest ticks (i.e. closest to the testing bird) will be selected as the predicted species.
Based on this, Test Bird 2 is predicted to be a Qax.
Test Bird 9 Against Training Birds
The following shows the trellis plot of Test Bird 9 (in blue dotted line) against the 2081 Training Birds (black solid line).
The ticks in green represent the top 3 (or more if there is a tie) closest training species to the testing species, by parameter. The species with the highest ticks (i.e. closest to the testing bird) will be selected as the predicted species.
Based on this, Test Bird 9 seems like a Vermillion Trillon.
Results
From the above plots, based on the most number of parameters closest to the species’ mean, Test Bird 2 seems like a Qax while Test Bird 9 seems like a Vermillion Trillon. Unfortunately, this does not match our earlier predictions by visualizing the amplitude plot. We conclude that this method may not be ideal as it is a numerical representation, while the amplitude plots are more likely to be more reflective (though less representative of the entire training population).
As such, we will rely on Method 1 (Envelope Plot) & Method 2 (Oscillogram Plot), and leave Method 3 (Trellis Plot) out for our concluding hypothesis. The results are as follows, where Test Bird 2 and 9 are predicted to be Rose Pipits.
6. Coordinate Plot
Let’s analyse the coordinates to understand whether these 2 predicted birds are found near the Rose Pipit clusters. In fact, these two birds (represented in green below) are not found in the two clusters near the dumping site (represented in blue). But they did appear together, which makes sense since birds of the same species tend to fly together.
Concluding Hypothesis: Pipits NOT being found across the Preserve, based on Kasios Birds
Key Observations
1. Only 2 out of the 15 birds have resemblance to the Rose Pipits.
2. These 2 birds were not found near the dumping site, neither were they found in the previous 2 clusters identified.
Hypothesis: Pipits not found across preserve
Given that only 2 of the 15 birds provided by Kasios were likely to be Pipits, Kasios' claim that the Pipits were thriving across the Preserve is doubted. Based on the set of bird calls supplied by Kasios, it does not support the claim of Pipits being found across the Preserve.