ISSS608 2017-18 T3 Assign Akanksha Shrirang Yadav Insights

From Visual Analytics and Applications
Jump to navigation Jump to search

MC1 AkankshaYadav.jpg VAST Mini Challenge 1: Are The Beloved Pipits Disappearing?

Overview

Data Preparation & Approach

Discovering Trends - Exploratory Data Analysis

Is Kasios Singing A True Song?

The State Of Pipits

Conclusion

 


Analysis and Classification of Audio Files Through Visualizations

For analysis of question 2, two different approaches were implemented in order to evaluate the legitimacy of Kasios’ claim about the pipits.

  • Comparison of Oscillograms of the Audio Files
  • Comparison of Distribution of Plots Selected Audio Features with the Features of the Test Files

These methods were compared at the end to see if they produced similar or different results.

As the number of samples for each species is not sufficient enough, running a machine learning algorithm over such training data might yield poor results with low accuracy. Hence, in this analysis, the only focus was to do visual comparison of the plots obtained through both approaches.


Other approaches tried:

Spectrogram of all the files were plotted, but were not discernible enough for comparison of the audio files. Additionally, time measurement plots were also plotted to see if they can be useful in analysis. However, similar issue occurred while interpreting the plot. Hence, ultimately a simple oscillogram was chosen instead.


Conversion Of MP3 Audio Files to WAV

All the visualizations were built in R. As majority of the R packages can only read audio file in ‘.WAV’ format, all the audio files (All Birds & Test) were converted to ‘.WAV’ format using tuneR package.


Approach 1 - Oscillogram Comparison

  • The provided audio files of all the birds have quality scores associated with them wherein the value of this score can be A, B, C, D, E or No Score. However, no description is provided about the different quality scores. For the analysis, audio files without any score were not taken into consideration.
  • Hence, prior to beginning with the analysis, a set of 5 files were chosen with quality scores A, B, C, D & E for “Rose-crested Blue Pipit”. Out of these 5 files, the file with score A was found to be the clearest & without much noise. All other files were found to have varying degrees of noise in them.

The oscillogram plots of the chosen files are as shown below:

Score A File:

T3 Assign QualityA.png


Score B File:

T3 Assign QualityB.png


Score C File:

T3 Assign QualityC.png


Score D File:

T3 Assign QualityD.png


Score E File:

T3 Assign QualityE.png


Next, 19 training birds were chosen one from each species of the birds and their oscillograms were plotted. Similarly, oscillograms were plotted for the test files provided by Kasios. All the test plots were compared with the chosen 19 birds individually to obtain the best estimate

Now, let’s look at the oscillogram plots of 19 species of the birds:

Bent-beak Riffraff

T3 Assign Audio1.png


Blue-collared Zipper

T3 Assign Audio2.png


Bombadil

T3 Assign Audio3.png


Broad-winged Jojo

T3 Assign Audio4.png


Canadian Cootamum

T3 Assign Audio5.png


Carries Champagne Pipit

T3 Assign Audio6.png


Darkwing Sparrow

T3 Assign Audio7.png


Eastern Corn Skeet

T3 Assign Audio8.png


Green-tipped Scarlet Pipit

T3 Assign Audio9.png


Lesser Birchbeere

T3 Assign Audio10.png


Orange Pine Plover

T3 Assign Audio11.png


Ordinary Snape

T3 Assign Audio12.png


Pinkfinch

T3 Assign Audio13.png


Purple Tooting Tout

T3 Assign Audio14.png


Qax

T3 Assign Audio15.png


Queenscoat

T3 Assign Audio16.png


Rose-crested Blue Pipit

T3 Assign Audio17.png


Scrawny Jay

T3 Assign Audio18.png


Vermillion Trillian

T3 Assign Audio19.png


The oscillogram plots of the test audio files are shown below:

Test Sample 1

T3 Assign Test1.png


Test Sample 2

T3 Assign Test2.png


Test Sample 3

T3 Assign Test3.png


Test Sample 4

T3 Assign Test4.png


Test Sample 5

T3 Assign Test5.png


Test Sample 6

T3 Assign Test6.png


Test Sample 7

T3 Assign Test7.png


Test Sample 8

T3 Assign Test8.png


Test Sample 9

T3 Assign Test9.png


Test Sample 10

T3 Assign Test10.png


Test Sample 11

T3 Assign Test11.png


Test Sample 12

T3 Assign Test12.png


Test Sample 13

T3 Assign Test13.png


Test Sample 14

T3 Assign Test14.png


Test Sample 15

T3 Assign Test15.png


Comparing the test plots obtained with train plots:

Examples are shown for a couple of the plots:


Estimated Outcome:


Approach 2 – Comparison of the Distribution Plots

For this analysis, various features of the audio files were retrieved using ‘analyzeFolder’ function found in ‘soundgen’ package in R. This function has the capability of processing batch files as well as also outputs the result in a summary dataframe which contains all the extracted features for all the files.

70 such features were obtained from the audio files. To reduce the number of variables, correlation plot was obtained to discard highly correlated variables. The variables were then clustered in hierarchical clusters using ‘Variable Clustering’ method in SAS JMP. Finally, the following 6 features were selected for further analysis:

<<6 variables>>

Next, the distributions for these variables for all the 19 species were plotted.

Similar features were obtained for the test audio files.

The 6 features of the test files were individually compared with the 6 features of 19 species using density plot in R.

The resultant plots are displayed here for a sample bird for illustration -> Test Sample 3

Bent-beak Riffraff
T3 Assign Species1 attributes.png
 Blue-collared Zipper
T3 Assign Species2 attributes.png
Bombadil
T3 Assign Species3 attributes.png
 Broad-winged Jojo
T3 Assign Species4 attributes.png
Canadian Cootamum
T3 Assign Species5 attributes.png
 Carries Champagne Pipit
T3 Assign Species6 attributes.png
Darkwing Sparrow
T3 Assign Species7 attributes.png
 Eastern Corn Skeet
T3 Assign Species8 attributes.png
Green-tipped Scarlet Pipit
T3 Assign Species9 attributes.png
 Lesser Birchbeere
T3 Assign Species10 attributes.png
Orange Pine Plover
T3 Assign Species11 attributes.png
 Ordinary Snape
T3 Assign Species12 attributes.png
Pinkfinch
T3 Assign Species13 attributes.png
 Purple Tooting Tout
T3 Assign Species14 attributes.png
Qax
T3 Assign Species15 attributes.png
 Queenscoat
T3 Assign Species16 attributes.png
Rose-crested Blue Pipit
T3 Assign Species17 attributes.png
 Scrawny Jay
T3 Assign Species18 attributes.png
Vermillion Trillian
T3 Assign Species19 attributes.png


Upon visual examination and comparison of the plots, the estimated outcome is:


Note that, both the outcomes do not match as they are based on only visual inspection. Although, by using the second approach, an attempt is made to classify the birds based on the statistical distributions of the species, it is still uncertain to an extent.


Plot for the test birds using the results from distribution analysis:

Retrieved from ‘https://wiki.smu.edu.sg/1718t3isss608/index.php?title=ISSS608_2017-18_T3_Assign_Akanksha_Shrirang_Yadav_Insights&oldid=8178’