Difference between revisions of "ISSS608 2017-18 T3 Assign Akanksha Shrirang Yadav Insights"

From Visual Analytics and Applications
Jump to navigation Jump to search
Line 33: Line 33:
 
<br/>
 
<br/>
  
 +
<div style=background:#2B3856 border:#A3BFB1>
 +
<font size = 3; color="#FFFFFF">Analysis and Classification of Audio Files Through Visualizations</font>   
 +
</div>
  
−
Analysis and Classification of Audio Files Through Visualizations
+
For analysis of question 2, two different approaches were implemented in order to evaluate the legitimacy of Kasios’ claim about the pipits.
  
−
For analysis of question 2, two different methods approaches implemented to in order to evaluate the legitimacy of Kasios’ claim about the pipits.
+
* Comparison of Oscillograms of the Audio Files
 +
* Comparison of Distribution of Plots Selected Audio Features with the Features of the Test Files
  
−
Comparison of Oscillograms of the Audio Files
+
These methods were compared at the end to see if they produced similar or different results.
−
Comparison of Distribution of Plots Selected Audio Features with the Features of the Test Files
 
−
These methods were compared at the end to see if they produced different results.
 
  
 
As the number of samples for each species is not sufficient enough, running a machine learning algorithm over such training data might yield poor results with low accuracy. Hence, in this analysis, the only focus was to do visual comparison of the plots obtained through both approaches.
 
As the number of samples for each species is not sufficient enough, running a machine learning algorithm over such training data might yield poor results with low accuracy. Hence, in this analysis, the only focus was to do visual comparison of the plots obtained through both approaches.
  
−
Other approaches tried:
 
  
−
Spectrogram of all the files were plotted, but were not discernible enough for comparison of the audio files. Hence, ultimately a simple oscillogram was chosen instead.
+
'''Other approaches tried:'''
 +
 
 +
Spectrogram of all the files were plotted, but were not discernible enough for comparison of the audio files. Additionally, time measurement plots were also plotted to see if they can be useful in analysis. However, similar issue occurred while interpreting the plot. Hence, ultimately a simple oscillogram was chosen instead.
  
 
   
 
   
  
−
Conversion Of MP3 Audio Files to WAV
+
'''''Conversion Of MP3 Audio Files to WAV'''''
  
 
All the visualizations were built in R. As majority of the R packages can only read audio file in ‘.WAV’ format, all the audio files (All Birds & Test) were converted to ‘.WAV’ format using tuneR package.
 
All the visualizations were built in R. As majority of the R packages can only read audio file in ‘.WAV’ format, all the audio files (All Birds & Test) were converted to ‘.WAV’ format using tuneR package.
  
 
   
 
   
 +
<div style=background:#2B3856 border:#A3BFB1>
 +
<font size = 3; color="#FFFFFF">Approach 1 - Oscillogram Comparison</font>   
 +
</div>
  
−
Approach 1 - Oscillogram Comparison
+
* <u>The provided audio files of all the birds have quality scores associated with them wherein the value of this score can be A, B, C, D, E or No Score.</u> However, no description is provided about the different quality scores. For the analysis, audio files without any score were not taken into consideration.
  
−
The provided audio files of all the birds have quality scores associated with them wherein the value of this score can be A, B, C, D, E or No Score. However, no description is provided about the different quality scores. For the analysis, audio files without any score were not taken into consideration.
+
* Hence, prior to beginning with the analysis, '''<u>a set of 5 files were chosen with quality scores A, B, C, D & E for “Rose-crested Blue Pipit”.</u>''' Out of these 5 files, the file with score A was found to be the clearest & without much noise. All other files were found to have varying degrees of noise in them.
−
 
 
−
Hence, prior to beginning with the analysis, a set of 5 files were chosen with quality scores A, B, C, D & E for “Rose-crested Blue Pipit”. Out of these 5 files, the file with score A was found to be the clearest & without much noise. All other files were found to have varying degrees of noise in them.
 
  
 
The oscillogram plots of the chosen files are as shown below:
 
The oscillogram plots of the chosen files are as shown below:
  
−
+
<div>
 +
'''<big>Score A File:</big>'''                                     
 +
[[File:T3_Assign_QualityA.png|460px|center]]
 +
<br/>
 +
'''<big>Score B File:</big>'''
 +
[[File:T3_Assign_QualityB.png|460px|center]]
 +
<br/>
 +
'''<big>Score C File:</big>'''
 +
[[File:T3_Assign_QualityC.png|460px|center]]
 +
<br/>
 +
'''<big>Score D File:</big>'''
 +
[[File:T3_Assign_QualityD.png|460px|center]]
 +
<br/>
 +
'''<big>Score E File:</big>'''
 +
[[File:T3_Assign_QualityE.png|500px|center]]
 +
</div>
  
−
Next, 19 training birds were chosen one from each species of the birds and their oscillograms were plotted. Similarly, oscillograms were plotted for the test files provided by Kasios. All the test plots were compared with the chosen 19 birds individually to obtain the best estimate
+
<br/>
  
−
+
Next, <u>19 training birds were chosen one from each species of the birds and their oscillograms were plotted.</u> Similarly, oscillograms were plotted for the test files provided by Kasios. All the test plots were compared with the chosen 19 birds individually to obtain the best estimate
  
 
Now, let’s look at the oscillogram plots of 19 species of the birds:
 
Now, let’s look at the oscillogram plots of 19 species of the birds:
  
−
<<figure>> 19
+
<div>
 +
'''<big>Bent-beak Riffraff</big>'''                                     
 +
[[File:T3_Assign_Audio1.png|500px|center]]
 +
<br/>
 +
'''<big>Blue-collared Zipper</big>'''
 +
[[File:T3_Assign_Audio2.png|500px|center]]
 +
<br/>
 +
'''<big>Bombadil</big>'''
 +
[[File:T3_Assign_Audio3.png|500px|center]]
 +
<br/>
 +
'''<big>Broad-winged Jojo</big>'''
 +
[[File:T3_Assign_Audio4.png|500px|center]]
 +
<br/>
 +
'''<big>Canadian Cootamum</big>'''
 +
[[File:T3_Assign_Audio5.png|500px|center]]
 +
<br/>
 +
'''<big>Carries Champagne Pipit</big>'''                                     
 +
[[File:T3_Assign_Audio6.png|500px|center]]
 +
<br/>
 +
'''<big>Darkwing Sparrow</big>'''
 +
[[File:T3_Assign_Audio7.png|500px|center]]
 +
<br/>
 +
'''<big>Eastern Corn Skeet</big>'''
 +
[[File:T3_Assign_Audio8.png|500px|center]]
 +
<br/>
 +
'''<big>Green-tipped Scarlet Pipit</big>'''
 +
[[File:T3_Assign_Audio9.png|500px|center]]
 +
<br/>
 +
'''<big>Lesser Birchbeere</big>'''
 +
[[File:T3_Assign_Audio10.png|500px|center]]
 +
<br/>
 +
'''<big>Orange Pine Plover</big>'''                                     
 +
[[File:T3_Assign_Audio11.png|500px|center]]
 +
<br/>
 +
'''<big>Ordinary Snape</big>'''
 +
[[File:T3_Assign_Audio12.png|500px|center]]
 +
<br/>
 +
'''<big>Pinkfinch</big>'''
 +
[[File:T3_Assign_Audio13.png|500px|center]]
 +
<br/>
 +
'''<big>Purple Tooting Tout</big>'''
 +
[[File:T3_Assign_Audio14.png|500px|center]]
 +
<br/>
 +
'''<big>Qax</big>'''
 +
[[File:T3_Assign_Audio15.png|500px|center]]
 +
<br/>
 +
'''<big>Queenscoat</big>'''                                     
 +
[[File:T3_Assign_Audio16.png|500px|center]]
 +
<br/>
 +
'''<big>Rose-crested Blue Pipit</big>'''
 +
[[File:T3_Assign_Audio17.png|500px|center]]
 +
<br/>
 +
'''<big>Scrawny Jay</big>'''
 +
[[File:T3_Assign_Audio18.png|500px|center]]
 +
<br/>
 +
'''<big>Vermillion Trillian</big>'''
 +
[[File:T3_Assign_Audio19.png|500px|center]]
  
−
+
</div>
 +
<br/>
  
 
The oscillogram plots of the test audio files are shown below:
 
The oscillogram plots of the test audio files are shown below:
  
−
+
<div>
 +
'''<big>Score A File:</big>'''                                     
 +
[[File:T3_Assign_Test1.png|460px|center]]
 +
<br/>
 +
'''<big>Score B File:</big>'''
 +
[[File:T3_Assign_QualityB.png|460px|center]]
 +
<br/>
 +
'''<big>Score C File:</big>'''
 +
[[File:T3_Assign_QualityC.png|460px|center]]
 +
<br/>
 +
'''<big>Score D File:</big>'''
 +
[[File:T3_Assign_QualityD.png|460px|center]]
 +
<br/>
 +
'''<big>Score E File:</big>'''
 +
[[File:T3_Assign_QualityE.png|500px|center]]
 +
</div>
 +
<br/>
  
 
Comparing the test plots obtained with train plots:
 
Comparing the test plots obtained with train plots:

Revision as of 20:09, 8 July 2018

MC1 AkankshaYadav.jpg VAST Mini Challenge 1: Are The Beloved Pipits Disappearing?

Overview

Data Preparation & Approach

Discovering Trends - Exploratory Data Analysis

Is Kasios Singing A True Song?

The State Of Pipits

Conclusion

 


Analysis and Classification of Audio Files Through Visualizations

For analysis of question 2, two different approaches were implemented in order to evaluate the legitimacy of Kasios’ claim about the pipits.

  • Comparison of Oscillograms of the Audio Files
  • Comparison of Distribution of Plots Selected Audio Features with the Features of the Test Files

These methods were compared at the end to see if they produced similar or different results.

As the number of samples for each species is not sufficient enough, running a machine learning algorithm over such training data might yield poor results with low accuracy. Hence, in this analysis, the only focus was to do visual comparison of the plots obtained through both approaches.


Other approaches tried:

Spectrogram of all the files were plotted, but were not discernible enough for comparison of the audio files. Additionally, time measurement plots were also plotted to see if they can be useful in analysis. However, similar issue occurred while interpreting the plot. Hence, ultimately a simple oscillogram was chosen instead.


Conversion Of MP3 Audio Files to WAV

All the visualizations were built in R. As majority of the R packages can only read audio file in ‘.WAV’ format, all the audio files (All Birds & Test) were converted to ‘.WAV’ format using tuneR package.


Approach 1 - Oscillogram Comparison

  • The provided audio files of all the birds have quality scores associated with them wherein the value of this score can be A, B, C, D, E or No Score. However, no description is provided about the different quality scores. For the analysis, audio files without any score were not taken into consideration.
  • Hence, prior to beginning with the analysis, a set of 5 files were chosen with quality scores A, B, C, D & E for “Rose-crested Blue Pipit”. Out of these 5 files, the file with score A was found to be the clearest & without much noise. All other files were found to have varying degrees of noise in them.

The oscillogram plots of the chosen files are as shown below:

Score A File:

T3 Assign QualityA.png


Score B File:

T3 Assign QualityB.png


Score C File:

T3 Assign QualityC.png


Score D File:

T3 Assign QualityD.png


Score E File:

T3 Assign QualityE.png


Next, 19 training birds were chosen one from each species of the birds and their oscillograms were plotted. Similarly, oscillograms were plotted for the test files provided by Kasios. All the test plots were compared with the chosen 19 birds individually to obtain the best estimate

Now, let’s look at the oscillogram plots of 19 species of the birds:

Bent-beak Riffraff

T3 Assign Audio1.png


Blue-collared Zipper

T3 Assign Audio2.png


Bombadil

T3 Assign Audio3.png


Broad-winged Jojo

T3 Assign Audio4.png


Canadian Cootamum

T3 Assign Audio5.png


Carries Champagne Pipit

T3 Assign Audio6.png


Darkwing Sparrow

T3 Assign Audio7.png


Eastern Corn Skeet

T3 Assign Audio8.png


Green-tipped Scarlet Pipit

T3 Assign Audio9.png


Lesser Birchbeere

T3 Assign Audio10.png


Orange Pine Plover

T3 Assign Audio11.png


Ordinary Snape

T3 Assign Audio12.png


Pinkfinch

T3 Assign Audio13.png


Purple Tooting Tout

T3 Assign Audio14.png


Qax

T3 Assign Audio15.png


Queenscoat

T3 Assign Audio16.png


Rose-crested Blue Pipit

T3 Assign Audio17.png


Scrawny Jay

T3 Assign Audio18.png


Vermillion Trillian

T3 Assign Audio19.png


The oscillogram plots of the test audio files are shown below:

Score A File:

T3 Assign Test1.png


Score B File:

T3 Assign QualityB.png


Score C File:

T3 Assign QualityC.png


Score D File:

T3 Assign QualityD.png


Score E File:

T3 Assign QualityE.png


Comparing the test plots obtained with train plots:

Examples are shown for a couple of the plots:


Estimated Outcome:


Approach 2 – Comparison of the Distribution Plots

For this analysis, various features of the audio files were retrieved using ‘analyzeFolder’ function found in ‘soundgen’ package in R. This function has the capability of processing batch files as well as also outputs the result in a summary dataframe which contains all the extracted features for all the files.

70 such features were obtained from the audio files. To reduce the number of variables, correlation plot was obtained to discard highly correlated variables. The variables were then clustered in hierarchical clusters using ‘Variable Clustering’ method in SAS JMP. Finally, the following 6 features were selected for further analysis:

<<6 variables>>

Next, the distributions for these variables for all the 19 species were plotted.

Similar features were obtained for the test audio files.

The 6 features of the test files were individually compared with the 6 features of 19 species using density plot in R.

The resultant plots are displayed here for a sample bird for illustration -> Test Sample 3


Upon visual examination and comparison of the plots, the estimated outcome is:


Note that, both the outcomes do not match as they are based on only visual inspection. Although, by using the second approach, an attempt is made to classify the birds based on the statistical distributions of the species, it is still uncertain to an extent.


Plot for the test birds using the results from distribution analysis:

Retrieved from ‘https://wiki.smu.edu.sg/1718t3isss608/index.php?title=ISSS608_2017-18_T3_Assign_Akanksha_Shrirang_Yadav_Insights&oldid=8021’