Difference between revisions of "ISSS608 2017-18 T3 Assign Lu Yanzhang Data Preparation Methodology"
Line 29: | Line 29: | ||
The following tools have been used in this assignment | The following tools have been used in this assignment | ||
− | 1. '''Python''' - Timestamp transformation and new data source generation | + | 1. '''Python''' - Timestamp transformation and new data source generation. |
+ | |||
The following packages are used in this assignment: | The following packages are used in this assignment: | ||
− | pandas, numpy, glob, datetime | + | pandas, numpy, glob, datetime. |
− | |||
2. '''JMP Pro''' - Data preparation | 2. '''JMP Pro''' - Data preparation |
Revision as of 13:30, 10 July 2018
|
|
|
|
Tools
The following tools have been used in this assignment
1. Python - Timestamp transformation and new data source generation.
The following packages are used in this assignment: pandas, numpy, glob, datetime.
2. JMP Pro - Data preparation
3. Tableau - Visualization
4. Gephi - Social network modelling and visualization
Text File Preparation
Text file AllBirdsV4.csv's format contains inconsistent data values and missing data.
1. Format Date Field to MM/DD/YYYY
2. Omit data with no date value. As date field is important for us to identify the existent of certain bird specie
3. Recode the time to 24 hour scale with format hh:mm. Recode empty time to 12:00, early morning to 8:00 and am to 9:00
Audio File Processing
1. To process the audio files, following R packages are loaded in this assignment: soundgen, tuneR, seewave
2. As the function analyzeFolder() which converts audio files to dataframe can only read WAV format, it is necessary for me to convert MP3 format to WAV format. In the first step, I convert the all the MP3 audio files to WAV format
3. Not all the audio files are good quality. Some audio contains noise which will distract the audio classification task. Extract all the audio files which quality is 'A'
4. Call analyzeFolder() to read all the wav file as dataframe. And store the dataframe as csv format. Audio files in both All Birds and Test Birds from Kasios are required to process as above
The actual code for audio processing can be found at here