music explorer

by geoff peters and gabriel lo

Music Explorer For Windows

Authors:
Geoff Peters (home page) (gpeters at sfu dot ca)
Gabriel Lo (gslo at sfu dot ca).

Home page: http://www.sfu.ca/~gpeters/music

System Requirements
Download

Installing

Screenshot

How To Use

Discussion

How It Works

References
Change Log
Known Issues/Bugs

Wish List

License

A similar program to Music Explorer that has many more features (but costs money) is Transcribe! by Seventh String Software.

Links:

Visit Vancouver Restaurants.

Look here for the best Vancouver Wedding Music.

Restaurants Vancouver

Want to increase traffic to your web site? Read about my Search Engine Optimization tips for Google ranking.

Vancouver Flowers

Music Explorer is a free tool to learn about the nature of music in a visual way. It can also be used by musicians to help transcribe recorded music by identifying hard-to-hear chords. 

  • It allows you to visually identify which notes are being played in a clip of music and instantly see how music is made up of many overtones.

  • It uses the standard SMPTE time format to quickly select a certain part of a sound file to play and analyze. (SMPTE is made up of Hours::Minutes::Seconds::Frames where Frames range from 0 to 29.)

  • It also allows pitches to be identified by clicking on the piano keyboard, and an appropriate sine wave will be played.

As stated in the license at the bottom of this document, this is free software. Source code is available for C++/MFC, and can be used in projects licensed under the GNU Public License. The source code would be useful for anyone trying to write a waveform analyzer, sound player, or interactive piano keyboard.

System Requirements

To run:

  • Microsoft Windows 95 or higher.

  • Sound card

To compile:

  • Visual C++ 6.0

It may be possible to get the application to run under Linux using WINE. If you have success with this, please let the authors know.

Download

File Size Date Description
musicexp101.zip 2.21 MB June 7, 2003 Windows installation package and source code.

 

Installing

To install the program, unzip the file musicexp10.zip using Winzip, and run the file setup.exe.

If you need Winzip, get a free evaluation copy at the WinZip® Home Page.

Screenshot

The following screen shows a short sound file being analyzed (from Kenny Barron - Passion Flower Wanton Spirit). By looking at the spikes on the graph above the notes D, F#, A, C, E, and G# on the piano keyboard, you can tell that these pitches are present in the currently selected part of the sound file. If you know a bit of jazz theory, you would know that these notes make up a D9(#11) chord.

How to Use

1. Open a file

Open a WAV file by choosing File, Open. Most WAV file formats are supported, and more will be added in the future. After a file is selected, there will be a short delay while the file is processed. If you are trying to open a music file in MP3 format, you will have to convert it to WAV first, using an MP3 decoder. If you are trying to open a music file on an audio CD, you will need to convert it to a WAV using a "ripper". One free decoder and ripper utility is called dbPowerAmp.

2. Select a range

Select a range by clicking and dragging a region on the waveform, or by using the Functions, Select Range dialog. The currently selected range will be highlighted and the start and end times of the selection will be displayed. The frequency spectrum of the currently selected range will be shown above the piano keyboard.

3. Click the keys on the piano keyboard

To identify pitches, click the keys on the piano keyboard that are below the “spikes” on the graph. While the mouse is held down, the sine wave corresponding to the key’s frequency will be played, and the name of the note will be displayed in the status bar. For example, if you click on the middle C, then Clicked: 60 (C5) will be displayed, which means that the MIDI note number of middle C is 60, and it a C in the 5th octave.

4. Play the selection

Click the Play button on the toolbar to play the current selection.

Discussion

An interesting way to use this program is to discover overtones in music. By analyzing a recording of a piano playing a note or a chord, you can visually see that many more frequencies than the “fundamental” are being played.

How It Works

Programming Language

This application was developed using Visual C++ 6.0 and MFC. The application uses the Document/View architecture.

Keyboard

The keyboard is drawn as a repeating set of octaves which fit across the entire width of the window. Initially, each key was drawn as a rectangle, the result of which was thick borders around the edges of the notes. In addition, this method did not align properly with the frequency spectrum graph, which required that the width of each note be equal. The keys are represented by their MIDI note numbers, which can be converted to the name of the note and back.

To fix this issue, only the top half of the keyboard was drawn (up until the end of the black key) with each note having equal width. The bottom of half of the keyboard was drawn with lines to indicate the extent of the white keys. With these lines, the thick borders around the edges of the notes were eliminated, as only the black keys needed to be drawn as rectangles. However, in the bottom half of the keyboard, the white keys were not of equal width. This created another issue which needed to be solved when it came time to implement note clicking.

To calculate which note has been clicked on, the first thing which is figured out is which half of the keyboard has been clicked in – the top half or the bottom half. If the top half has been clicked in, then the distance from the left edge of the keyboard to point where the mouse button was clicked, divided by the width of a key (since they are of equal width), is a valid representation of the note that has been clicked. On the other hand, if the bottom half to keyboard was where the mouse was clicked, then some mapping needed to be done. First, if the point that was clicked would correspond to a black key in the top half of the keyboard, that piece of information could be used to determine what key it is in the bottom half. Since the lines for the white keys are always drawn in the middle of the black keys, if the point is to the left of the middle of the black, then the corresponding white key in the bottom half is the previous key. Otherwise, the corresponding white key is the next key. If the point that was clicked in the bottom half of the keyboard corresponds to a white key, then it is that white key which was clicked.

Sine Wave Generator

The algorithm to generate the sine wave uses a table lookup method for increased speed. The algorithm populates a table of 1024 entries with values from the math library’s sine function. Then, the current position in the table is calculated for each sample requested by using a normalized frequency.

Wave File Loader and Converter

The code which decompresses wave files was obtained from A Programmer’s Guide to Sound by Tim Kientzle. However, this code does not support all wave compression types, so in a future version a different decompression library will be used instead. The Music Explorer reads the entire wave file, converts it to mono, 16 bit, and saves it into a new temporary file. Because the file is in this specified format, it is easy to seek to any position within the file, which is useful when only certain selections need to be analyzed and read. This file is saved to the current Windows temporary directory.

Wave File Player

Although A Programmer’s Guide To Sound provided a class WinPlayer that uses the Windows Wave API functions to play sound, it was not thread-safe, and did not have any facility for stopping a sound file once it begins playing. This was a major problem, since it is necessary to be able to stop a sound file once started, and multiple threads are needed to allow the user to continue to use the application at the same time a sound is playing.

To resolve this problem, we rewrote the WinPlayer to a new, thread-safe, more functional version. This involved creating a thread-safe queue class to hold samples waiting to be played, adding several new worker threads, and to coordinate the stopping and starting of threads using events and critical sections. To help us deal with the low-level Windows Wave API functions (which are not very easy to use), we found and modified a wrapper class which implements standard waveform APIs in object oriented way (CWaveForm).

Waveform Graph

To draw the waveform graph, a representation of one element of the waveform had to be created. This was designed as a WavePulse, which contained the time the pulse occurs, and its magnitude.The time for a pulse is represented using the CAudioTime, an extension of the SMPTE time code. Instead of using lines to draw each WavePulse, rectangles were used to create a more continuous waveform.This required the calculation of where the next WavePulse would begin, and if there wasn’t one, then were the next unit of time would begin. Both of these were time-to-pixel conversions.

Calculating a selection range involved converting from the pixel values where the mouse was clicked, and the time that it represented. That is, it was a pixel-to-time conversion.

Fourier Transform

A Fourier transform was used to obtain the frequency spectrum of a currently selected sound clip. The Fourier transform was computed using a Fourier Transform library called FFTW (http://www.fftw.org). For selections with less than 5000 samples, an arbitrary-length algorithm is used. For selections with >5000 samples, the nearest power of two is calculated, and an algorithm is used which works on data sets with sizes of powers of two. This allows extremely large selections (such as 2 million samples) to be analyzed in mere seconds.

Frequency Spectrum Graph

The Fourier transform produces a resulting data set which needs to be scaled in both the horizontal and vertical directions in order to be displayed in the graph.

For the horizontal direction, the formula equal temperament tuning was used to obtain a logarithmic scaling function.

For the vertical direction, the U-LAW logarithmic scaling function was used. (U-LAW is a common standard which is also used for encoding samples.)

Time (SMPTE)

In order to keep track of the current selection, a special time representation was used.

The SMPTE time format is a standard developed by the Society of Motion Picture and Television Engineers for recording equipment. It is represented by Hours::Minutes::Seconds::Frames where frames range from 0 to 29.

A special class was created to keep track of SMPTE time, to calculate ranges and convert between numbers of samples and time values.

Internally, more accuracy is stored, which is necessary to draw a more detailed-looking waveform.

References

Code was obtained and modified from the following sources:

This program was originally a project for CMPT 365 - Multimedia Systems at Simon Fraser University in British Columbia, Canada. The professor was Mark Drew and the teaching assistant Cheng Lu.

Change Log

Version Date Changes
1.01 June 7, 2003
  • Changed the way the waveform is displayed so that it displays an average of the magnitude of all samples at each time interval instead of the first sample's magnitude in a time interval.
  • Enabled the scrollbar "page" functionality.
1.00 April 8, 2003 Initial release

 

Known Issues / Bugs

  • Crashes if a selection is made while the wave file is playing.

  • The select range dialog doesn't check if the inputted range is actually valid. So make sure to type carefully.

  • The rightmost scroll bars seem to work strangely.

Wish-list

Please note that if you wish to code the following features, please check with the Music Explorer web page (listed at the top of this document) to make sure that it hasn’t been done already. Also, if you wish to contribute your work to Music Explorer, please contact the authors.

  • Use a different wave file opener that supports more compression formats, and eliminate the need to use a temporary Samplefile (the following library supports seeking: http://www.zip.com.au/~erikd/libsndfile/)

  • Allow the audio playback to be slowed down by a given amount, using the Windows Wave API functions that do this.

  • Draw the waveform with a fast version of Catmull-Rom splines, as described in the following article: http://stevehollasch.com/cgindex/curves/catmull-rom.html A faster version of this algorithm is in:

A recursive evaluation algorithm for a class of Catmull-Rom splines
Phillip J. Barry , Ronald N. Goldman
ACM SIGGRAPH Computer Graphics , Proceedings of the 15th annual conference on Computer graphics and interactive techniques
June 1988
Volume 22 Issue 4

License

Copyright (C) 2003 Geoff Peters and Gabriel Lo

This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR APARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA2111-1307, USA.

This web page copyright (c) 2003 Geoff Peters. Music Explorer is copyright (c) 2003 Geoff Peters and Gabriel Lo.