JHenius with Josiah Hester

Music Touch Squared Proposal – Josiah Hester

Abstract: Proposal for a mobile (iPhone, Android, Web) musical education environment and application suite.  Consisting of mainly three tools, a teacher’s exercise authoring tool, a teacher’s web based evaluation and statistical tool for targeting problem areas, and the student program for in real-time class exercises and out of class assignments.

Purpose and Summary:  The basic purpose of the tool is to help students learn, practice and hone their basic music theory skills such as harmonic and melodic dictation, interval training, chord and scale construction, key signature identification, rhythmic identification and many other possibilities, through exercises.  Each exercise can have many other constraints such as time limits, or clefs, as well as teacher defined constraints.  Students will do exercises on their device in class or at home and submit them instantaneously for grading and instant feedback.  Of interest is real time in class interaction: a teacher can give the class a short exercise in class and get the results instantly.  Another primary goal is minimum teacher work; the program will grade the exercise as well as gather statistical data.

The purpose will be accomplished though a variety of means, including;
1.    An authoring tool for teachers as well as students, this will facilitate both       parties being able to tailor exercises to their specific needs.
2.    The exercise tool itself.  It will consist of a basic interface showing current exercises and other minimal information.  From the interface you then “touch” one of the exercises to start it.
3.    In the future a database that will hold statistics and Meta data on students and exercises, which will be accessible from the web or mobile device.   This will allow teachers to identify problem areas and holes in their teaching as well as struggling demographics of students.  It will also allow them to tailor homework to students, making education personal once again.  (Ironically by depersonalizing it through computers!)
4.    Another key priority is availability.  Through abstraction, we will try to have iPhone, Android and Web apps that all have the same functionality for the student exercise application.   This can be accomplished through abstraction in Java, use Applets for the web and Android for the phone.  After abstraction the “bridge” module should just be busy work.  

Authoring Tool:
The main goal of the authoring tool is to provide power and diversity through simplicity.  Since the target audience is expected to know very little about computers, the interface will be as “touchable” and intuitive as possible.  Also to further accomplish this goal, there will be succinct but helpful mouse over tooltips.

The authoring tool will basically be an empty window, which the teacher will populate with one of three items: 1.) A staff,  2.) A keyboard, 3.) A textbox.  These items will be held in a “toolbox” of some sort, maybe something like Microsoft Word’s formatting palette, or the widget manager on a Mac desktop.  Either way it must be unobtrusive to the view of the exercise but able to be easily called up at any time.  
Each of these items will have certain inputs that are considered “correct.”  
Another approach is to just have a lot of pre made exercises, which the teacher can then use and just change a few settings.  A real time interaction would be very cool though.

Student Exercise Tool:
The exercise tool is how the student does the exercises.  

Web Based Tool
This online tool will use a Java online solution such as Joomla or Liferay to hold multiple web services for teacher and student involving music and education.  This will be the starting point for most people and a way to spread the word about the application.
1.    Evaluation Database Tool
Gives the teacher information and statistics about homework problems etc.  Needs to be very simple and straightforward as music teachers and artsy types are NOTORIOUS for incapability in computing.  Simplified database model and easy to understand, non-technical data.  Preferably all problems should be pinpointed by the service, minimizing teacher involvement.  From this tool the computer could make decisions about homework assignments and send them to the student.
2.    Student Exercises and Homepage
Students will have their own ID and login to access assignments etc.  They can  do the exercises online or with their awesome Android or iPhone.  Using Joomla simplifies this entire process greatly.
Notes:
1.    Get Dr. Rash inspired about music stuff.
2.    May have to take Music Fundamentals class to help understand music learning better.
3.    Sign up for section 5 of 481.  Code: 18782
4.    ABSTRACT the model.  Because Android intentionally keeps logic and UI separate it is much easier anyway.  The majority of our code may be abstraction. (Hopefully).  This needs to be well thought out and planned.  Or we could just plow ahead and fix later like usual.
5.    For the implementation of teachers sending exercises, use the BroadcastReviewer class and NotificationManager class, this will allow the hone to open the exercise as well as skip interfaces and go straight to the exercise.
6.    Study and evaluate the easiest ways of note entry with a touch.  Evaluate 1.) Simple note picking through GUI buttons. 2.) Linear entry, putting finger on screen puts note, moving up and down moves on staff, left and right changes note value.  3.) Clicking onto staff, then dragging.  4.) Using multi-touch: one finger anchors, second finger slides up or down to get note value/

Research:
1.    The Google Phone has touch, and multitouch capabilities as well as single finger scroll. Its hardware features are comparable to the iPhone.
2.    Android SDK’s debugging and application process is much smoother, and streamlined and the debugging Is arguably better.
3.    AT&T May take up the Android phone but it is doubtful before Christmas 2008.  If by 2009, great visibility.
4.    iPhone is AT&T only, and that will not change.
5.    Android is T-Mobile only currently.
6.    For web solutions, note that Joomla is completely free.  Also, if they want, possibilities for Digital Chalk as well. E-learning solutions galore.
7.    Note the Digital Chalk video and synchronized PowerPoint model.

Synthtation Pad: Methods for Efficient Music Learning and Composition using Tablet Architecture

Introduction:
Current music notation software (Finale, Cakewalk) while powerful and complete does not offer the budding musician or composer the freedom of movement and expression necessary to hold a student’s interest.  In this paper I propose a method of streamlining the music creation process by using the Tablet PC, as well as a variety of other methods for learning and efficiently generating musical synthesis.
Ideas\Key Concepts:

Take full advantage of the Tablet\Stylus architecture for composition by using the stylus to input notes using one of two methods:
1.  Older Method (Forsberg ,Dieterich, and Zeleznik) using stylus gestures to differentiate between notes.
 
2.  Newer Method: To us the dragability of the stylus to good effect.  The user would first place the position they wanted the note to be on the staff,  a “ghost” image of the note would appear (much like in Finale), note that the initial value of the note will depend on 1.) The key signature of the piece; i.e. if the piece was in 4/4 time the ghost note will be a quarter note, in 6/8 the ghost note will be an eight, and 2.) The user preferences.  Once the user chooses a location on the staff they may then drag left or right (while clicked, or after clicking) dragging left will decrease the time value of the note whilst dragging right will increase the value.  This amount of drag needed will be changeable through user preferences so that as the user continues to use this, minute gestures and flicks of the hand will give the desired result.  This method is extremely quick and efficient and has less probability of error than the above method since everyone’s handwriting is very different.  It also benefits from the familiarity of the “pen” or stylus while still using the computer to do al the hard work of formatting.  Note that this solution could also be implemented with a mouse nearly as effectively. This works since music is in essence sequential, each note holds it’s own disjoint time slot, therefore moving right or left on the staff is not possible for a note, so we can use this movement for functionality.  Advantages of this method over the old method are 1.) Ease of learning; you don’t have to learn any gestures, you just move the stylus, the ghost appears and you click and drag.  2.) Exact, what you see in the ghost is what happens, no room for error as with gestures.  3.) Fast.  Since you never have to take your stylus off the paper, theoretically you would be faster, much like how cursive is faster to write. 4.) Fewer errors equals better efficiency, since you’re not doing gestures you don’t have to spend time going back and fixing bad gestures.
b.  Use the stylus to enter a chord and have SPad construct the chord for you on the staff.  This function would allow you to compose in only chords.  For example you have the staff and above the staff you could use the stylus to write “Gm.” This would create a G minor chord on the open position on the staff (the triad G-Bb-D).SPad could create an arpeggiated baseline or melody based on the length of the chord, depending on user input and preference.  For this to be implemented SPad would have to be able to recognize a wide variety of musical notation chords, from jazz to rock.  
c.  Another Method: Using one gesture for a note then adding to the note bars to make a note larger or smaller.

Take Advantage of the Stylus for Educational Purposes:
    a.  Using the chord recognition software a student can learn different chords.
    b. The benefits of having an easy to use system for quick notation and playback         give the student immediate feedback.  
c. Use of score analysis and chord analysis allow the teacher to evaluate the progress of students.  Could be used in Music Theory classes to evaluate whether the student entered a correct chord or note, whether there harmonies were correct etc.  This ability for instant feedback allows the teacher and student to pinpoint problems and weaknesses.

Use Note Recognition for Composition and Education:
a. Sometimes a musical idea will pop into your head abut you never have enough time to completely write it down or figure it out on an instrument.  As a solution to this problem, I propose we use Note recognition software inside the Synthtation Pad to create scores.  The algorithm would first record the piece then evaluate and show the respective notes.  
b. Educationally this could be used for sight reading practice, the teacher would give a melody to be sung into the SPad, the student would sing it (with given starting pitch), then SPad would evaluate how close the sung melody was to the original.  Fuzzy intervals would have to be used for tempo and pitch.

 Use of Networking for Education
Another integral part of the education system will be the ability for a class to be networked to the teacher.  This will allow for teachers to send scores, questions, melodies etc. to the student.  The student ma then attempt to answer the question or harmonize the melody, then send the answer back to the teacher.  SPad will auto grade all assignments (unless told not to) and put the grades in a table of students, which the teacher will use to monitor the progress of the class and identify trends and weaknesses.