New Technology: Haptic Feedback for Touchscreens

Rubato Productions latest product, the Concert Hands System
is featured in Tick Tock Books UK HI-Tech World book COOL STUFF at Amazon.co.uk
The web link to purchase the book: http://www.amazon.co.uk/Cool-Stuff-tech-World-Clash/dp/1848982178 

Rubato Productions Haptic Piano Technology

                  Creates a New Category in the New Grove Dictionary of Musical Instruments

‘Haptic instrument’ refers to new types of computer controlled instruments.  In particular, a haptic instrument consists of actuators that exert forces on the musician, sensors that detect player gestures, an algorithm that determines what forces to exert on the musician, and a controller that runs the algorithm and interfaces with the sensors and actuators.  An ‘actively controlled acoustic musical instrument’ is a special case where an acoustic instrument is augmented with sensors, actuators and a controller.  The interface is the acoustic instrument itself while the controller determines the acoustic behavior.  ‘Augmented’ or ‘hyper’ instruments are haptic forms of traditional instruments with sensors to enable the player to control augmentation of the existing sound, such as for the “Overtone Violin.”

Keyboard instruments, especially acoustic and electronic forms of the piano, organ and carillon have received significant study. Piano and organ research is discussed below. Haptics is also used to design electronic keyboards that feel more like acoustic pianos and mechanical organs, as well as attempts towards programmable touch-adjustable keyboards capable of mimicking different traditional keyboards such as organ, harpsichord, piano, or clavichord.

For keyboards, the haptic feedback event begins when the finger approaches the key and ends with the final disconnect between the finger and key.  The player may sense forces caused by events such as flexing of action parts such as keys, compression of materials, inertia due to mass distribution in the action components, mechanical connections, relative timing of events, lost motion and bouncing. Keyboard arrangement and size affecting player inputs are considered haptic as well as ergonomic issues. The clavichord provides a unique example amongst keyboards in that the player is in direct control of string contact and sound production as long as the note sounds.  Harpsichordist fingers are in key contact through the quill pluck and initiation of the sound production. Pianists effectively throw the hammer at the strings through the action mechanism.  Organs are discussed separately below.

The specific circumstances of keyboard haptics study have practical difficulties. Acoustic and mechanical response is difficult to measure meaningfully.  Keyboard actions are difficult to study accurately due to the difficulty of isolating extraneous movement and because the actions are enclosed within the instrument; therefore, effective computer models have to be developed to supplement analysis. Realistic simulation of acoustic keyboards remains a sought-after goal. Psychological factors further complicate the pursuit of accurate objective results.

Piano: Piano pedagogues and performers have been discussing matters of touch, tone, learning, and performance effects at least since the early 19th century. Discussion of the effects of touch on tone was joined by theories of the merits of rigidity vs. relaxation of the muscles of the fingers, wrist, forearm, upper arm and shoulder.

Current areas of study and application in piano haptics include: Physiology of the arm, hands, finger and finger pads and what this implies for performance and injury; pedagogical concerns of learning and motor performance; analysis, efficiency and effectiveness of different types of touch and movement for performance problems such as rapid octaves, rapid passages, trills, leaps, repetitions and control of timing and volume; whether different types of touch can affect the quality of sound produced for a given type of action; psychological effects on perceptions of touch and tone, performance or instrument quality; how pianists respond to subtle action regulation and differing material factors; and differences between player response and action capability for different types of actions, including historic or experimental actions.    

Several commercial products employ piano haptic research and technology. Concert Hands™ by Rubato Productions utilizes robotics and aspects of haptic technology to teach piano playing by physically moving the student's wrists and cueing the individual fingers along with visual guidance within a displayed music score.

Concert Hands On Japanese TV http://www.youtube.com/watchv=1dYCCOt9kyc