The physics inside piano website

september 24, 2007

Machines will be able to process and play moods

What are e-motions? are they cognitive processes? Could a machine have an intangible mind? Machines might "understand" and replicate something that is purely human thought as emotions? Can moods be interpreted as sensorymotor controlled wake of progressive impulses through passive-skeletal and active-muscular coordinated energies?

Poets call emotions suggestive thoughts that move the soul, brain research criterion describe them roughly as neural currents accross the hippocampus. Let's dare on say simply that opposite primitives emotions are easy to express with gestural motions as playing piano or forte. A mood from the other side may stand as an emotional state of intermediate duration between an emotion and a disposition to achieve a subjective expression of feel.

Isn't a Stars War fancy, technology is currently developing highly skilled human simulators capable to achieve very complex operations: biologic computers, nanotechnologies, neural science, etc. will expand their applications towards specific artificial intelligence projects in order to realise a vast spectrum of man everyday life activity, the (binary?) coding for the man body is real.

The first question to place about this I think is: it really makes sense to develop artificial intelligence projects for playing a musical instrument like piano? There might be multiple points of view to answer this.

artistic body binary emulation The utility in doing so can be seen for instance in learning to play without bad habits, to analyse and classify all of those movements which permit to a high-level pianist to perform classic piano compositions without technical errors, or in other words to identify wrong haptic kinesthetic gestures which cause stress, reinforcement of walls of velocity in hands at playing, etc. (see Human-Markup-Language (HumanML) project).
Hands are capable of a vast combination of movement patterns performed in a variety of positions, their movement is hence difficult to classify. Moreover, specific pattern training offers not only benefits for increased hands performance but is also the base for pursuing aesthetic goals, proof of this is the figurative arts development. Again, for the piano performing, the scene is that of fingertips surface provided with touch sensitive papilas, through which passive-skeletal and active-muscular neural data converges in time over infinitesimal areas from papilas to contact zones on piano keys surface. Playing finger, as has been expossed, is supposed to follow Euclidean space trajectories, mathematically represented as composed paths through fragments of concentrical conical surfaces.

Someone could argue at this point about individual style particularities, robotics could hardly reproduce human feelings, etc.; true, some amount of this issue is a matter for aestheticians tough. Framing instead the speech from the point of view of energy transferring and three-dimensional dynamic aspects alone, the under construction model considers 23 spatial coordinates of movement for the parts of the skeleton involved in the act of playing piano, having its ground state in the pelvis; i.e. the basis for adding muscle and neural components. An important beside aspect is the exclusion of all not-necessary motions from the analysis, the system must be limited (huge otherwise) excluding from the analysis all not relevant random magnitudes in the act of touching while developing musical artistic content, this is not less complicated too because the criterion to follow necessarily imply an interpretative assessment.

As far as hands motion is concerning we can consider the space over the keyboard parallel x-y planes as the effective mean square wrist motions space, and assume a (pseudo) distribution function for the trajectories of fingertips in connected staked surfaces (Gaussians-like digit-xtro in the image) over such a space.

Plane over keyboard as the effective wrist motions spatial domain

Now simultaneous music appreciation and musical expression at play involve both passive and active neuronal activity, reflexive and discursive processes in time are undergoing, so as aesthetic perception and critical reflection are deeply interlaced; it is clear that in order to obtain a fluid beautiful expressivity the awareness must be effortless and guaranteed by well regulated instrument's action. The mood expresses itself, by far, with at least two kinds of intentions which are translated into different movements, that is, performance-based intentions and aesthetic-based intentions, which correspond to alternating class movements from discursive to reflexive states of mind. So once an e-motional intent has been conceived the corresponding motion, or series of hands movements, needs to include at least the requirements listed in the table:

Attributes and requirements of hands playing piano movements
Intention Focus Motion Alignement and positioning Stabilization Propioception and neural demand
Performance-based goals Enhancing muscular performance Unrestricted (often unilateral) Varied, asymmetrical More self-supported High propioception, high neural demand
Aesthetic-based goals Enhancing muscular accuracy Restricted (eventually bilateral) Consistent, symmetrical Moody kinesthetic-supported Low propioception, high neural availability

At the moment (september 2007) I am considering whether it is convenient to operate in the model a power law which describes the relationship between the geometric properties of wrist trajectories (curvature radius) and kinematic motions (tangential velocities) in curved drawing movements which shape the digit-xtro function. Provided we can intuit the characteristics of such law for wrists motion we can extrapolate its meaning for more specific joints such as phalanges and fingertips dynamics. Although the power law is a general law of motion, there are conditions under which diminishes its wanted significance however, the power law hence may be less explanatory of movements around some joints, and this should help to identify "wrong" paths at play: considering how varying motion around different joints influence the fit of the power law.
From the other side, without specific data is only possible to speculate about motions associated with wrist and fingers which can be confronted when produced by some not-well-defined isometric force or bound elicited forces progression. The power law needs to be most explanatory about wrist motion and isometric force production, but least explanatory of fingertips motion. Given the complexity of the system the fit of the power law for finger and wrist motion may suggest separate laws for each joint system. By inspection of frame coordinates the fit of the power law seems to work out better for wrist than for fingertips motion, some attempt with virtual work has been weaved, tough without no significant outcome results. There is some doubt however that the power law explains better motion around fewer or simpler joint systems, and hence a fuller description mathematic tool is needed as the general kinematics deal with 23 independent coordinates. There is an interesting and inspiring narrative work on this item, according to author (Chuan C. Chang) the act of playing piano may be describable in terms of thermodynamics; cannonical ensembles would be applicable to describe it, at least in principle. In the section about statistical analysis of piano playing I will examine those ideas trying to review and extend that criterion.

One level back

Valid XHTML 1.0 Transitional