From Chiptune to Chopin, how do the best video games use classical music?
How music helps us learn
- 28 Feb 17
The musical score is an integral part of horror, drama, fantasy and sci-fi. We anticipate that the instruments will amplify our emotions, carry our mood from scene to scene. Yet, documentaries are often devoid of the careful application of score.
If we apply the same techniques, the same instrumentation, we not only keep the audience engaged but help the viewer retain the information given. Filmmaker Kai Staats describes how he uses the musical score to support storytelling in science outreach and education.
Rhythm as a powerful tool
We have for uncounted millennia told stories through an oral tradition that was coupled with rhythm. The cadence, that proven, perfected system for keeping time helps us not only store information in such a way as to be more readily retrieved, but to engage our audience as well.
Not long ago, poems were presented as far greater than a few sentences or paragraphs. They filled entire volumes as the adventures unfold. Now, we struggle to recall even the phone number of our closest friend, yet our brains are capable of storing so much more.
When we fall to communication in little more than sound bites, Tweets and Facebook posts, we ignore the dynamic evolution which produced our musical brains. The instant gratification of our modern age has in some ways taken from us one of our most endearing gifts, the capacity to tell stories which captivate those who are within audible range. Certainly, it is the words that compel us to listen carefully but it is the music that helps us to remember the actors, the stage and the story.
I have grown to believe the score is as important in documentary and educational film as it is in a horror, drama, fantasy and sci-fi. The melody sets the tone, alerting the audience to the mood. The beats, those are the poetic cues, synchronizing the information through the delivery vehicle to the recipient who craves more.
When we, as filmmakers interweave music with words and visuals, we wield a deeply powerful tool. When used intentionally, carefully, we have the capacity to entice our audience, unknowingly, to more fully absorb the story. While I do not claim mastery of this tool, I have done my best to deeply integrate both purchased and original music in my work in films for science outreach and education.
Editing to the score
I edit to the score, as many do. I can lay down only a few minutes of a time-line before I feel lost without music to guide me, to tell me how next to shape the visuals. Having listened to dozens of tracks on-line, I download a few which feel appropriate for the mood.
I select the best fit, apply it to the editor time-line, close my eyes and play it over and over again. My index finger hovers over the m key, inserting markers at every quarter note or at least once per bar. I then adjust the markers to match, perfectly. The visuals follow, marching to the beat.
This process may sound mechanical, too robotic by the description alone - but the end result is subtle, the audience not necessarily recognizing why they are drawn in. With a jump cut, an off-axis swap of the point of view, or the termination of the interview matching the final vibrations of the trailing voice of a solo string, the audience is transformed from watching a movie to listening too. Fully engrossed, fully engaged in the rhythmic, melodic, visual story.
As movie goers, we expect the momentary throng of a cello when a character is nearly killed. We cherish the moments when the orchestra rises, the hero silhouetted against two setting suns. As a filmmaker I feel compelled to attempt something similar in science outreach too.
When a new subject is introduced, the score changes. When doubt is raised, the score mirrors the challenge of the issue at hand. When we spin away from planet Earth, I want my audience to feel lonely, even abandoned...but eager for the next adventure in gravitational-wave astronomy.
An adventure in modern lore
I am keen to insert musical interludes following the delivery of intense information such that the audience has a chance to let it all sink in. If an astrophysicist has shared something about the birth of the universe, no matter the level of mathematics or physics involved, the audience needs a moment to contemplate, to recover.
Science outreach films are a kind of translation, a means to tell a complex story to an audience who is not well versed in the medium. The translation should not be seen as a reduction of complexity of the underlying principals, rather an increase in intensity of the human story.
If left to the data, that story is dry, unbearable for those who do not speak the language of mathematics. If left to the original storyteller alone, he or she may be hit or miss as to how the story unfolds.
Storytelling is not what chemists, biologists, sociologists and astrophysicists are taught in those many years in graduate studies and post-doctoral research fellowships. But it is no secret that those scientists are still children, eager to tell of the unraveling mysteries of the world. When a little data is mixed with a light treatment of computer generated art is fueled by raw enthusiasm and united with a musical score, then we have a story to tell!
An adventure in modern lore.
About Kai Staats
Kai Staats, MSc, is a filmmaker, lecturer, visiting scientist and traveler. For ten years Kai served as CEO of a Linux OS and HPC systems provider; processing images at NASA, conducting real-time sonar imaging for the Navy, and bioinformatics at DoE labs. Now Kai engages his passion for storytelling through film. LIGO Detection marks his second National Science Foundation funded film, third for the gravitational wave observatory that detected two merging black holes in 2015. At Embry-Riddle Aeronautical University Kai guides the application of machine learning for the isolation of supernovae in LIGO data. When not at his computer, he explores those places which cell towers cannot reach.
What are the five elements of hip hop? From DJing to beat boxing, graffiti to B-boys, we reveal the music, art and dance that underpin hip hop
Don’t know your arias from your overtures? Our guide will demystify the top classical music forms