Cake
  • Log In
  • Sign Up
    • One of the most interesting presentations I saw during the Collision Conference in Toronto was that of Poppy Crum, PhD. Neuroscientist and Technologist, of Dolby Labs. The session's overview: "In her presentation, Poppy talks about optimizing the human experience by using intelligent audio and video technologies, sensors, and computational advancements. She will discuss how ML/AI has enabled and democratized a transformation in the quality and capability of experiences of the elite to the global consumer."

      Poppy began with some classical music.

      Why? Because JS Bach was not just a musician, a composer - he was a storyteller.

      “Bach was a master of understanding human experience and human emotion: unlike vision where our brain is getting a representation of space, with sound it’s different. You have all these different sound waves that your brain has to unpack and then reassign to create the experiences we have. How he did it became the rules of counterpoint, and for centuries composers have used them as a recipe book, because there are so many different ways our brains can unpack those sounds. If I’m a composer, and I follow these rules, you as a human will experience sound in this way.”

      This isn't limited to JS Bach: these types of storytelling techniques through sound can be heard in Angolese storytelling music as well.

      And when you're hearing music, what are you truly hearing?

      “What you’re hearing is your brain reorganizing that information…artists and storytellers have this amazing understanding of the brain that gets passed on.”

      “Leo Tolstoy wrote an essay called “what is art,”in which he defined art as existing when the intent of the creator is experienced by the individual on the other end. And today, we can actually know what you’re feeling. With the ubiquity of censors in your environment, we can understand the human experience, and take Tolstoy’s musings to a completely different level.”

      How does technology factor into this?

    • The science that Poppy and the team at Dolby Laboratories has access to can enhance storytelling through unprecedented visibility into how stories move us at the most fundamental level. Poppy shared:

      “There’s a chemical composition to our breath, sitting in this room, changing with our emotions. We can see joy, fear, suspense in the chemical composition of our breath.”

      The below is an image of audiences watching THE HUNGER GAMES and their bio-physical reaction to the suspense and action in the story.

    • Below, you can see the audience's breath and reactions to the edge-of-your-seat documentary FREE SOLO with each peak and valley charted out through reactions.

      So what does Poppy do with this information?

      “As neuroscientists, how do we take the physical world and understand how information is modified, passed on, lost - the kind of errors that happen?.. And how I can help creators create experiences that are translated to something that helps people tell their stories.”

      “Today there are so many transformations happening with our technology that we’re able to think about what that code is, and taking it to a new level. There’s an emotional meaning in what they create, so that no matter who we are or where we’re consuming that content, it’s experienced.”

      “Everything we experience affects how we experience information. What’s becoming more important is we have to be able to control and predict how we’re experiencing that information, because a one-size-fits-all approach doesn’t work.”

    • So how does Poppy utilize these insights and advances to help support storytelling?

      “There are 3 things I focus on: control, knowing how perception works and the physical world, all the malleable transitions in our bodies. Insight, how someone is experiencing that information and how we can enhance that. And creation, new tools for it.”

      Do people feel uncomfortable with the fact that we display so much of our interior reactions to stories? Not necessarily, as our bodies already broadcast our feelings in a variety of ways, and now we just have the ability to pick up on these subtle cues.

      “As humans, we may not like the idea of technology knowing what we’re experiencing, but in fact, we already do share these things. We haven’t changed as humans. The ubiquity of sensors in our environment has changed.”

      For example, pupil dilation is a huge signifier:

      “Your eye is responding to your world around you, and can give us great insight into what you’re experiencing.”

      For example - in the below chart, you can see people's physiological responses to key moments in soccer / football games, and they're off the charts!

    • Your body's physical responses to experiences like movies can be as easy to read as a book. Poppy described this as follows:

      “Your body radiates your stories of how you experience information. In fact, when we were developing Dolby Vision, which is one of our new tools that’s been out for 4 years, we were working on tools that would allow you to get up to incredible numbers of candelas per square meter…we could use thermal scanners and capture that the display was constant, but the people were reacting to the content, giving off heat, as though it was real. With the fire, just based on the image reaching my retina, I’m going to react as if it’s real.”

      The most interesting part of the presentation was how sharing a storytelling experience, like seeing a play or watching a movie in a theater, can actually add impact and dimension to the experience as opposed to watching something by yourself:

      “Many of the ways we’re understanding experience is alone, what does my body do. But we’re social creatures, and some of the most powerful experiences we have are when we’re together. One of the species I’ve worked with a lot is the Common Marmoset, a beautiful creature that’s extremely social…in this case they live in the Amazon in Brazil, but they live in an environment where they can’t see each other. Unlike other monkeys that are very visual, they’re very vocal. And the dominant female’s mental state affects the biology of all the other females in their proximity. I'm talking about the power of being human, together, and experiences. What we’ve found is when you’re having powerful emotional experiences, we actually synchronize our neural signatures.”

      Poppy then shared with the audience a clip from the action movie BABY DRIVER, and then explained what was seen in the below chart - red are the reactions from people watching the film together, while blue is reactions from people watching the film by themselves.

      “You see huge changes in the highly emotional scenes. Just being in the same space affects how you see the world. This is the power of why we’re here. This is why going to the cinema is so different than a virtual experience. It’s how we communicate when we’re proximal.”

    • This actually makes a TON of sense to me. I went to see John Wick 3 the other day and the audience reactions made the movie better. Hearing people Oooo and grunt when some excessive violence happened enhanced the experience. It became a communal experience. Very interesting article!