Search Weight Loss Topics:




Apr 18

Is VR the next frontier in fitness? – The Verge

Fitness Comments Off on Is VR the next frontier in fitness? – The Verge

In the pandemic weve had to shift our lives to be socially distant we work remotely, we shop online, and we workout at home. Last year, I signed up for a virtual reality workout app called Supernatural, which runs on the Oculus Quest. Now my wife and I use our Quest 2 almost every day to work out in VR. It feels like were leaving the house, the music is great, and the workouts are no joke.

My guest on Decoder this week is Chris Milk, founder and CEO of Within which makes Supernatural. Milk has been making VR experiences for a long time hes a true pioneer of the field. Supernatural feels like his biggest hit yet, and it feels like fitness might be the perfect application for VR the one that makes people go out and buy a VR headset just to use.

The basic concept of Supernatural is really simple: you put on the headset, and then use the controllers to swat at targets flying in time to music. Its kind of like dancing or sword fighting. Dance fighting. Most importantly, Supernatural is fun there isnt a single other VR app that could get my wife to wear the headset, but Supernatural is so compelling we keep our Quest 2 on a charging dock in the bedroom. And almost anyone whos ever tried Supernatural on our headset has gone out and bought a Quest 2 almost right away. Like I said, it feels like fitness is the perfect application for VR.

So, I wanted to know a lot more about Supernatural, how the business works, and what kinds of strange music licensing deals are needed to make it run. Milk and I talked about all of that, but, as youll see, hes a much bigger thinker than that. Really, this conversation has a little bit of everything.

Chris Milk, founder and CEO of Within. Here we go.

This transcript has been lightly edited for clarity.

Your company, Within, has an app on the Oculus Quest platform called Supernatural. Its a workout app. The best way that I can describe it is dance fighting. It is very fun.

You have a long journey with VR to get to this app, which I think is one of the killer apps of the platform. I want to talk to you about the whole space, but give me just a quick rundown of Within, and your background in VR, and sort of how youve gotten to this point.

I was speaking to my mom recently and she reminded me, as a kid, I was obsessed with skiing, magic, and music videos. All of them in a strange way have ended up in what Im doing today, but especially so in Supernatural.

In more recent times, Ive spent most of my career exploring how we might use new technologies to create more meaningful human experiences, ideally, more life-changing human experiences. Most of them have been centered around storytelling, entertainment, music, art, and thinking about what we can make that would be impossible to make in the legacy artistic mediums that we were born with.

My career, I started in directing music videos. I worked with Kanye, Arcade Fire, Johnny Cash, U2, many others; moved into interactive music videos on the web as sort of the broadcast medium was transferring from television to the internet, and music videos basically staying the same, just being broadcast to YouTube instead of MTV. I took that opportunity of this new toolset to explore what kind of things I could do with music videos, if I used the tools that the format now offered.

That then led to immersive digital installations in museums. And that led to early in, I think it was 2012, I started experimenting and thinking about VR. In 2013, I was working on my first movie and pitching, at the same time, a startup, neither of which I had much idea of what I was doing on either front.

Then, I think 2014, I got a greenlight feature film starring Nick Nolte, incredible actor, dream project. Still one of the best unproduced scripts out there and Ive been developing it for probably three years. Ben Horowitz said that Andreessen Horowitz would lead the seed round of the company, and I promptly stepped off the movie and founded the company.

In the early years, we were trying to crack immersive entertainment, essentially figure out what comes after film and television, empowered by the technology of VR. The longer story there is that we were trying to solve a problem that not many people had at all, but through a number of different discoveries, we realized we could use immersive entertainment to solve a different problem.

One that my co-founder and I, Aaron, both had, which was that we needed to exercise but we vigorously disliked the act of actually doing so. We sort of thought deeply about what it would take to love rather than loathe exercise. We thought we could fill a gap in the marketplace that we saw. We knew it was there because we were personally sitting right in the middle of it. We quietly pivoted the company in 2018 and we went back into stealth mode. We built Supernatural over the course of two years. We released it in April of last year.

Is this the vision for where you thought you would be with VR? Your background is making music videos and feature films and big creative projects. And Supernatural just feels like a company that will scale the way a regular company scales.

Thats a great question with a complex answer. I think we were looking at this new technology and thinking as we have most new technologies. I mean, when I say we, primarily Aaron Koblin, my co-founder, and I, who have been working together for a lot longer than weve had this company. He used to be at the Google Creative Lab, and I was an independent music video director.

Those aforementioned interactive music videos, most of them were him and I collaborating, him at Google and me sort of as an independent artist and director, and really thinking about what can these new technologies offer that we cant do with what is currently out there? And when you think about the mediums and the art forms that we love, like cinema and television, radio, these art forms, these artistic mediums came forth through technological innovation, right? It wasnt Orson Welles tinkering around in his shop trying to figure out how to make a movie, that he invented cinema.

The technology of cinema was invented by scientists and inventors and technicians. And it was artists that took those technological innovations and figured out how to make, essentially, meaningful human experiences with them. So those technologies, because were on an exponential timeline of technological innovation, those kind of innovations previously came out every few decades or so. Now our working premise is that theyre coming out every year.

Were exploring these new technologies and thinking about what could be the next great medium, what comes after cinema, what comes after television. Thats what a lot of our explorations have been around, utilizing technologies on the internet like personalization and interactivity, and then moving into more immersive technologies like motion capture and virtual reality and augmented reality, and just thinking hard: where can these technologies take us as a species? Where can they take us from an artistic standpoint?

Ultimately, what were trying to do is make something that is meaningful to another human being in the way that other things and these other artistic mediums have resonated with us. Ultimately, were trying to make something that is life-changing. That is what great art, what great media, what great things do in your life. I think what weve come to learn, though, which is really interesting and we didnt originally understand, is that you can, that changing someones life is hard to do with a movie, its hard to do in a music video.

I think we both sort of had lofty ideals of whats possible, but youre not going to change someones life with the music video that you make. You might change someones life on the album. Ive had my life change from certain albums.

Same.

That really speaks to the sort of magical power of music, but Ive never had my life changed by a movie and I was, much of my life, trying to be a filmmaker.

I think whats cool is what we learned through the process of this, that if you can utilize all that same intangible magic of art and storytelling and crafted human experience, but calibrate it for a higher purpose; that rather than just entertainment, calibrate it to health, calibrate it to making your life last longer, making your life better from a day-to-day basis, let you have more energy, let you have a clearer head, let you be stronger. Then, you can actually use the power of art for a different purpose and one that actually does hold that original potential of changing someones life.

Thats really interesting because you gave this very famous TED talk in 2015 saying, VR was an empathy machine. I think that was really influential for a lot of people. Even my first experience using VR headsets for your products was, were going to take you to a different place and youre going to feel this experience that somebody else feels.

Supernatural is very much an experience about yourself, and I hear what youre saying; it makes sense that youre an artist, youre creative, youre able to think about how people will feel emotionally and how you could guide those emotions and use them for Supernatural, but its still a very, I mean this as literally as possible, a very self-centered experience. How do you connect those two ideas?

Yeah, its a little bit deceptive when you look at most of the interactive, immersive experiences currently, Supernatural included, are experiences that youre on your own. That absolutely does not have to be the case. It will not be the case for very much longer when you think about what this technology is. Its you tapping into a digital format that is connected to the internet, which is connected to anyone else that is also connected to the internet.

There havent been a lot of things that really strongly utilized the co-presence immersive nature, immersive powers of virtual reality but the small number of experiments and projects that have, are absolutely incredible. The sense of presence that you can feel with another person and the human connectivity that you can experience with another person, especially another person that you have a relationship with, that youre already close to and familiar with, is unlike anything else that exists.

I mean, just to give you a couple of examples, theres a mini-golf game on the Oculus headset and I routinely play mini-golf on the weekend with my best friend in San Francisco,I havent seen in ages. Growing up in junior high and high school, we used to work at a mini-golf range, coincidentally, together. The experience of wandering around this magical VR mini-golf range as we just sort of casually talk about whats happening with us recently while playing mini-golf is totally the experience that we would have if we were actually together, and you get that sense of human connectivity that you wouldnt get over a Zoom, you wouldnt get over a phone call.

One thing I ask every CEO and executive who comes on the show, what is your decision-making framework? Youre building something entirely new, how do you evaluate and prioritize what to do? How do you make decisions?

Yeah, good question. I mean, of course, every decision is different and has a multitude of different factors and variables that youre considering, but I think on the highest level the way we think about it is, we are humans and we are using technology to make things for other humans. And that technology is just a tool, and not to get so caught up in the technology, but really think about what is the end experience for that member and what is going to matter to them? Whats going to resonate with them? Whats going to make their life better?

We have this metric that is sort of a unique metric, Im guessing. We call it JAW. Its J-A-W. It stands for joy, awe and wonder. And If theres a decision were trying to make about a new feature or a new direction of the product or even a different layout of the menu, weve think about will this bring joy, awe or wonder to the experience for that member experiencing it?

One of the things about Supernatural that strikes me is that its very similar to the first hit app on the Quest, Beat Saber. What is your relationship to Beat Saber and the other apps like that? Because the first time I ever put on a VR headset, someones like, Youve got to play Beat Saber. Obviously Supernatural builds on the mechanic. Youve extended it, but how do you see that relationship?

I think were utilizing a mechanic that goes all the way back to Dance Dance Revolution and Guitar Hero. Theres been many VR rhythm apps. I mean, the first one that we were looking at was actually Audioshield, which was a great workout in and of itself. I mean, moving your body to music, that you are striking things out of the air, is the rhythm game paradigm of virtual reality. Its hugely effective if youre going to try to get someone to have fun and have a cardio activity at the same time.

What we did with Supernatural was, we saw that works well. That is fun, but how do we build it from the ground up as a legitimate fitness system? Obviously, we have coaches. Our movements are designed to be healthy over repetitive use over long periods of time. We work with lots of different experts to make sure of that.

Part of the education of discovery that we had, also, as a journey with the company was, in the earlier years, we were building a lot of these interactive music experiences. We were showing them at film festivals. Most people in the world havent seen them because we didnt release them publicly on the app stores. But were building a lot of different sort of music-driven, cohabitated social experiences like weve been talking about, working with Pharrell, and OKGo, and Justice, and building these really cool, immersive, music-driven experiences where youre moving your body and youre inhabiting crazy, magical experiences. Its sort of basically the evolution of music videos, going back to the early beginning.

What we saw with these experiences, though, as we were showing them at Sundance and Tribeca and South by [Southwest] and all those places was, people were coming out of them sweating profusely which was the first problem that we had to solve because youre putting a bunch of people in them in a row and they were laughing. They were totally filled with joy. They routinely thought that they were in there for a shorter amount of time than they actually were. Theyd do something for 15 minutes and think that they were in there for five minutes.

I dont know the last time you were on a treadmill and half an hour seemed like 15 minutes. Its certainly never happened to me. We thought, in the process of thinking about how we use this technology for this other use case, that definitely went into it.

Also, a number of studies around your perception of discomfort is lowered in virtual reality, which is why theyre using it in clinical settings and in childbirth and burn centers and chemotherapy centers. Theres something about your perception being removed from your physical body in space that just makes stuff hurt less, but it also makes it easier to push yourself harder when youre doing a cardio workout. It does hurt less and yes, its part of the fact that youre basically playing a sport that is fun.

Theres a major difference between being in a gym and a coach or instructor saying like, Okay, were going to do 50 squats in a row. Here we go. Ready? One, two. That experience is totally different than surfing and going into a tube and squatting and holding for 20 seconds while the barrel forms around you, but the physical benefit and activity is the same. Youre never in the tube going, How long am I going to have to hold the squat for? Youre just enjoying the experience.

Taking these things that we learn through the entertainment experiences that we were making, and utilizing the fun mechanic of a beat game construct. Then pulling in all the resources and expertise of people that specialize in human movement and physical training, and then really building a system from the ground up to incorporate all those things for the best, most awesome, fitness experience that you can build in virtual reality.

Lets talk about the headsets for a minute. One of my theses with computers, I hope everyone forgives me, I just have a lot of ideas about computers generally, but computers that you put on your body, it seems like a killer app for all of them, whether its headsets or smartwatches or whatever, is health and fitness, right? The Apple Watch became an exponentially successful product when Apple shifted its focus to health and fitness.

Supernatural is the thing that is getting a lot of people to buy a Quest and strap basically an Android computer on their face every day. Is there another killer app for the headset, for traditional VR headsets, like we have right now, that can drive adoption and get that installed base growing the way that everyone has always told me that it will?

Yeah. Ill say a couple things. I think what youre saying is absolutely correct in terms of virtual reality; fitness is the killer use case for VR. It will be the first driving force of mass adoption through a normal consumer audience. Just as an example of what were seeing from our membership base, were 50/50 split, women and men, where, I think over 60 percent are over 40. This is not what a typical VR demographic looks like.

The reason for that is, its using the technology for a different use case, and one that it works incredibly well for. When you think about going back to the technology of cinema, that was a linear sequence of still frames played one after another, not at first, but after not that long, sync sound went along with it. That technology, though, enabled the birth of cinema, but it also enables television, it also enables a Zoom call, it enables a how-to video that youre watching on YouTube. It even enables a spreadsheet in Excel, that youre moving numbers around. That technology of a linear sequence of still frames is what powers most of the things that we deal with on a daily basis on rectangles in our life, which there are many, as you know.

Thats sort of like a case study, and where does the technology go once its used for one thing, and the answer is it essentially goes everywhere. What youre seeing in virtual reality right now is very much what usually happens. I mean, the first thing being Chris Dixon talks about it all new disruptive technologies first appear as though they are toys. Virtual reality is a perfect example of that, but its primarily understood to be a video game system right now. But when you take the technology and you apply it to exercise or wellness as were doing, its a totally different paradigm and it is one that many people are much more open and willing to adopt the technology, because it solves a problem in their life.

Most people dont buy new technologies just because theyre new technology. I do. Most people dont. Most people buy new technology because it serves a purpose and it solves a problem in their life. You dont buy a television set because you want a black rectangle on your wall. You buy it because your favorite television show comes through it. The new technology of virtual reality needs not just your favorite television show, but it needs all the other use cases that could be enabled by applying the technology to that use case.

Lets talk about Within and Supernatural. How many employees does Within have?

It depends on if a few of them accepted an offer today, but were about 70.

How many of them are focused on Supernatural itself?

All of them.

Okay. Supernatural, the fitness platform, thats your entire focus?

Yeah.

What does it take to make a Supernatural workout experience? Who is the staff that produces that all at once?

Theres really two sides to our business, theres the technology business and theres a content business. [On the] technology side; all the multitude of complex systems that you might imagine to make something like an interactive, immersive VR and subscription platform work, from the sexy things like how the balls explode, to the less sexy things like back end account management all of that requires a large engineering team.

Then, theres the content side of it. Theres actually a few different pipelines that go into making the content. Theres the coach capture production pipeline, which is shooting the coaches and our 3D capture system that lives in Culver City. I should point out that we are, from a content standpoint, we are producing and launching new content every single day. Thats the flag we planted in the ground. When we launched, we said were going to launch a new workout a day.

Every morning, as you know, as a member, you open up the app on your iPhone or your Android phone, or you put the headset on, and theres a new workout that has a new musical theme or sometimes its just sort of an overall theme, but its new every day. Its a surprise under the Christmas tree of what the workout of the day is.

What goes into the content side of making that, the coach production pipeline, were shooting coaches five days a week to release classes, to release workouts every single day. Theres the environment pipeline, which is all the beautiful destinations that youre finding yourself in, song to song, those are not just a flat 360 photo. Thats actually a 3D model with super high-res images mapped on top of the 3D model. Thats a pipeline that builds those.

There is a mapping pipeline for the layout of the targets that are flying at you that are synced to music. Thats a team of trainers and choreographers that build the patterns of the targets and the placement of them that are synced to the music. We have a map-making tool that we built internally, that the mappers are actually in virtual reality and laying out the targets in space.

They can sort of forward the unit, they can reverse and play and stop and rewind the map over and over again as they physically place the targets in the air to go along with a picture. And they can see where the beats are up in space so they can line up the targets to the beat. Thats a very manual and creative process. Its not an easy thing to do.

Were basically, essentially puppeteering your movements through what is essentially dance, to music that you love, and that involves a really talented person that can choreograph those movements. It feels, as you said, like youre sort of, what was your terminology that you used at the beginning?

My wife and I call it dance fighting. I mean youre holding lightsabers and youre dancing, so we just got there.

Yeah. Theres an element of sort of kung fu martial artness to it, but youre also moving your body in choreographed ways to music. Theres also a hidden element of dance and moving your body to music is just naturally fun. So thats the beat mapping pipeline and then theres music curation and overall creative workout construction.

Every workout has a theme, has a playlist, has a creative thumbnail, has a creative name, has a description. Theres a creative content team that does all of that as well. Were programming and thinking about workouts that are months ahead. We really put a lot of thought and care into making each of these workouts. Its not just thrown together on the day, shoot it live, post it up. Theres a lot of curation and crafting and artistry from that team that really goes into making each one of those workouts.

Let me ask what the cost curve is of that, and Ill make the comparison to Peloton because it is such an easy comparison. Peloton, yeah, they have a bike and they have a software platform, and mostly though, theyre shooting videos of their trainers on a bike pedaling really hard and turning the resistance up and down. Theres some sensors and leaderboard stuff around it that makes that experience feel more interactive, but at the end of the day, its people on a bicycle and thats whats happening there.

You are describing a much more costly creative project, right? You had to build a software pipeline to do the choreography and the mapping, you have to build a software pipeline for volumetric 3D models of locations, you presumably have to acquire the photography and the imagery from all those far-flung locations youre in. Its just a lot of new things that are either at the top of the cost curve, or you have to invent them, which means youre going up the cost curve. Do you see that coming down or is that the moat?

Its a little bit of both. The cost does come down as we get better at doing this and building more scalable systems, but it is what it takes to make something like this. By our estimates, the business model does pencil out.

Thats good to hear.

Always good to hear, but we also, honestly, we think the market is a lot larger. There is a person that enjoys cardio exercise at home and takes joy from running on a treadmill or pedaling on a stationary bike. I, unfortunately, am not one of those people. By the way, God bless those people, I wish I was one of those people, but I think theres a lot more people like me out there.

I mean, 50 percent of gym memberships go completely unused year to year. Thats people that have actually gotten up and gone to a gym and signed up, and just not used their membership. They just havent found the solution that works for them. We dont actually look at our competition as a stationary bike. We look at our competition as the couch.

How many members do you have right now?

Thats a great question. We, of course, dont broadcast our current membership-based numbers, but what I can say is that our goal for the first year was to have as many members as is reported that a large, well-known, connected fitness service had after their first year. Were not at our first year yet, but its looking like we will have 10 times that number.

A large, well-known, connected fitness service. Is it the one I keep talking about?

You can jump to your own conclusions.

Whats your churn like? Its $19 a month. Are people signing up, paying the money and staying there? Are they trying it out and leaving? What does that life cycle look like?

Yeah, our churn is low and comparable to other hardware-based connected fitness services. The thing thats special about Supernatural honestly, is the cost of the hardware is like one-seventh of the cost of a spin cycle. Were half the cost of a monthly subscription fee. I havent measured the volume of a spin cycle versus a VR headset, but probably like one-30th, one-50th the size of the thing.

We take up zero floor space. You put it in a drawer when youre done, disappears, and you throw it in your suitcase and take it wherever you want to go. Thats really a different paradigm. Going back to the last question, I think its one that makes it a lot more accessible to a lot more people.

I want to ask one more question about the famous connected fitness company. Then, I actually do want to talk about headsets and platforms. Peloton, very famously in a big fight with the music industry, right? They dont pay the fees the industry wants. They have a point of view about it. The music comes and goes. Peloton users are notoriously kind of angsty about it.

Supernatural has the music. This year you made a deal with Universal Music Publishing Group, and you have similar agreements with other labels, so there is a trove of popular music in the app to workout to.

Is that, Chris Milk has directed videos with Kanye West, hes getting a deal? Is that youve structured a new kind of deal? Is that a cost thats going to accelerate as you become more successful? How is that working, because it seems like one of the hardest challenges for all of these services right now?

Yeah, its definitely, its not an easy thing to do as a small startup. I will say that when we started building this, we knew that we needed the best music, that this just was not going to work using needle drop or stock music; that if youre going to move your body to music, it needs to be music that you love. it needs to be music that youre familiar with and that youre passionate about, and it cant be music that youve never heard before.

To do that requires relationships with a huge amount of large entities in the music business. The first thing that I did was call the people that I know, my old friends in the music industry from being a music video director, and said, Listen, I know others have done this incorrectly. I want to do this right. I want to be a partner to your industry. I think theres something thats really special that we could build that is going to require some different type of creative thinking around deal structures and how we think about this platform in comparison to other platforms.

That was on day one, I made those calls. It was a collaboration from the get-go. It was never feign ignorance and beg forgiveness later. It was definitely Were going to do this together and were going to do it the right way. When you start working with the amount of music that were working with, theres really not an off-the-shelf tool that allows you to do what you need to do, which involves ingesting the hundreds of thousands of songs from many different sources and keeping track of them and keeping track of the rights.

Wait, are you paying royalty rates based on plays? If I play one playlist versus another, you pay different royalty rates?

Its really more complex than that.

Really?

Yeah. We could do a whole show on how music deals work. Its incredibly complicated and requires an entirely custom backend system to service it, but again, its what you have to do if youre going to create the kind of system that Supernatural is.

Well, I would compare that to the Spotify business model, where you pay Spotify some amount of money a month, theyve just allocated some chunk of your subscription fee towards royalty payments, and then they have their own backend to allocate those further. Is that how you think about the $19 a month, that just some percentage of it is allocated towards music royalty payments?

Yeah, I mean, in the most general terms, when you see a piece of music in a movie, thats a sync license and when you see music in Spotify or Apple Music, thats a licensing deal. Every song that you might potentially play, its not just a deal with one entity. There might be a multitude of rights holders because youve got the record label, but then you also have the publisher, and you have to have both. The publisher, if you have a song that had 12 songwriters on it, you might have 12 different publishers.

Youve got to make sure that every single thing that youre playing is cleared and theres nothing that just tells you like, heres your list of songs that youre good. Its really, its complex to say the least, and its something that its taken really years to get to the point where it works as effectively as it does right now.

Is your license primarily a sync license to make a derivative work on the music or are you just licensing the track?

What I can say is that we have the rights to hundreds of thousands of songs and that catalogue grows on a regular basis.

Do you have some sort of like custom deal that youve worked out to enable this product?

I mean, theres no off-the-shelf deal when it comes to music licensing. Everything is a custom negotiation across the board.

Thats totally fair.

It is a testament to the sort of openness and creative thinking of the music industry and how it thinks of service like Supernatural. Because it required a lot of creative thinking and constructive deal-making to make this whole thing work for everyone involved.

Have you had artists use it? Are they freaking out over it?

Yeah. Theres a few. Theres a number of artists that have used it. I wont name anyone specifically, but theres been artists that Im friends with that are using this service that I have to notify tomorrow that theyre actually there in the service so theyre not surprised.

Thats amazing. Are you going to do larger programming with artists, having them be coaches, having them talk to people? That seems like a real natural opportunity.

Continue reading here:
Is VR the next frontier in fitness? - The Verge

Related Posts

    Your Full Name

    Your Email

    Your Phone Number

    Select your age (30+ only)

    Select Your US State

    Program Choice

    Confirm over 30 years old

    Yes

    Confirm that you resident in USA

    Yes

    This is a Serious Inquiry

    Yes

    Message:



    matomo tracker