Microsoft was showing off Project Natal last week in New York. It was nothing new, just the familiar hand ball game that was demoed at E3 last year, but it did give me an opportunity to get a closer look at the progress that's been made since last summer. Specifically just how close the motion capture is to real life (which is something that Kotaku discussed today). I brought with me a trusty stop watch so you can get an idea as to how long it takes for your movement to be recreated on-screen.
I had a pretty large sample size, sitting through 5 demos, capturing about 40 different movements from a variety of journalists. Across those 40 movements, the fastest life-to-screen transition was .08 seconds, while the slowest was .12 seconds. A tenth of a second was the consistent average, though.
What that means is that, in its current state, Natal is not instant, one-to-one motion capture. It's close, very close, but once you start playing you'll undoubtedly notice a tiny delay. It's to be expected, certainly. Even high-end Hollywood studio motion capture devices have a slight delay, and a camera sitting in front of your TV doesn't come close to the fidelity of 30 cameras in a massive studio.
Interestingly, it would appear that the demo being shown in New York is identical to the one shown at E3, right down to the delay. I wasn't allowed to film the screen, but this exclusive CNET video from last August does feature person-and-screen footage and it gives a great sense of what you can expect, delay-wise. The appropriate footage kicks in at 2:05:
So is this a problem? Not really. It's true that games like "Wii Sports Resort" have all but done away with the delay, but that was three years after the release of the Wii, and they had to introduce a new hardware peripheral to make it happen. In theory Natal's delay could be mitigated by firmware updates and better programming over time. Also, I'd be shocked if Microsoft showed the same ball demo at E3 2010, and for all I know they're sitting on the killer app that features instant motion capture right out of the gate.
Even if they aren't, though, Natal is undoubtedly a game-changer. There's something innately freeing about not having to hold a controller, and I can definitely foresee a Wii-esque revival of non-gamers getting back into the action. After all, a tenth of a second isn't that long, right?
Are you ready for a trip down the rabbit hole? Tim Burton, Johnny Depp and Disney are adding a strange new chapter to the Lewis Carroll classic with their "Alice in Wonderland," a film that presents a young woman who finds herself in the world of the Mad Hatter, the Cheshire Cat and the Red Queen. She is welcomed as a returning visitor -- but is she, in fact, the same Alice who roamed the trippy realm as a child? Time will tell.
The cinematographer Dariusz Wolski enjoyed his work with Tim Burton on "Sweeney Todd: The Demon Barber of Fleet Street" so he said yes when the director offered him the director of photography post on "Alice in Wonderland." At the time, though, Wolski didn't know that, like Alice, he was taking a tumble into a strange and unsettling world.
"'Alice' was the most unusual thing I had ever done in my life," said the 53-year-old native of Warsaw. "Tim called me and said 'I am making this movie, will you do it?' I said sure because I like the guy. I had no idea what it was going to be."
And how would he describe that experience?
"A debacle," Wolski said with exaggerated distress. "I think Tim hated the green more than I did by the end."
The green, of course, is the vast emerald -surface set for the film; the actors did their work for Wolski's camera while moving through a green void that was designed to hold space for the computer artists who would later fill in the digitally designed landscapes of Wonderland.
"It was quite absurd," says Wolski, who previous credits include the "Pirates of the Caribbean" films, "The Crow" and "Crimson Tide." "You look through the camera and all you see is green. 'OK so there will be a castle there, a tree here and a hill there. And a moat, yes, a moat about there. There's this entire world that will be created but but it's not there on camera. It's...difficult."
The nature of the film changed as it went along -- it's a hybrid of animation and live action, but the ratio changed as it got underway as Burton tilted toward more performances by live actors. The green set became vital to the vision Burton desired but the director -- and his cinematographer -- had never spent so much time experimenting in that particular type of movie-making laboratory.
There were other challenges. The character Alice changes sizes and that means Wolski and Burton had to compute the angles and orientation of each scene.
"Sometimes she is six inches, sometimes she is two feet, sometimes she is eight feet. The eye-lines change, everything changes. It was a very bizarre project. And lighting? You're lighting blindly. everything will be filled in later after you are done. There is a lot of use of your imagination."
Tim Burton pulled out a surprising cinematic reference when asked about his version of Tweedledee and Tweedledum for "Alice in Wonderland."
"I kept thinking about the twins in 'The Shining,' " Burton said, referring to the chilling Stanley Kubrick horror film that celebrates its 30th anniversary this year. "But really any kind of twins. There's always something scary about them, in a way. Or there can be."
Burton said the strange, corpulent version of Tweedledee and Tweedledum was shaped largely by the performance of the actor who plays both, British comedian Matt Lucas, who did a "great job" tapping into the eerie nature of the brothers.
"We didn't want to do motion capture because I'm, personally, not really into that and since I like these actors I thought it would be really nice to use them -- with actors like Matt or Johnny [Depp as the Mad Hatter] or Helena [Bonham Carter as the Red Queen] I wanted to get what they brought to it," Burton explained. "So with Matt it's kind of a mix of animation and him. It's a weird mixture of things which gives his characters the disturbing quality that they so richly deserve."
Rachel Abramowitz had a front-page story in the Los Angeles Times this week on the angst among Hollywood actors as they watch more major filmmakers embrace performance-capture techniques and animation approaches. Here's a great follow-up as she talks to Steven Spielberg about the making of "Tintin."
Steven Spielberg says there was only one reason to make his new “The Adventures of Tintin: The Secret of the Unicorn” with the cutting-edge performance-capture technology that James Cameron used on “Avatar.”
“It was based on my respect for the art of Hergé and wanting to get as close to that art as I could," says the director, referring to Tintin’s author-illustrator, who created the international blockbuster graphic novel series (200 million copies in print) starring intrepid cub reporter Tintin, and his irrepressible canine companion, Snowy, as they venture through the pre-WWII world.
“Hergé wrote about fictional people in a real world, not in a fantasy universe," Spielberg said. "It was the real universe he was working with, and he used National Geographic to research his adventure stories. It just seemed that live action would be too stylized for an audience to relate to. You’d have to have costumes that are a little outrageous when you see actors wearing them. The costumes seem to fit better when the medium chosen is a digital one.”
“Tintin” stars Jamie Bell (“King Kong”) as the title character, Andy Serkis (Gollum in the “Lord of the Rings” trilogy) as his buddy Captain Haddock, and Daniel Craig (Bond, James Bond) as the evil Red Rackham. Produced by Peter Jackson, with the animation done by Jackson’s Weta Workshop, the film is due in theaters in 2011.
Like Cameron, Spielberg shot the actors on a special performance-capture stage. The performers donned lycra suits, covered in reflective markers, and their every movement was tracked by more than 100 cameras. They also wore a head-rigging with a camera near their jawline that recorded intensely detailed data of their faces -- enough detail to avoid the "dead eye" faces that had an unsettling lack of movement or emotion in many previous motion-capture films. Ultimately, all the camera data was fed into a computer to create a 3-D replica of the actor. The digital document of the actor and the performance is so all-enveloping that the director, in this case Spielberg, can go back and change the "camera" movement and orientation long after the actor has left the set.
For the director of such films as “E.T.: The Extra-Terrestrial,” “Jurassic Park” and “Schindler’s List,” the new experience was transporting.
“I just adored it,“ he says. “It made me more like a painter than ever before. I got a chance to do so many jobs that I don’t often do as a director. You get to paint with this device that puts you into a virtual world, and allows you to make your shots and block all the actors with a small hand-held device only three times as large as an Xbox game controller.”
With that small monitor, Spielberg could look down and watch what the actors were doing -- in real time -- on a screen that showed them in the film universe. Working on the motion-capture stage -- which is called the volume -- Spielberg was routinely dazzled by the liberating artistic value of the new science.
“When Captain Haddock runs across the volume, the cameras capture all the information of his physical and emotional moves," the director said. "So as Andy Serkis runs across the stage, there’s Captain Haddock on the monitor, in full anime, running along the streets of Belgium. Not only are the actors represented in real time, they enter into a three-dimensional world.”
So though Jamie Bell will be digitally made to look exactly like Hergé's classic renderings of Tintin, “it will be Jamie Bell’s complete physical and emotional performance,” Spielberg said. He added: “If Tintin makes you feel something, it’s Jamie Bell’s soul you’re sensing."
The other movie James Cameron was doing - Battle Angel
Well, it might have a new name when it is released.
Avatar has come and... I think is still coming in other cinemas, but how about that other project Cameron was talking about since 2005 a film called‘Battle Angel: Alita,’. Now that James Cameron has finished his "magnum opus" his producing partner, Jon Landau, says they can now give some attention on what Cameron is passionate about and that is Battle Angel.
Though Mr. Landau was quick to point out that Cameron is planning to shoot an “Avatar” sequel first. But he gave as the affirmation that the filmmaker’s intends to adapt Yukito Kishiro’s manga classic after that.
“We had a wonderful writer who came in and collaborated with Jim, Laeta Kalogridis, who worked on it; Laeta brought to our attention that there was much more to this world of Battle Angel than we ever knew,” Landau said. “We were familiar with the anime that had been produced, we were familiar with one of the main books. But she opened us up to the other nine books that exist, and how rich that world is.”
“We really wanted to take our time in developing a large arc-ing story that really encompasses the whole world,” Landau said. “We were very close to doing that movie before we did ‘Avatar,’ and I think you’ll see that resurface. That’s a story in our minds about: What does it mean to be human? What does it mean to fall in love? Are you human if you have a heart? Are you human if you have a mind? Or are you human if you just have a body? It’s a journey of self-discovery for a young girl.”
It was reported from the past few years that Cameron and co. will drop the name "Alita" from the title and simply call the film “Battle Angel.” But according to Landau, that will not be the movie’s name — and for good reason.
“I’m telling people that we have to call it ‘Alita: Battle Angel,’ because Jim only does T&A movies,” Landau jokingly said then listing the examples. “’Titanic,’ ‘Aliens,’ ‘Terminator,’ ‘Abyss,’ ‘True Lies,’ ‘Avatar.’”
“So, we can’t call it ‘Battle Angel’,” he insisted. “We have to call it ‘Alita: Battle Angel.’”
Perhaps you thought Quantic Dream wrote the story, designed the characters, and then sought out voices to go along with those characters. Well, not quite.
In reality, they got the actors and then created the characters in the story based on the hired voice talents' physical appearances. Therefore, when you're controlling Ethan Mars, Scott Shelby, Madison Paige, and Norman Jayden, you are taking control of the virtual images of Pascal Langdale, Sam Douglas, Jacqui Ainsley and Leon Ockenden respectively. If you want proof, check out CVG; they show you some photos of the actors alongside their on-screen personas in Heavy Rain. You will notice that Madison's hair is short and black while Jacqui Ainsley's hair is long and blonde but beyond that, it's the same body and face (a hot body and face, too). But the other actors are all very, very close to their virtual counterparts; Scott Shelby is Sam Douglas, for crying out loud. They also don't show you Aurélie Bancilhon, who plays Lauren Winter and again, the real and virtual version of that woman is spot-on. If you caught the tech demo featuring her some time ago, where she acts out a scene (not from the game) for an audition, you know who she is. She was also in the demo; she was the prostitute in "Sleazy Place."
When you get the game next week, you can see even more after unlocking extra content when playing. There are audition scenes for each of the four major actors, plus that tech demo for Bancilhon, and they even take you behind the scenes for motion capture. All those action scenes you see, like when Madison is attacked in her apartment? Acted out in real-time by real actors, my friends.
Last month, Morgan Freeman was part of an Oscars roundtable hosted by Newsweek and bristled a bit about the intensifying use of performance-capture work in filmmaking and the purest nature of acting. "I think it's a bit faddish, because it's really cartoons. ... If I can look in your eyes and see a completely different person, that's what I want." Rachel Abramowitz of the Los Angeles Times picks up that thread with an insightful article on the acting community's reaction to the alien allure of "Avatar." This is a longer version of her story that appeared on the front page of Thursday's paper.
Director James Cameron had many reasons to be happy the morning that this year's Oscar nominations were announced; his blockbuster film "Avatar" tied for the most with nine, including best picture and best director. But he was dismayed that his cast, including stars Zoe Saldana, Sam Worthington and Sigourney Weaver, was shut out.
In fact, unlike the great majority of best picture nominees, the "Avatar" actors have not nabbed a single major critic's award, or guild prize. The snubs reflect the apparent ambivalence of the film community -- especially actors -- to "Avatar" and its revolutionary use of "performance capture," a new technology that combines human actors with computer-generated animation to create the blue, 10-foot-tall creatures who are the heart of the movie.
To the uninitiated, it raises basic questions: Is this acting, or is it animation? And, does this suggest that actors could become obsolete? It's an issue that provokes a strong response from Hollywood figures, from best actor nominees Jeff Bridges and Jeremy Renner, to directors Cameron and Steven Spielberg.
"I'm sure they could do it now if they wanted. Actors will kind of be a thing of the past," Bridges told The Times the day nominations were announced. "We'll be turned into combinations. A director will be able to say, 'I want 60% Clooney; give me 10% Bridges; and throw some Charles Bronson in there.' They'll come up with a new guy who will look like nobody who has ever lived and that person or thing will be huge," he said.
Renner, nominated for "The Hurt Locker," put it this way: "Some movies are actors' kind of movies and some movies are more directors' movies. 'Avatar' is a spectacle. It's a beautiful experience, but it's not really an actors' kind of movie. It doesn't really allow for an actor to truly tell a story. The director's telling the story in that one."
Perhaps mindful that actors make up the largest Oscar voting bloc, Cameron fiercely promotes the contributions of his cast to the success of "Avatar." He and other advocates of performance capture (known as "motion capture" in its previous, less sophisticated incarnation), including Spielberg, say not enough actors have experienced the process to appreciate it.
"There's a learning curve for the acting community, and they're not up to speed yet," Cameron said. "We didn't get out and proselytize with the Screen Actors Guild as we probably should have to raise awareness. Not only should they not be afraid of it, they should be excited about it. There is a new set of possibilities, after a century of doing movie acting in the same way."
Cameron describes it as "an actor-driven process." "I'm not interested in being an animator. . . . That's what Pixar does. What I do is talk to actors. 'Here's a scene. Let's see what you can come up with,' and when I walk away at the end of the day, it's done in my mind. In the actor's mind, it's done. There may be a whole team of animators to make sure what we've done is preserved, but that's their problem. Their job is to use the actor's performance as an absolute template without variance for what comes out the other end. "
"I like to think of it as digital makeup, not augmented animation," said Spielberg, who is using Cameron's "Avatar" technology in his new movie, "The Adventures of Tintin: The Secret of the Unicorn." "It's basically the actual performance of the actual actor, and what you're simply experiencing is makeup."
In the case of "Avatar," he said, "the digital makeup is so thin you actually see everything that Zoe is doing. Every nuance of that performance comes through digitally."
Spielberg and Cameron say that making a movie in performance capture is, for the actors, very similar to performing a play. "Motion capture brings the director back to a kind of intimacy that actors and directors only know when they're working in live theater," Spielberg said.
Filming takes place on a spare motion-capture stage called "the volume." Actors wear skin-tight bodysuits with reflective markers; every movement is tracked by an array of more than 100 fixed cameras. There's another specialized head-rig camera to record the actor's face and eyes.
"The virtual camera is always active," explained "Avatar" producer Jon Landau. Gone is the need for camera and lighting set-ups, makeup retouches and costume fittings. Scenes do not need to be shot repeatedly from different camera angles. Instead, the camera data are fed into a computer that creates a 3-D replica of the actor's every movement, and the director can just add his camera moves -- from any perspective -- digitally.
"There's a purity to it. You can't rely on anything else but your own skill as an actor; [it] enables the actor to shoot the scene in one take without worrying where the camera is," said Andy Serkis, a veteran British stage actor who pioneered motion-capture acting as Gollum in Peter Jackson's "Lord of the Rings" trilogy. Serkis followed that up with the title role in Jackson's remake of "King Kong" and is currently performing in Spielberg's "Tintin."
"If you don't have the performance, the rest is dressing," Serkis said. "You can't enhance a bad performance with animation. You can't dial it up, lift the lip or the eyebrow. It has to be right at the core moment. It's the same as conventional shooting." For actors to not recognize "performance capture as acting is bad and disrespectful. It's also Luddite."
In the case of "Avatar," some complain that Cameron's characters are too one-dimensional to merit their actors a nomination, but others believe that "Avatar" star Saldana in particular, whose every minute on screen is in performance capture, was robbed of recognition.
"Zoe played Neytiri with such strength, grace and force. If the audience realized just how much, they would have appreciated the performance more," said "Avatar" co-star Weaver. "The technology is so innovative, and it will just continue to get more innovative -- we might as well recognize [the contributions of actors] now."
From a filmmakers' standpoint, filming in performance capture is unusually free and fast. On a typical day of a live-action production, a director might complete a dozen or so scenes in which the lights, cameras, scenery and actors are repositioned. Spielberg said that on "Tintin" he completed 75 set-ups a day on the motion-capture stage, and finished principal photography in 30 days. That's less than half the time it would have taken to shoot a live-action version of the film.
More than that, Spielberg said, the performance being captured is no less the work of his stars than any other film. “It will be Jamie Bell’s complete physical and emotional performance,” the filmmaker said of the actor in the title role. “If Tintin makes you feel something, it’s Jamie Bell’s soul you’re sensing."
Spielberg said the technology frees him up to focus more on the art.
"It allows the director and cast to focus on the performance," said Spielberg. "The director sits right on the floor [amid the actors]. Because he's not wearing a motion-capture suit, he appears invisible."
"One hundred percent of my focus is on the actors," Cameron said. "I'm not thinking about the lighting, the dolly, or waiting around ... to light the shot."
Though veterans speak enthusiastically about the performance-capture technique, questions remain. Many wonder whether Saldana will get the kind of career boost usually associated with co-starring in a box-office bonanza. The Screen Actors Guild recently appointed a committee to look into what SAG President Ken Howard described as "pay and recognition" issues associated with performance capture in both film and video games. In fact, studios haven't even formally recognized SAG's jurisdiction over the work, leaving it up to each employer to decide whether the performers receive standard union benefits such as minimum pay or meal breaks.
Moreover, the actors are not the only ones unsure about their primacy in the process. There's also a branch of animators who don't want their contributions overlooked. Cameron points out that it took a team of 20 or more animators at the Weta Workshop in New Zealand nine months to fully animate each "Avatar" character.
"The academy has to come to terms with where [performance capture] goes," said director Henry Selick, whose "Coraline" is nominated for best animated film. "Is it animation? Is it a new category? I'm like the academy. I don't know where it fits. I will tell you this, animators have to work very, very hard with the motion-capture data. After the performance is captured, it's not just plugged into the computer which spits out big blue people. It's a hybrid."
-- Rachel Abramowitz
Photo credits: "Avatar" film scenes and on-the-set photos: Fox. Illustration of Neytiri of "Avatar" receiving an Oscar: Alex Gross / For The Times. James Cameron goes native: Kevin Lingenfelser
Avatar has come and... I think is still coming in other cinemas, but how about that other project Cameron was talking about since 2005 a 
Heavy Rain