Here’s a behind the scenes video of some of the processes we used to make “Uncle Griot”. It’s accompanied below by an interview I did with the “Florida Animation Festival” about some of our techniques, inspirations and the big challenges we faced in getting it made at all!
FAF: What do you think are the benefits of working in CGI animation? The drawbacks?
PC: I’ve found one of the biggest benefits of working in CGI is also one of its biggest drawbacks. It gives you an almost limitless potential for telling your story in any environment you can possibly imagine (or are yet to imagine) with any characters you want, human or otherwise.
The big danger here is that this lack of creative boundaries can actually hinder rather than help the creative process–particularly if you don’t have a producer constantly reminding you of cost and timescale. They have a way of making you focus on specific choices! We found this problem especially true with the outdoor natural environments. For conventional filmmaking, there’s a certain amount of design in the environment at the pre-production stage and much of that you can get from location scouting. But the key difference here is that the final cinematographic process is also very reactive to the pre-existing design (i.e. the natural world). The director can react on set to light and form by just moving around. It’s a reactive as well as a prescriptive process. The problem with CGI landscape design is that it’s entirely design driven by concept art. Even if you start from photos or paintings, you have to build the landscape in a way that works from multiple angles, so it’s very easy to lose that important ability that I believe is essential to good art: reacting to things beyond your control, to mistakes as well as external agents and limitations. I’ve seen it with professional feature directors (and myself). You can end up with a “kid-in-the-candy-store” effect where too much choice means you ended up piling too many things in, or getting distracted by some other cool idea you suddenly realise is possible. In my opinion, this is one of the reasons a lot of CGI can sometimes look flat, contrived and uninspired. It’s over-designed. Our issue was how do we make the environment in a way that we can react to it as if we are on set? We never answered this problem completely in my opinion, although I’m mostly very happy with our landscape; but we had to do a lot of very time-consuming set dressing and landscape design per shot to camera. (Matte painting can be a savior in these instances if you have only a small amount of parallax or a static camera.) For our next project, the feature film that the short was developed from, we are planning to use as much real life scan data as we can as a starting point and also plan our shots using a game engine such as UNREAL. We are hoping this will give us the ability to throw ourselves into a world where we can react to colour, light, and form as well as design it.
In many respects we are using CGI to mimic a real film process, as that fits our narrative style of magical realism. It also gives us a fairly unique approach to our shot design. We captured all the performances on set in a similar way to a stage play. We had limited space so we concentrated on getting the best, most honest performance we could with only minimal blocking for camera. Our plan was always to try and react to the performances in post and frame and edit shots accordingly. This gave us a lot of freedom to build edits based on beats coming directly from the performance, and also overcome the restrictions of the performance space. This is, as far as I know, an untested approach, as the editing of a shoot, is normally restricted by the camera coverage, or by each shot being blocked specifically for a single camera. We had complete freedom to put the cameras anywhere we wanted after the shoot. It allowed us to have more improvised, as well as heavily scripted scenes which really helped us get the most out of our child actors, as the physical blocking for camera was less of an issue. You can see this particularly in our “Uncle Griot” short, where the interaction between the four-legged Griot (played as a reference for the animators by me crawling around on set) and Stina was largely improvised. This process allowed us to then rebuild the performances with basic previz rigs and apply our “egg head” models: projected facial performances from the actors head cams. We could then review the scenes and try and frame and edit shots sympathetically to the performances and our narrative intentions. This freedom also had its drawbacks though, as you need to be very clear on the intent of every shot, as the infinite potential can be completely overwhelming!
From a technical point of view, the biggest hurdle with this kind of high realism CGI is faces (in our case we were not going for photo real, but idealised). Doing it well gives you the chance to really communicate some subtle and nuanced performance beats, but it also requires a lot of work. As someone who has a lot of experience in the realistic facial animation process, (having worked professionally animating “Gollum” on “Lord of the Rings” as well as other characters) I knew how challenging it was and built a very detailed and realistic facial animation system before we even planned the shoot. We used blend shapes and other rigging tools and based it on the Facial Acting Coding system of Dr. Paul Ekman (this forms the basis of most film industry facial animation systems) This is essentially a sculpting task and requires good observation skills and a lot of patience. The next stage was translating the performances from the actors’ faces onto the rig. This required the best quality locked-off video we could get. (we got this by making our own face cams from a weightlifters neck rig and some webcams, all very Heath Robinson, but it did the job!) The next challenge was copying this data onto the rigs. We used FACEWARE software to do the first pass. And then did a pass of hand keying to finesse it. There is as yet no quick way to automate this process completely and it’s our biggest concern for the full feature development we are currently planning. (Even the big studios have admitted it always comes back to hand keying at some point. There is, as yet, no real substitute for the animator’s eye!)
FAF: Watching your behind the scenes video, I was awestruck by the use of light in your film. Can you talk a little about this?
PC: The world of our film exists in a place between reality and dreams. The mountains Stina inhabits are above the cloud layer, with nothing but the sun, a thin atmosphere, and stars above her. We wanted a dreamlike quality to the light. As well as being very influenced by the dreamlike, light-soaked cinematography of Terrence Malick, a big reference for us was sunlight as seen in space. We particularly wanted to get the prismatic effect of light breaking into its component parts. We used the “light leak” trick, often seen in film these days to try and get this effect. Some of it was done in post using Nuke and reacted procedurally to the light in the rendered frames, whereas other times we used stock footage of light leaks through real camera lenses composited over the renders. In each case, we processed the effect to try and amplify the prismatic separation. I also noticed that when you split out the prismatic colours of white light, even subtly, it really seems to give the colours in your image a lot more vibrancy, and I think this helped with the dream-like effect.
Our feature has a specific colour design journey that develops with the story, and as this short was taken from our feature project, we also used that to hold with the lighting design. At this stage in the film we wanted to get a real feeling of space, sky, and sun and the sense of infinite possibility this can give, as this reflects Stina’s point in her story arc. We tried to keep the horizon clear in most shots, allowing as much blue light from our sky (projected by an HDRI dome light) and have this contrasting vibrantly with a lemon-tinted sun. This also allowed us to make the most of reflections from our environment cloud map, which we moved slightly for composition every shot.
As we wanted a “filmic” look we also tried to mimic the onset film lighting process. Our render engine: Arnold, allowed us to use 3D modeled bounce boards. We physically placed them in the scene to add fill light bounced up from the sun. We also deliberately used the real-life limitations of keeping them out of shot and having to move them along with the actors. We also added ”beauty” lighting with spot and rims to amplify the staging for the close-ups and get that characteristic eye-lit effect you see in glossy Hollywood films. We set these lights up after getting the environment lighting how we wanted it, so again this allowed us to mimic the process of real on set lighting.
One of the advances we are really hoping to make for our next stage in the feature production is recreating as much of this lighting process as possible in a game engine. We are hoping this will give us a much clearer sense of the final look at a previz stage, which means less (costly) revisions when we do the final lighting. It would be amazing to set up a reactive environment that we could experiment in and it would also help us make the process as organic as possible.
Just a quick post to say our short film “Uncle Griot” has finally been finished. Here’s our first stab at a poster for it:
It’s been a slow and sometimes tortuous process as we choose a particularly difficult combination of elements to animate. Basically everything moved; the grass moved because of the storm, Stina’s hair flicked, the cloth moved, the faces moved; one of the stars of the short is totally covered in dense hair that had to react to his body and the wind, the trees rustled, the leaves flew around and the clouds moved. This sort of stuff is all par-for-the-course in feature production, where you have multiple departments all tied together with a coherent pipeline supported by tools programmers. But when you can count the number of VFX artists working on it on one hand, it can become tricky getting it all done efficiently and in a decent amount of time. The effort was all worth it in the end though and we’re very excited about the final result. It offers a brief but rich slice of life in Stina’s universe. We spent the whole summer rendering the assorted passes and composited them as we went, finishing the final edit a few days ago. The plan now is to have a screening for the cast and crew, then fire it off to the international short film circuit. This is a fairly daunting prospect, as it’s a new and complex world of “dos & dont’s” that i’m not familiar with, and can apparently get very expensive if you don’t navigate it with caution. I intend to get advice on this. Also the short is book-ended with some music that we needed to get clearance for, which will require a certain amount of funds we are yet to secure (more news on this soon hopefully) I’ll be making some behind the scenes videos about some of the assorted and complex processes we used to make the short. Look out for these over the next few months. Its been very hard work with a lot of late nights, but it’s been worth it 🙂
Today I thought I’d talk a bit about our Comic Con visit, the Comic we made for the show and the extended Stina universe this comes from.
The show ran for 3 days and myself and a Stina team member: Fiona Ware-Heine were representing Stina on the Friday, along with our trusty mannequin “Claus”, resplendent in his Militia uniform (as worn by “Gunter” in the trailer)
We also had 500 freshly printed Stina comics with us. They were produced in just 4 weeks by our super talented story board artist Chavdar Yordanov, who worked all hours to get the drawing finished on time (by all accounts not actually going to bed at all on the deadline day!)
On finishing the last drawings at about 4 o’clock in the morning on the last possible day to allow us print time, he sent them over (he lives and works in Bulgaria) and I added the remaining graphics, colour work and inlay art then sent them off for printing.
The final printing was all very last minute and went right down to the wire, but we got it all out, just!
Some comics stacked and ready to go to the Comic Con punters (with a free sweet – that clinched it):
Although the comic carries the “Stina & the Wolf” logo it doesn’t actually tell Stina’s story, we are leaving that for the film. It tells another story from what we are calling our “extended Stina universe”. As well as the screenplay for the Stina film, we have created a richly detailed and expansive universe of cultures, characters and histories; this has already spawned a number of short stories (one of which: “the Stover” was the one transformed into the comic) as well as a role-playing environment that has allowed some of the Stina team to explore the world of the film for themselves:
It’s been a great chance to explore the Stina universe as actual characters who have to live in it, and it’s added real depth to the world of the film and created brand new stories that go beyond Stina’s experiences (not to mention beyond her continent!) The players have been able to shape events and ideas that are only hinted at in the screenplay, and this has been feeding back into and shaping the world lore.
I shall be publishing the short stories in a collection when we have enough to fill a decent sized volume. (We have 3 at the moment) And I’ll also be publishing the “Stover” comic online sometime soon. We are thinking of adding an extra tab to this web page for the comics and short stories, so keep your eyes peeled!
Our thanks go to the Peta, Naomi and the University for allowing us the opportunity to make the Comic, as they paid for its production. Hopefully the “#1” on the front of the comic will soon be joined by a “#2” and there is already talk of a University presence at the next (and apparently bigger) Comic Con in May 2017.
Until then, as always, we are getting closer and closer to finishing our 5 minute short film and have now finished calculating all of the dynamic animation passes (hair, grass, trees, plants, cloth, rope) and are about to start preparing all the shots for final render.
Much more on this soon..
I haven’t blogged about Stina for some time. This was mainly because there are only so many ways you can say “we are still working on the short film” before it gets a bit dull, particularly as the most exciting parts about the new film we want to keep under wraps until it’s out 😉
However, we are now “animation final” for all the animation and are getting close to starting the final renders off and are are just giving the shaders a final “spruce up” before sending it all off . We are scheduled to have an in-uni dedicated render farm after Christmas, so will probably start rendering when that’s up and running.
Today i just wanted to chat a bit about shader development. For those unfamiliar with the complex world of CGI, a “shader” is a computer algorithm that allows a 3d model to have surface properties that dictate the way it reacts to light. This includes the base colour of the object, often a painted or photographic image, the intensity and colour of the reflections and, particularly in the context of skin, the translucency of the object and the way that light changes colour as it hits different levels of depth. (this is how that unique “skin tone” effect is created) Also any tiny sculptural details such as pores or scratches that are too small to model.
This image above is only partially rendered, you can see the areas that are still being calculated look noisey and incomplete. For shaders such as these, although a lot simpler than skin, there is still a lot more work needed than meets the eye to add a level of textural realism. The buckle for example is actually made up of 2 shaders blended together; One is made to look like rust, using a painted image of rust colours combined with a very rough , reflective surface. And the other is a shiny metallic surface that reflects everything around it. These two are blended together. A “mask” is used to tell the renderer which shader to use where on the surface. In this case an image of scratches was used as a mask:
The image was colour corrected, so the contrast made the scratches distinct from the background metal and it was used as a mask to simulate the chrome metal plating being scrapped off the buckle, causing the ferrous metal beneath to rust. To add even more realism an “ambient occlusion” map was created, and this was added in addition to the scratch mask. This allowed rust to be visible in the areas where the surface joins are, as liquid would be more likely to settle and corrode in these areas over time. An “ambient occlusion” map is a texture created that assigns a colour (normally black) for areas that are in creases and another colour (normaly white) for areas that are clear of any other surfaces. an example here:
You can see here that ambient occlusion creates dark areas in the creases, perfect for blending in a rusty material that has formed into the cracks and scratches. You can also see the effect used here in the door handle and bindings used in the opening shot of our short film:
If you look carefully you can also see the same effect was used in the wood shader. The wood is lighter and bleached in areas that would get a lot more sun, and darker with more color saturation; as if the wood has retained its original staining in areas that would get less harsh sunlight. Although this may seem like a lot of unnecessary trouble to go to, these elements all add up to create a feel and atmosphere that although not immediately noticed, are registered subconsciously and if absent can make the render look very “CG”.
So the hardest shader by far to make on a project like this is the “hero” skin shader. And particularly tricky for us is hitting the right mix of realism and stylization. This is something i suspect we will be tweaking right up until we press “render” on the final film (or should i say “composite”, as we will probably fiddle with things a lot after we’ve rendered them, but that’s for another day)
The main tool in our shader box of tricks for skin is “sub-surface-scattering” or SSS for short. This technique usually involves creating 3 different colour maps to represent cross sections of the skins colour at different depths. In the case of skin: epidermis, dermis and subcutaneous:
In a basic rendered ball example, using just simple coloured textures for each of these layers a CGI skin shader looks like this:
However, to simulate the appearance of real skin on a person requires a lot of observational skill. You have to pick out what colour goes at what depth and where. And what part of the colour is actually not the skin itself, but the result of the oily resdue produced by the pores that permanently coat the skins surface, as this creates a slightly pearlescent sheen. This isn’t a new battle, artist having been obsessively trying to capture the various qualities of skin in art for hundreds of years:
The way this is handled in the CGI world, where we have to try and build a representation of skin that aspires to work under any lighting condition, is the “skin shader”. The default one in Arnold is illustrated well in their documentation here:
They allow for 5 layers in total to build up the final look. Almost like layering translucent paints over each other to get a final colour. They have two reflection layers “sheen” and “specluar” that allow for the oliy sheen that covers the whole face, and another layer of shine for moist areas such as the mouth and eyes, as well 3 layers of colour at different skin depths. Here is an example of some of Stina’s early development colour maps unwrapped as single images. This first was used at the Dermis level, the second at the Subcutaneous level:
One key thing in developing shaders like this is consistency. We use a single lighting environment for all our shaders. It has a simple studio style 3 point lighting set up, and a High Dynamic Range map that simulates a realistic forest environment. This gives us a nice range of 2 extreme lighting conditions. It is very important that all the shaders are made under the same lighting situation, as if they aren’t its very easy to end up with shaders that react unexpectedly. We also use a neutral mid grey shader ball so we have a constant object for reference. Here are some test renders using our forest shader “Look Dev” scene:
And here’s one using our studio lighting setup (notice the SSS on the teeth. I’ve posted a real photo also so you can see how we reference for the shader development. Reference is key to all of our shader work):
I shan’t go into the whole development process of the skin shader, as that’s an article in itself, but i’ll post an example below showing where we are now with the look development of it. I’ve also included a picture from the film “Tin Tin”, as the film we are making is similar in nature, in that we are trying to find a middle ground between realism and sylisation, which is very tricky! Broadly speaking, we are trying to achieve “hyper realism”: exaggerating textural elements as a means to emphasis ideas in the story; making her skin slightly too perfect, too translucent, too opalescent, her eyes slightly too intense in the way they catch the light. It’s a slow and iterative process and experimentation is the key. We are now at the point where we are hovering at the door of photo realism, possibly even more than “Tin Tin”, and will use the short film to see how well this works with the animation, as a moving shader is quite a different experience to a cunningly framed still, particularly when its talking!
This was our first serious foray into external funding for the project and was a big learning experience for all of us. We worked very hard for 6 months on our Kickstarter campaign, with endless rewrites and redesigns of the Kickstarter content, promotional work on Facebook and Twitter, video interviews and promotional videos and produced a ton of graphics and banners. But unfortunately we didn’t hit our target of £50,000.
We did receive a lot of great support however and got a lot of useful exposure on social media (our thanks to LiasonPR for their work here) but at close of play we only managed to hit £4,332 towards our target.
It was always going to be a tough challenge raising that much money on Kickstarter and had we been funding a £4,000 short (which we are effectively making at present with no direct cash funding! Although with much appreciated continued support from our industry sponsors) we would have succeeded.
We are now looking at other funding ventures down more traditional film funding routes; more on this in the coming months.
Podcasts and Q&A
During and after our Kickstarter we started doing some podcasts and video sessions. As director I was finally forced to come out from behind the camera and perform along with some of best and brightest Stina crew members (thanks to student Oliver Hermann for doing a great job comparing these sessions; if CGI doesn’t work out for you there’s always the chat show circuit..) The results are available in audio or video format below:
These were the result of Q&A sessions posted on Facebook and Kickstarter (Particular thanks to James Devonshire here for all his excellent questions. They gave us a lot to talk about!)
We are now into our post-university-term “Stina crunch time” period, where we work full time like a professional studio and try and produce some finished work for the film. We had a very successful stand at the university end of year show last week, with 2 full size manikins scaring passes by in assorted costumes from the film, props from our shoot and lots of pics and videos to look at. (We won’t be posting pictures of this, as it has some reveals from our new short) We have a smaller crew than usual this year, but are luckily some way through the production process already, and are very excited about the results of the 4 minute film we’re working on. It contains a reveal of a new character and is looking like our best work to date! A few video render tests below (lots of thanks to Anatoliy Yudanov here for helping out with some tree animation):
Hopefully this piece will form the core of our next big fund raising push. More news on this in the coming months, but we are hoping to get it all wrapped up by September. (depending on team numbers etc.) More about this as it progresses..
It’s been a few months since the last blog but we’ve been very busy. Preparations for our kickstarter are well under way with the layout, background info and rewards now complete and we’re busy finalising our presentation video with the help of some talented film students (Directing myself wasn’t really working out, so we needed help). This was after a few abortive attempts at comedy, mostly involving me strapped into a harness and spinning from a tree in front of a make shift green screen with help from the most dedicated of students (plus some very bemused onlookers; for health and safety reasons we strung me up in a local park, well away from university property!)
We have also completed the first assemble of the whole film from on-set reference footage and audio, storyboards and music. It’s an old film cliche, but it really has been a case of making the film 3 times: once when you write it, once when you direct it and once when you edit it. There have been profound changes each time and each stage has seen a new creative challenge and added an extra layer of depth and clarity to the story telling. What we have now isn’t technically an “edit” yet, (it’s an assemble/first cut hybrid for those who know film jargon) as I’m just using the onset reference camera from the Mocap shoot, plus the hand held reference footage I shot on my iPhone whilst directing. This is all interspersed with storyboards and concept art. I’ve still been able to make a lot of editing decisions though, based on the rhythm of the performances and the wider story beats and dramatic arc. This will be refined into a much more polished edit once we add framed cameras in CG, but what we have now is a version of the film that’s watchable from start to finish, and has most of the character drama of the movie (if none of the visual flair, being mostly played out in a small room with leotards!)
The exciting thing is that it works, even at this stage. And it’s a very emotional ride. First reactions from those who have seen the assemble so far have been tears. I suspect this was provoked by the ending and Becky’s amazing ability to lose herself completely in the emotion of a scene. (whilst surrounded by a large busy crew, me barking orders, and supermarket style lighting; no mean feat!) Obviously no tears fell from my eyes, as I’m a bloke.
It’s going to take years yet to transform the 1 hour 50 cut into fully computer generated footage complete with character bodies and sets etc. and will require a lot more very talented artists to contribute to it, but it’s going to be a journey well worth the effort. From what we have already, I think the final product is going to be magical.
So, the Stina project completes it’s second trip to Siggraph, the biggest and most prestigious Computer Graphics and VFX conference in the world (first held in 1974) This year it was held in Vancouver in the impressive conference center located on the water front:
It was a great event and exciting for lots of different reasons (I may have had a few plays on the latest Occulus Rift). In attendance was myself, Alex and one of our students Oliver Herman, who was there as a Siggraph helper, but also undercover to help us promote the film.
Our first event was a presentation on how we’ve been using the film as a method of teaching our students. We were in-between two other presentations by other film projects being developed in the education sector. After some initial panic about absolutely nothing working with our powerpoint, we finally got it going and got ready for our biggest presentation to date:
The talk went very well by all accounts, with some great questions at the end, although the turnout was rather smaller than we’d hoped for (Around 35ish, we were competing with some much higher profile presentations that day unfortunately) We were very excited to see that Jorge had come to see us however, a head of one of our main sponsors SOLID ANGLE. It was also very interesting to see what other universities had been up to and we got some useful insights into improvements we could make in scheduling our student milestones.
Before our next presentation we had some time to check out the conference exhibition hall where Stina was making a debut on few of our sponsor’s stands:
At the VICON stand
At the FACEWARE stand
And playing out in the dallies reel over lunch.
We also had a chance to talk to the guys at FABRIC ENGINE who make a piece of software that may just save our bacon when we start transferring our entire film pipeline over to Maya (which, for those who don’t know we have had to do as the software we have made our film on up until now: Softimage, has now been canceled by the people that made it: Autodesk) This software will allow us to use the skills and methods we have developed in ICE, (a system of visual programming integrated into Softimage) within Maya, as the new release of FABRIC ENGINE will effectively allow ICE within Maya. This really did make our day, as up until that point the assumption was that all the skills we and the students had developed using ICE were now effectively redundant. We are also planning to open up this software to all the students outside the Stina project, as FABRIC ENGINE is now being adopted by commercial studios and will soon become an invaluable programming tool in a graduate’s arsenal of skills.
For our final presentation we were lucky enough to be sandwiched in-between some pretty high profile and exciting speakers, so our audience was guaranteed to be a lot larger:
Alex and I had 90 seconds to deliver our double act as the trailer played out on the screen. Luckily it was a lot more packed as we’d hoped, and we got a fantastic reception. It seems that the trailer is provoking a lot of very positive feedback from people who get see a lot of professional feature level work, which is very encouraging indeed. Also the trailer is now playing over on CGBROS, where we’ve had over 1000 likes so far (plus a lot of heated debate about whether the trailer is too obscure or not.. which i’m personally very happy about!)
The rest of the conference led to some great new discoveries for software solutions for the film: SPEEDTREE Looked ideal to help us solve our extensive foliage problem, allowing us to make dynamic forests and plants; and QUALOTH looked like it might be a very useful augmentation to our Marvelous Designer cloth pipeline, particularly as we now have to switch to Maya. (where it has an integrated plugin already developed.)
After a fairly modest amount of after show partying (thanks to VICON and FACEWARE here) and managing to catch up with our friends Lynette and George from 3d TOTAL , the conference came to an end. We were flattered by the praise and attention received by the trailer, which is testament to the hard work put in by all the students (now all listed under “The Team” tab above) Hopefully this, along with our recent CGBROS publicity has raised our profile to the level that will allow us to do some serious fundraising to take the film to it’s next big milestone: completing the previz. Watch this space..