Friday, April 29, 2016

Apple Motion: 3D for 2D

Late in 2015 I had the opportunity to produce 2 spots for Children’s Wish Foundation in Canada. The process behind the visuals was a bit unique, so I thought I’d detail it here for those interested.  

Motion is always a lot more verbose an application than people give it credit for.  I'd wager it can do most of what a lot of editors  are using After Effects for.  Honestly I think the biggest thing holding it back is some key 3rd party support by the same plug-in makers that bring a lot of the muscle to the table in Ae.  I’m looking at you, Trapcode and VideoCopilot!


The concepts behind the spots had been scripted and storyboarded during the Summer of 2015, with an eye to a handcrafted aesthetic.  The mandate was for simple colours and textures, and we planned to animate the characters using pretty standard Forward Kinematic puppet techniques.  Early on, we’d considered building backgrounds out of actual construction paper, but we realized we didn't have time for real-world construction. We also considered backgrounds built out of texture elements in Photoshop, but I was looking to add some extra pizazz to the design beyond a simple texture collage.   Whatever route we took, at the time we were given the go ahead- we only had 3 weeks to produce both 30 seconds spots from character design to final delivery; so we had to be very schedule conscious.

Character Designs by Ben Mazzotta

With Approved character designs as our touchstone, I started investigated adding 3D geometry and shading over flat textures.  As a Apple Motion user, the answer was obvious: mObject, the 3D text & object tool from one of the best 3rd party developer’s for Final Cut Pro X and Motion- MotionVFX.


mObject interface


Since it’s release in 2014, mObject has been updated several times with new features, UI enhancements, and overall performance improvements.  And while the 3D Text feature added to Final Cut Pro X 10.2 and Motion 5.2 last April has potentially "sherlocked" one aspect of mObject’s feature set- it’s ability to import and work with full, textured 3D models in either FCP X or Motion continues to make it a compelling purchase.

mObject can import .obj files with high polygon detail and high resolution texture maps.  But for my purposes I wouldn't be needing either of those things!  My plan was to build sets out of 3D objects, and then overlay the shadow information onto simple textures; bringing something extra to the backgrounds that at the same time wouldn't conflict with the approved character design.

One of the most important additions to mObject since it’s release has been the addition of Ambient Occlusion [AO].  This feature simulates the bounce lighting that occurs in the real world, and I remember distinctly when this became a “thing” in 3D animation working with Lightwave3D in the early 2000’s.  It was an immense render pig that created some very beautiful results even with simple models.  And now here we are with simulated Ambient Occlusion in Motion via mObject.  What a world!

I thought Ambient Occlusion would bring a nice feeling of real-world "miniature sets" to the backgrounds.

Ambient Occlusion off (L) and on (R)

For the purposes of this tutorial I’ll break down the first shot in the second of the two spots.  There’s a lot going on so it makes for a good case-study.

Sample storyboard.
Based on my storyboards, I constructed the basics of the room out of Primitives that come built into mObject.  This gave me a quick and simple way to roughly block out the room for the camera, without getting bogged down in a lot of the details.  Because the timeline was so short, it was important for me to be able to supply the animator with blocking so he could get drawing asap. In the end, I actually ended up using much of this basic architecture in the final scene, as these primitives are light-weight... and really, a wall is a wall!

Basic Geometry of hospital room created with Primitives in mObject

Next came finding scene specific furniture and decorations.  TurboSquid was my go-to source for all the models in the spot.  It’s not a free service, but if you’re getting paid for something like this, then a couple of bucks isn’t much to pay for good models considering the time you're saving.  I think the most I paid for any individual model was $30, with a total expenditure across the 2 spots of about $150.

Again, my goal here was finding high quality but simple models.  It almost would have been easier if I was wanting to do something MORE detailed as far as objects were concerned, since that’s what’s assumed most people are looking for.  “Simple” models can often mean low-polygon, and that can be seen in the final renders.  But in the end, TurboSquid’s selection was broad enough that I was able to find something for everything I needed.

One thing to note for anyone who’s planning on using downloadable models in mObject.  Even if the models are in the correct .obj format- it can be a real voodoo whether textures import properly; depending on what program the model was created in, and how it was converted.  Knowing the basics about a good 3D modelling program like the open-source Blender can be a real help here.  I didn’t need the textures, but it can be essential for cleaning up models and deleting elements you don’t need, since there’s no ability to alter model geometry within mObject itself.  After import into mObject, all the models were resurfaced with a plain white texture, with no reflectivity or specularity, since all I was really  looking for is form, and how it casts and receives shadows.

Here’s the scene with all of the final models in place in mObject.

Room with final models.


And here they are in the final shot.

Basic scene setup from Motion camera view.

At this point I provided final reference images to the animator, who could then move ahead while I worked on finalizing the scene.  The characters were created in Photoshop, and animated in After Effects, since that's the program he was most comfortable with.  Some animation for both spots was done in Motion- and technically there's no reason why it all couldn't have been.  Just timing.


Lighting mObject scenes can be done with standard lighting setups created within mObject, or using Motion’s own lights.  Important to note here that for these lights to work, they must be within the same Group as the mObject generator.  In this particular scene, I was simulating a time-lapse shot, so I needed to animate some key lights and overall ambience as the shot cycles from day to night and back.

Lighting setup with Ambient Occlusion turned on.

This leads us to the most complicated part of this project- the 2D texturing.  I’d built up a library of paper, wood, fabric, and subtle metal textures, and mocked up the final texture layout using my temp stills in Photoshop.  This way we could be sure that the background textures and colour pallet wouldn't clash with the character designs.

Originally, I had hoped to apply the textures  in mObject using a front projection map which, as the name implies, projects the textures from behind the camera onto the objects, so that the textures DON’T wrap around the models the way you'd normall want.  Unfortunately, projection maps aren’t amongst mObjects various surface wrap modes.  Drat!

This meant I had to create individual hold-out mattes for each of the object and textures in the scene.  Basically, this meant turning on and off the different objects, or specific surfaces on a given object, and using that as the Image Mask for the raw texture layers.  In a perfect world I could have actually used multiple mObject generators for the Image Mask (and this does work), but having duplicate mObject generators in your Project can cause real slowdowns… and crashing.  Oh, so much crashing….

So, once I was happy with the setup of the scene and any camera moves, I rendered still pngs or ProRes444s movies for each texture element.  This took a while for complex scenes with lots of textures… but hey!  Art!

If there was Camera movement in the shot, it was important to match the Z position of the  texture planes to match the objects properly in 3D space, so that textures wouldn’t “slide” relative to the holdouts. Below is a sample of the textures with the Image Masks applied.  You'll note I ended adding perspective to the floor texture; it's perspective made it look too weird when the texture was completely flat. 


Flat textures with Image Mask holdouts from mObject.

After the Image Mask renders, the final pass from mObject was the lighting pass with camera movement and all models turned on, so that you can get all the interactive shadows and Ambient Occlusion interactions in the scene.

This image or movie would then be reimported and overlaid over the texture layers using the Multiply transfer mode, thereby applying the 3 Dimensional shading to all the flat 3D textures.  This was the toughest shot in the second spot, so in the interest of time the push in was achieved with a simple scale shift in Final Cut.  If we'd had time, an actual 3D camera move like you see in other shots would have been my preference.


Shadow pass overlaid onto flat texture layers.

Characters were then added in, and if necessary some of the texture hold outs were used as Image Masks to put characters “between” layers of the background, for example in this scene behind the bed or out in the hallway.


Final Composite with Characters.



In this first shot, simulating a time-lapse, there's actually no "animation" per se, but lots and lots of still drawings cycled at a regular interval.  A high speed cloud or stars layer was added outside the window, and elements like the blinds, bed cover, and TV were animated to add some more movement to the room. The fast moving shadows on the floor and wall help to sell the effect too.  Finally,  bit of grading with Color Finale to show the shift to night and there you have it.  

We had fooled around with different effects to try and enhance the time-lapse effect, but the characters just ended up looking smeary and out of line with the overall aesthetic.  In the end, turning on Frame Blend in Final Cut Pro X's retime window with a very minor speed change gave us just a hint of a fade as the characters appeared, disappeared and moved around in the shot.

Each spot ending up being about 10 days of total production.  Here's the second one, Izaak Gets a Gaming System, in it's entirety.





I use mObject a lot for corporate work, putting clients websites on 3D models of Phones, tablets and computers.  For that alone it's made it's money back for me.  This was a fun exercise in blending 3D and 2D in an interesting way, and I honestly had no idea what the results would look like when I started.  Surprises are great... as long as the client likes them too!

Tuesday, October 14, 2014

ROLES REDUX: PART1

My most popular article was the one I wrote last year on ROLES.  I’ve received lots of great feedback on it and a lot of great questions as well.

So I decided to take that feedback and roll it into two followup pieces looking at the two places I think Roles needs to expand to meet their full potential- and prove once and for all that a “trackless” timeline needn't give up functionality for the fluidity it provides.

In this first article, I’ll look at what Roles are, how they’re applied, and different ways the process could be improved for efficiency.

For the purposes of illustration, I’ll be using screen grabs from NOW YOU KNOW, a 39x7min Kids TV series that I edited on X this year.  Thanks to Little Engine Productions for permission.


ROLES AS IS


Roles are accessed via the Modify>Assign Roles menu.  By default, there are 2 video Roles [Video and Titles], and three Audio Roles [Dialogue, Music, and Effects].



Right away FCP X does something clever- it automatically assigns Roles to all imported media, and it’s actually right a surprising amount of the time.  Any audio that comes in attached to video is assumed to be Dialogue, which for most shoots it often would be.  Pure audio tracks are tagged as either Dialogue, Music or Effects.  In my own experience over the past 3 years using X, it’s usually correct about 3/4 of the time.  I’m not sure how Final Cut is making this call.  Channel Orientation?  Length?  Some hidden metadata that I’m unaware of?  Regardless, it’s great that we’re not just given a blank slate and have to tag everything from scratch.  And in many short form projects, these three Audio Roles may be all you need to get the job done.

If you want to see what Roles Final Cut has assigned for an individual clip, you can find it in the Inspector window Info tab



You can also see Roles for all clips in an existing Project by changing the Timeline appearance settings to “Show Roles”.



And you can see all the Roles currently used in a Project via the Timeline Index.  Here you can highlight specific Roles.



But the default Roles are just the beginning.  Modify>Edit Roles allows you to Create not only new Video and Audio Roles, but Sub-Roles for existing categories.  As I mentioned, Dialogue, Music and Effects can probably get you by in a simple project; but in longer or more complex timelines more granularity will be needed.  So besides other obvious categories like Ambience and Voice Over; Sub-Roles for Dialogue by character or different categories of Sound Effects can help to create more clean separation of elements that will have your Audio Engineer professing his love for you (or maybe just swearing at you less for once). 

Here are the list of Role I created for the first Season of NOW YOU KNOW.





APPLYING ROLES


Currently, applying Roles to a clip can be done in a couple of ways.

First is via the Event Browser.  You can change the Role assignment of a clip either from the Modify Menu, the Info tab of the Inspector, or via Command keys- CTRL+OPT+ M [Music], E [Effects], or D [Dialogue].  If you have multiple clips selected in the Event Browser, you can apply a Role change to all the selected clips at once, like you can with most attribute changes.  It’s always better to be able to do this before you start editing- assigning a Role in the Event Browser once means you don’t have to apply it to each individual instance of the clip after you’ve edited it into your Timeline.  The problem is that currently you can only apply ONE ROLE to a clip in the Event Browser.  Meaning if your camera has multichannel audio or you've created Sync Clips from second-source sound, and you want to assign multiple Roles or Sub-Roles to individual Audio Components, there’s no way to do it in the Event Browser. 

You CAN get around this limitation, by taking each clip and using the “Open in Timeline” function, which opens all parts of a clip into it’s own mini-Timeline.  From here you can expand your Audio Components just like in a Project, and assign audio Roles to each channel [thanks to Ben Balser for this tip], but doing this operation for each clip in even a medium-sized project, let alone a large documentary or feature film with 1000’s of clips would be insane.

"Open in Timeline" of a clip, with a single Audio Component selected.

But if you have lots of Audio Components and sub-Roles in your Project, you may be better off to just edit untagged clips into your Timeline and assign them later.  I say this because once you have your Project laid out with all the Audio Components expanded, you can multi-select lots of Audio Components across lots of clips that share the same Role or sub-Role and tag them en mass.  But unless you know the layout of the Audio Components by heart, or have taken the time to name your Audio Components before editing (which I do suggest), this can take a bit of diligent work.

Timeline with multiple components selected to add common Role: "Baboo"


The goal of Roles is to be able to attach and pass on Audio metadata in a way that isn’t slave to the static nature of Tracks.  But in it’s current incarnation, assigning Roles has several stumbling blocks.

First, the inability to effectively apply Roles to individual Audio Components either in the Ingest or Event Browser stages.

Second, Command keys cannot be assigned to custom Roles or Sub-Roles- meaning you have to apply them each time using the mouse via the Menu>Modify>Assign Roles or Info tab Roles drop down menu.

Third, like all metadata, changes made to the Role of a clip in the Event Browser do not “filter down” to clips already edited into a Project.  Neither is there an option for changes you make to a clip in a Project to filter to any other instances of that clip in the Project, or back up to it’s Master Clip in the Library.

Finally, a visual representation of a Project that organizes Roles for easy identification.  This last one is so important that it will be the subject of the second part of this article.

ROLES DONE RIGHT


Let’s take a step back from all the holes in the current process to look at a ray of sunshine in just how great this can all work. It should come as no surprise that if something good is going on with Metadata, it probably has something to do with Philip and Greg at Intelligent Assistance.

People know Sync-N-Link X is an amazing tool for syncing dailies and automatically creating either FCP X Sync Clips or Multicam Clips from on-set timecode.  But the extra awesome part you may not know is that if your Audio Recordist has used equipment that utilizes iXML- then the names the Recordist gives those channels on set will be ported over to Final Cut and assigned as BOTH your Audio Component names AND Role assignments.  At present, FCPx does not itself support iXML, so this feature is exclusive to Intelligent Assistance’s software.

Sync-N-Link X Roles Interface


This means that without a lick of work, the editor has Role tagging for all Dialogue from a shoot done before they even start work.  Between this and Final Cut’s automatic tagging for Music and  Effects elements, the Editor has much of his Audio prep already done.

That’s just awesome and something I’m very much hoping to take advantage of on NOW YOU KNOW’s Second Season.


STREAMLINING ROLES


But for those of us not lucky enough to get on-set Roles assignments, there needs to be a more efficient way to assign Roles to large volume of clips BEFORE you edit them into your sequence.  Let’s look at a couple of ways this could be done.

First, we need the ability to assign more complex Roles to multiple clips at once at the logging stage.  Initially I wondered if this could be done at the Import Window, but I think much like Keywording, Batch Renaming, Syncing, and most other pre-editorial tasks; this would be better accomplished in the Event Browser.  The key improvement need to be-

  1. The ability to apply Roles to individual Audio Components
  2. The ability to apply multiple Roles to several clips with the same Audio Component configuration.
  3. Ability to apply custom Roles faster and more efficiently.

Here are a couple of ways this could be instituted.  None of these are brand new ideas, but leverage how we already apply other kids of metadata within FCP X.

If you know that clips have the same Audio Component configuration, you can change the name of individual Audio Components for multiple selected clips in the Audio tab of the inspector.  It would be great if we could assign audio Roles in the same way.

Mockup of selecting and assigning Role to a single Audio Component


Aiding in this would be a “Keyword”-style Roles HUD.  Allowing you to assign Roles and Sub-Roles to command-keys for quick assignment to selected items, either in the Event Browser or Project Timeline.

HUD for Assigning Custom Role Shortcuts


Another way to do this would be to assign Roles to a single clip, and then have the ability to copy/paste those Role assignments to other clips.  Right now Paste Attributes is a function exclusive to clips in a Project, so that would be new functionality in the Event Browser.


Mockup of Paste Attributes including Roles in Event Browser.

With 10.1, Final Cut Pro went from a dual Event/Project structure to the new combined Library model.  We’ve already seen this combined database pay off in new features like Used/Unused Event media indicators, and Roles would definitely benefit from increased communication between Project and Events.

This would mean that changes made to Roles in a master clip in the Event Browser could automatically filter down to all instances of that clip in edits already in Progress.  Or conversely, being able to make changes to clips in an Project, and upstream those property changes back to the media in the Event Browser, so that any further material from that clip that used will be properly tagged.

I could also see a big advantage to having a unified Roles interface, which could allow an editor to change already assigned Role names globally in a Library, say if a character name is incorrect and you need to change DIALOGUE JIM to DIALOGUE FRED, or if part way thru an edit Audio Post requests a specific naming structure.  Little things like this will make working with Roles easier for everyone.

Assigning Roles may seem like a lot of work, but like with many things in FCP X, up-front work pays off in spades over the course of the edit- especially if it goes on for months as in TV episodic, documentary or feature work.


NEXT


In the second article on Roles, I’ll revisit the Roles-based Project organization.  Looking at how a complicated edit, with all different kinds of audio, would be improved by better visual separation.



Monday, September 29, 2014

Final Cut Pro X: What’s Next [10.1.4]

EDIT- CHART UPDATED NOVEMBER 25th 

In my last post I laid out the past maintenance release dates to get a sense of when FCP X 10.1.3 might be released.  Dates ranged from 28-70 days after a feature release, with an average of 50 days. Apple was actually pretty accommodating with that one- releasing on August 19th, 53 days after 10.1.2.  That's remarkably close to the average interval time.

So why don't we do it again, shall we?

First off we have to make the assumption that the next release WILL be a feature release.  That's may not necessarily be the case, since FCP X has released multiple maintenance releases in a row: 10.0.4 to 10.0.5, and three maintenance releases in a row with 10.0.7, 10.0.8, and 10.0.9.

But let's be optimistic, shall we?

By all accounts Yosemite is due to come out in October, and if there are any technologies that FCP X can lean on then we could see those in an update this year.  iCloud Drive could be leveraged to improve Collaborative Workflows, and Extensions could be an interesting alternative to traditional plug-ins, but that's all supposition at this point.  However, there have been reports from 9to5mac about fall updates,

"Sources also say that Apple is preparing updates to iMovie and Final Cut Pro with improved tools for editing high-resolution, 4K footage."

On the FCP X side, at least, I find this report interesting, since it's been Retina ready since the 10.0.5 update with the first Retina MacBookPro, and it already a very capable 4K editing platform.  So, I'm honestly stumped at what additional 4K improvements could be made outside of just pure performance, or perhaps h.265 support.

Finally, as we all know Yosemite is giving OSX a bit of a visual overhaul, flattening the interface and bringing it more in line with the changes made in iOS7 last year.  Whether, and on what schedule these UI changes could make their way over to Apple's ProApps remains to be seen.

Personally, I'm still hoping upcoming updates see a focus on the use of Roles in both Project organization and enhancing FCP X's audio toolset.

Anyways, on to the numbers!

Apple has released 5 feature updates since Final Cut Pro X was released in June 2011.  Here they are with the day intervals since the previous launch.

10.0.0-10.0.1 91 DAYS
10.0.2-10.0.3 76 DAYS
10.0.5-10.0.6 134 DAYS
10.0.9-10.1.0 148 DAYS
10.1.1-10.1.2 162 DAYS

Feature updates have generally been getting further apart since the heady, early days of Final Cut Pro X development.  Perhaps that means the "urgency" around getting updates out the door has died down, since many of the early Final Cut Pro X functional stumbling blocks have now been addressed.  Or it could mean that the upcoming functional improvements are bigger than those released in early days. Especially if the FCP X development team are having to go through the process of rethinking how "feature-x" works in the new paradigm. Either way, there's a fairly broad range of dates to look at here. If we average these intervals out, the mean is 122 days, or about 4 months.

Once again, I've mapped each of the update intervals on the graphic, showing how they land on the calendar if we take the release of 10.1.3 as the starting point.


click to enlarge
*updated November 25th



So as you can see, the most optimistic update schedule for 10.1.4 would be after the release of Yosemite [marked on the chart for late October] in early November, but the truth is we haven't had an update that quick a while. The average is more likely, and more interesting, since it would put it almost on the same date at 10.1 was released last year, December 19.  I'm not seeing that as a prediction– that's just where the average falls.

If we're being pessimistic [or is that realistic?], then we might not see another feature update all this year– if 10.1.4 takes as long as 10.1.2, then it will be almost February 2015 before it comes along.

Or as I said, 10.1.4 could be a second maintenance for this year, and reset the clock again for what would most probably be an early 2015 10.1.5 future update.

Thursday, August 07, 2014

Final Cut Pro X: What’s Next [10.1.3]

We’re about a month out from the release of FCP X 10.1.2, so I thought I’d just throw out some quick thoughts on the what and when of the release of the next update 10.1.3.

Based on any reasonable expectations, and 3 years worth of update stats, 10.1.3 will be a fairly minor, but much needed maintenance update.  As with any feature release, FCP X 10.1.2 brought new bugs that need to be sorted out, and a relatively quick turnaround maintenance release has been standard practice after every feature update since Final Cut Pro X launched in June 2011.

Those looking for another big feature release should keep their expectations in check until later this year, after the release of OSX Yosemite sometime around late October.  At some point after that, we could see an other feature update that could include a visual overhaul to bring Final Cut in line with the flattened look of OSX 10.10, or new features that Yosemite enables like Extensions or iCloud Drive support, or just general improvements we're all waiting for that aren't OSX related at all.

But let’s not make a mountain out of a molehill here.

4 for 4, FCP X has had a maintenance update following a feature releases.  The timing for these is as follows

10.0.1-10.0.2   57 DAYS
10.0.3-10.0.4   70 DAYS
10.0.6-10.0.7   44 DAYS
10.1.0-10.1.1   28 DAYS

That gives us a nice range data to work with, which led me to throw together this quick chart [updated August 18th to add average info & update "TODAY" date]-

click to enlarge



So basically, applying the intervals for the 4 post-feature maintenance updates so far, we have a range of 28-70 days after a feature release for the next maintenance update.

Meaning that with the June 27th release of 10.1.2, we could expect 10.1.3 to come anywhere between July 25th [28 days] and September 5th [70 days].  We’re already 41 days past 10.1.2 [now 52 days as of editing August 18th],  so hopefully we’ll be seeing this update within a month of this writing.

Up to relatively recently FCP X updates were pretty consistently launched on Tuesdays, but that seems to have been falling by the wayside recently- with updates scattered across the whole week.  It was nice while it lasted, but the “Rule of Tuesdays” seems to be a thing of the past.


-MRM

Wednesday, May 14, 2014

CLIP LINKS

In Final Cut Pro X, any audio, video, or graphic element not in the Primary Storyline must be connected to Primary Storyline as a Connected Clip. This is great for moving around shots with attached titles, b-roll, sound effects, or green-screen backgrounds- no lassoing of elements like in Legacy FCP days.  Just pick the clip you want, and anything you’ve chosen to associate with that clip will follow along.

I would like to see the FCP X push the association between Primary Storyline clips and their Connected Clip siblings even further with a proposed feature I'll dub Clip Linking.

While a Primary Storyline’s position in the Project timeline affects Connected clips, any trimming you do to the Primary Storyline clip affects only itself.  In our example below, we’ll look at a Primary Storyline Clip with 2 connected clips: a Title clip ending at the same time as the Primary Storyline clip, and an audio clip that overhangs into the next shot.  



Now, if you shorten your Primary Storyline clip by -1:00, the Connected Clips duration won’t change, leaving your title and audio clip hanging over the following shot.  The next step would be to manually shorten each of your connected elements to match the timing change to the Primary Storyline clip.



This is not what we're looking for.  So. what are our options?

Unlike in some other NLE’s, in FCP X you cannot currently select and trim multiple clips at the same time.  And even if you could, you'd have to manually select them all.  What a pain!

In FCP X there is a function to select multiple clips and use CTRL+D to alter their length. This can be either as an absolute value [2:00], or as a differential [-24].  This works well, but  requires that you manual select all the clips you want to alter, AND that you know exactly how much you want to trim.  Still problematic.


Clip Linking would suppose that you set up associations between Primary Storyline clips and their Connected counterparts once, and then [like many tasks in FCP X] reap the benefits on repeated changes.

To do this, you would select the connected clips you want to add the behaviour to. 



A contextual menu would pop up asking you how you'd like the clips linked.  While we'll just look at Link Out, there could be options for Link In and Link All, depending on how you want the Connected Clips to behave.

Now let's look at our -1:00 trim edit again.



While only selecting and trimming the Primary Storyline clip, FCP X now assumes our Connect clip's edit Out point will mirror any trim, roll, or extend/shorten of the Primary Storyline clip.

To make isolated changes to the connected elements; say if you wanted to extend the bleed of the audio clip over the next shot- trimming that one clip doesn't filter back to the Primary, or any other connected clips.

Again, while setting up these Links takes a second for each link, I think it would save tones of clicking over the course of an edit.

What do you think?

Tuesday, May 13, 2014

ROLES

When Final Cut Pro X was first released in June of 2011, one of the biggest changes in philosophy was the abandonment of traditional Tracks.  Today we’re going to take a closer look at that decision; why it was made, what benefits it poses, and how further development of FCP X may quell some of that criticism.


TRACKS ARE A MEANS TO AN END, AND NOT AN END UNTO THEMSELVES

Track are, at their very core, an organizational tool.  A tool for organizing audio and video information visually in a way that makes locating it easier in complex projects.  In the case of audio, tracks are also used to group audio elements for the purpose of mixing and effects.

A Track is really just a folder of audio or video, laid out across a sequence of time.

But a Track is also a “dumb” organizational element.  Dialogue is on Tracks 1 and 2 simply because that’s where you the editor choose to put it.  But you might only need 1 Dialogue Track for a simple video, or you might need 10 Dialogue Tracks for a large scene involving lots of mics and many characters.  It’s completely arbitrary. 10 different editors are going to organize their audio/video Tracks 10 different ways.

Organization in Tracks is especially important in a collaborative workflow; where audio is passed off from editorial to post audio.  The editor always needs to supply support documentation, outlining what kind of audio is on what tracks.

Tracks in and of themselves aren’t important, they are simply a means to an end. The ability to organize information both in the timeline, the ability to group elements for common effects, and for the purpose of passing that work on cleanly to other departments- THAT’S the goal.  


TRACKS WON’T WORK IN X

The two big changes in the core philosophy of editing in FCP X were Connected Clips, and the Magnetic Timeline.   The key idea here was to set up a relationship-based methodology to content.  Elements in the Primary Storyline are considered the backbone of the cut, and any additional audio or video that is added is understood to be subservient to that “Primary” content.  In the last year of working with FCP X, I find this works remarkably well, especially for audio.  If you connect the SFX of a gunshot to a video clip in the Primary Storyline; once that relationship is established, it’s fixed.  Regardless of whether you trim, roll, slip, or even change the order of shots in the timeline.  

Where things start to get tricky, is when in the process of editing the Primary Storyline, connected clips come into conflict.  In legacy Final Cut Pro, if you trimmed a clip, and then closed the gap, you would either get a warning that the edit wasn't allowed due to a clip conflict, or overlapping elements would just be overwritten.  This is obviously not what you want, and it meant that the editor could be forever jostling audio elements from one track to another as the edit was refined, in order to resolve audio or video clip conflicts and make sure that elements overlapped as intended, all while trying to maintain the organizational structure of the tracks.

In Final Cut Pro X, clip conflict is resolved by one audio/video element “bumping” the other up or down; the integrity of both elements is maintained regardless of what goes on in the Primary Storyline.  The compromise, obviously, is that the vertical hierarchy of Tracks becomes impossible to maintain.  If TRACK 1&2 are DIALOGUE, and TRACK 3&4 are MUSIC- if 3 Dialogue elements overlap then a new “Track” [FCP X refers to them as “Lanes”] is added, and any structure based on tracks becomes moot.


SEA OF GREEN

The biggest detriments to the visual organization of audio in the FCP X timeline,  is that all audio only comes in two colors- blue for audio components, and green for everything else.  Secondly, all audio elements naturally gravitate towards the Primary Storyline.  In a simple projects with some interview audio, a music track, and perhaps a few sound effects, this isn’t so bad.  But as you get into more complicated projects with dozens of lanes of audio with no obvious separation or way to distinguish your elements visually; the audio in the timeline becomes a sea of anonymous green clips.


Simple FCP X Project with audio elements uniform colour and intermixed.  What's what here?
CLICK TO ENLARGE

ENTER: THE ROLE

In September of 2011, Apple released their first update to Final Cut Pro X: 10.0.1.  The update brought with it a host of improvements including a brand new feature called Roles.

Roles was a new form of metadata added to media in an Event, explicitly tagging clips with an audio or video “media type”.  Pre-defined Video Roles included b-roll and titles, while Audio Roles included dialogue, music, and effects.  But Roles are completely customizable, with the ability to create new Roles (Ambience, Voice Over) or Sub Roles (Dialogue- Nick, Dialogue- Sally).

Roles imprints information onto media that in other NLEs must simply be assumed because of the track it sits on. 

As well, the addition of a Role tab in the Timeline Index pane, allowed you to selectively turn on and off Roles, or highlight all media of a certain Role, in the project timeline.  Roles also allowed the user to export stems of Roles in multi-channel audio exports.  Finally, via the 3rd party program X2Pro, Roles could be exported via AAF for use in ProTools.  The added benefit being that Roles name information was automatically transferred to the DAW, eliminating the need for track reference sheets.

As intriguing and seemingly powerful Roles are, it still didn’t solve a key problem in FCP X, a visual organization to material in the timeline.


BUT WAIT!  APPLE ALREADY SOLVED THIS PROBLEM

When FCP X was first released, I remember reading a particularly maddening “review” of the software from a site I shan’t name.  The author went on about how he spent nearly an hour trying to figure out how to make Bins in FCP X, little realizing (and of course, not doing the research before writing his stupid article) that a new methodology had been implemented for organizing media: Keywords.

Keywords imprinted metadata onto media in an Event, allowing for an active organization of elements based on tagging media with single or multiple Keywords.  It’s a very fluid and extremely powerful way to organize media, especially with the addition of Smart Collections, which allow you to set up a “Smart Bin” which is made of any media (or selections of media) which satisfy a user-defined set of not only Keywords, but any combination of metadata criteria.

This is opposed to other NLEs, where you create a static Bin [really just a folder] which you drop clips into.  Lets say you create a bin called “canoes”.  The material that’s in it is about canoes only because you put it in that Bin, and not because the media explicitly demands it.  Bins are dumb.  Keyword Collections are smart.  Smart Collections are REALLY smart.

So if we take what I proposed earlier about a Track being essentially “Bin across time”, then we start to see a really interesting correlation:

Bins are to Keywords, as Tracks are to Roles.

So if we take this concept to it’s literal conclusion, then what the Project Timeline in FCP X is really missing are “Role Collections”.


ROLE COLLECTIONS

With Roles, organizational information has already been attached to the media, that information just needs to be used to sort the media in the Project Timeline.  By grouping Role elements together, and giving them a colour coding system, gives the timeline focus.

So if we imagine an empty timeline, and we add a new clip... lets say Voice Over- automatically a Voice Over Role Collection is created.  As you add more media, more Role Collections are added [Music, Dialogue, Sound Effect, Ambience].  And if you add media for which there already is a Collection, it’s automatically added to the appropriate place.

Same Project from above with Roles now separated vertically, and colour coding to allow easy identification
of video [Titles,
Graphics, Video] and audio [Voice Over, Music, and Sound Effects] elements.
CLICK TO ENLARGE


Much cleaner, yes?

A couple of notes about this proposed organizational model.

Some have tried to use Compound Clips in FCP X as a way to faux Tracks.  The problem is that a Compound Clip breaks individual clips connections to the Primary Storyline- this is obviously not what you want if the cut is still evolving.  With Role Collections, these links would be maintained.

Tracks are always one audio layer deep.  While Role Collections could have as many stacked Lanes as required based on the number of overlapping media clips.  So, for example, a “Dialogue” Role Collection could be 1, 2, or 10 Lanes high; depending on the number of overlapping clips.  However, if you need more organizational granularity, you can (via SubRoles) create as many sub Roles for Dialogue as you see fit, and in the end each one has it's own Lane.

Another nice addition here would be expanded Role controls in the Timeline Index, allowing the user to alter the order of Roles [much like how you can re-order your camera angles in a Multi-cam clip].  Additional controls could be for selectable Role colour, and icons.

With complex Timelines, vertical space is always at a premium.  The ability to collapse and expand Roll Collections into single-Lanes [much like LogicPro X’s Stack Tracks] would also be a big help.  Another idea would be to “solo” a Roll [like you can solo layers in Motion], this would allow the editor to focus on one specific Role element easily, with the option to hear it on its own, OR in the context of the entire mix.


ADDITIONAL BENEFITS

With Role Collections, we now have an organizing container which can now be assigned for more complex Mixing [should Apple decide to bring Bus-based mixing to Final Cut Pro X].  At the very least, it would allow for audio effects to be added to a whole group of clips via their Role Collection, and global volume adjustments to be made.

Seeing what a boon metadata-based organization has been in the Event Browser makes me hopeful that a more extensive implementation of Roles will result in a similar advances in timeline workflow.

Fingers crossed!