This is my favourite way to retime footage from 30 fps to 25 fps in high quality with After Effects. This method does not use any 3rd party plugins like Twixtor. Off we go:
Import your final film. It should have been edited at 30 (or 29.97 if you converted to cineform and maybe in some other cases too) fps. Go to the footage in the project window and choose Interpret footage. Set the frame rate to your target frame rate (in my case 25 fps).
Create a new comp based on this footage. It should be set to 25 fps as well. If you take a look then you will see that what is happening now is that we play the footage back with a lower speed. To correct this, apply the TimeWarp effect. You could use the speed parameter, but I prefer to use to keyframe the source frame parameter. Change your project settings to display frames instead of a timecode:
Note the length of your footage in frames. Now calculate the new end point: <number of source frames> / <original framerate> * <target framerate>. Round up if this results in a fraction instead of an integer number. Go to the frame that you calculated (which should be exactly at 5/6th of the total length of your footage/timeline). Add a keyframe for the Source Frame parameter here and set it to the last frame (the number of source frames). Go to the beginning of your comp and keyframe to source frame 0. Set the method that Timewarp uses to Frame Mix (for frame blending) or Pixel Motion (for frame generation). If you scroll through it now (can be very slow with pixel motion) you will see the retimed footage.
If you don’t have any hard cuts, you’re done now and can render the result. If you have hard cuts, you will want to protect them.
You can go to all hard cuts, keyframe it to one exact frame just before the cut and keyframe it to exactly one frame later after the cut. But this is tedious, so here is a quicker way:
Find all hard cuts and place layer markers there (or let the very nice Magnum script do the work for you). Then add this expression to the source frame parameter:
n = 0;
t = 0;
if (marker.numKeys > 0)
{
n = marker.nearestKey(time).index;
}
if (n > 0)
{
t = (time +thisComp.frameDuration) - marker.key(n).time;
if (Math.abs(t) <= thisComp.frameDuration)
Math.round(effect("Timewarp")(4))
else
effect("Timewarp")(4)
}
else
effect("Timewarp")(4)
It will keep the animated value unless there is a layer marker present at the current or the next frame in which case it will round the time to make sure that no blending across the hard cut occurs.
In a future post I will show a way to do the same thing in twixtor which handles the hard cuts differently.
If you get messed up frames in between you can either switch back to frame blending for the few frames that require it or use brainstorm to optimize the optical flow analysis parameters of Timewarp.
As the 5D mark II can still only record in 30 fps because there is no firmware update to record in more useful 25 fps (for pal DVD/TV) or 24 fps (for cinema projection), the need to retime footage from 30 fps to 25 fps definitely still exists. There are many methods to do this, but many are not as high quality as one would like. In this blog post I will explain what methods exists for retiming exist, what they look like and in the next follow up post I will show how to do high quality retiming in After Effects.
First, an overview of the problem: Your camera records in 30 frames per second (fps). You get the stuff into your editing software of choice, set the timeline to 30 fps and edit you film. You export your final film and up to here, everything is fine. But now you need a Pal DVD (to be played in Europe) or a film copy to screen in cinemas (lucky bastard). So now you have the problem of how to go from 30 fps to 25 or 24 fps.
If you had shot a silent film, you could simply play it back a little slower. A hypothetical 5 second film at 30 fps could be played at 25 fps and now be 6 seconds long. Everything would move a little slower, but maybe that could be cool. Unfortunately, if you try this with a film that uses audio, the sound would have to be slowed down as well and thus be significantly lower in pitch.
Since most films nowadays want to use sound, this leaves us with 3 options that slow down the images while maintaining the same duration and thus keep the audio unchanged.
All following samples will use PALs 25 fps, the same methods can of course be used to go to 24 fps, just the numbers are slightly different. The sample videos linked to below are all the same clip of some location scouting footage that I edited specifically so that one could study the effects of different retiming methods. Here is the original clip at 30 fps (29.97 to be exact because I converted it to cineform via NeoScene)
Frame dropping
The first option is frame dropping. It simply removes every 6th frame. Our 5 second clip from before would still be 5 seconds long, but of the original 150 frames, 25 would have been dropped. Here is a little graphic I made to illustrate what happens.In this example we see 6 frames at 30 fps that need to become 5 frames at 25 fps (so that at both frame rates, the clip would play in exactly 1/5th of a second or 0.2 seconds).
Frame dropping leads to a strange jerky sensation that can be seen here. Some people are more sensitive to this than others:
Advantages of this method
It works consistently
It renders very quickly
No problems with hard cuts
Disadvantages of this method
Looks very strange
Frame blending
Frame blending always blends two frames together. So the first frame A of our new 25 fps sequence is a blending of frame A and B in the original sequence; the next frame is constructed from B and C and so forth (the exact details how and which frames are blended can vary). Here is an illustration of what happens:
In this graphics this looks pretty bizarre, but since most individual frames of a film look very much alike, this usually works pretty well. It gets more noticeable when there are quick movements, especially when the scene was filmed with a short shutter time and the objects have little motion blur. Here is a frame where the blending can be seen very well:
The good thing about frame blending is that it looks relatively pleasant most of the time and it works in any case. One of the problems with it is that it will soften your hard cuts. Here is an example where what used to be a hard cut from this frame
To this one
Gets turned into this blended frame
Here is the sample clip with frame blending.
In after effects one can protect hard cuts (see this post for more information on how to do this). The resulting sample can be seen here:
Advantages of frame blending
It is more pleasant to watch than frame dropping
It renders quickly
Most of the time it looks ok
It works consistently
Disadvantages
Extra care needs to be taken to protect hard cuts
Noticeable "double exposure" effect
Frame generation
Frame generation is a process that goes by a myriad of names (aka Optical Flow Analysis, Pixel Motion, Motion Vectors, ...). The principle is always the same though: a clever algorithm tries to understand where similar groups of pixels move in the image and calculates in between frames by moving the pixel groups further than they used to move, an extra 1/6th of the original distance to be precise (in this particular case). The upside is that if it works, the image looks perfect and no matter how much movement is in a frame, there will never be the "double exposure" effect seen with frame blending. The downside is that even though these algorithms have gotten very good, sometimes they get badly messed up and produce something like this:
This happens basically at every hard cut, so you have to protect your hard cuts against this problem (see this post for more information on how to do this). However, it can even happen when there are no cuts, e.g. when the lighting changes dramatically (see this blog post for an image where that happened). If that happens, tweaking the parameters can often help, but sometimes you will have to resort to frame blending for a couple of frames where frame generation simply does not work.
Here is an example with pixel motion where hard cuts are unprotected (this sample uses twixtor instead of After Effects, but the results in this case are very much alike):
And here with hard cuts are protected (native After Effects again):
See the bottom of this post for some more clips retimed with the Twixtor plugin.
Advantages
If it works, the quality is very good
Disadvantages
Extra care needs to be taken to protect hard cuts
Can sometimes produce messed up frames, output has to be carefully watched and problems eliminated
Longer rendertimes
My advice would be to go with frame blending if are in a hurry, and use frame generation if you have time to check and maybe fix the output and want very high quality. In both cases you should remember to protect your hard cuts – something that to my knowledge can only be done in compositing programs like After Effects. If you have the option to use Twixtor, I can highly recommend it as it seems both more stable and significantly faster than TimeWarp. Details on how to use After Effects for retiming can be found in my next blog post.
For the sake of completeness, here are 3 samples of the result when using Twixtor with frame blending and frame generation:
Two weeks ago, a few friends from my film school took the Canon 5D Mark II to the Ostsee coast to test whether it would be suitable for the short film they want to shoot in August. They gave me the material they shot so I could try various retiming methods to see if it was possible to get a decent looking 25p copy of the material. (Update: I have since shot some more tests, read about them here. Update Nr 2: I have written two extensive blog posts on how retiming works and how to do it in After Effects. Find them here.)
Most technically minded filmmakers will be aware of why the 5D is such a hot topic right now, but for those who aren't, here is a short summary: The 5D is a DSLR Camera that can record video in 1080 30p (HDTV). Why does this matter?
I participated in the 48 hour film making session last weekend at my filmschool. I shot a 2 minute B-zombie homage with two classmates (Timo and Alexi) and a few folks who played the zombies. The film is online at vimeo, so I embedded it here in SD quality (but you can see it in HD directly on vimeo).
As I wrote above, it is a short film that was entirely done in 48 hours (that includes coming up with the idea, writing/planning, shooting, editing, effects), so it is pretty short and has some rough edges, but I'm quite happy with the result.
We shot the whole thing with my new Panasonic Lumix LX-3 Photocamera in HD Movie mode. The LX 3 lacks some important controls (shutter time, full manual control in movie mode, no focusing and zooming while shooting) but it did a pretty decent job nonetheless. I was especially surprised by the quality of the sound (of course it is not nearly good enough for anything serious, but it is interesting how usable it still is). We shot everything with available light only, and the camera handled it rather well (the 2.0 max aperture at the wide end is a great thing - go Leica!). The sound you hear is all from the camera, except for the music and a few Foley sounds.
The noise is pretty strong and we softened it a bit by processing the entire film in after effects with a slight glow (by adding a blurred version of the edit on top of the original and adding it with screen mode and about 40% transparency), but for render-time reasons we did not use any degraining solutions. If anyone is interested in the original material send me an e-mail and I will upload some of the original material to vimeo.
The music we used is Cellule from the Band Silence who were kind enough to put their song on Jamendo under a very liberal license (Thanks for the nice song)
We also used some sounds from freesound.org (see the credits for details)
The film is released under a creative commons license:
As some of the people who read this blog may know, I study directing at the filmschool filmArche in Berlin. Over the last year I have been asked to colour correct a number of movies since this is one of my favourite steps in filmmaking and I think I have become quite decent at it by now. I love to see my fellow filmmakers faces when I show them how much a shot can be enhanced / altered with subtle changes to the colour of the film.
But the workflow for high quality colour correction is in many cases pretty poor, especially at our school were almost everyone edits on a different system & editing software - some use avid, some final cut pro, and then there is the odd vegas or premiere user in between. I recently described our various setups and the solutions that are available today for solving this in a post to the rebel cafe: my post (the rebel cafe is a forum website for readers of Stu Maschwitz' DV Rebels Guide, a book about very low budget action filmmaking).
I usually had two choices: do the colour correction in the editing software that was used to edit the film, or import the final movie into after effects and do it there.
Working in After Effects is the better option quality wise. It can work in 32bit per channel (not 32bit per pixel!) floating point which means that no matter how many colour operations you stack ontop of each other, you will never loose any detail in your image. You can also make complex selections, use a number of very nice plugins etc. It's not realtime, but if you want the highest possible quality, it's the way to go.
Working in the editing software has some other advantages: one of the biggest is that each cut is still there and you can start grading right away. If you work in after effects, usually the first thing you have to do is split your layer at all edit points so that brightening up at one point in the film because that shot is too dark does not brighten up the whole film, only that shot. In the editing software you have all those edit points and so there is no additional work there. What's also nice is that it is (mostly) realtime. But, as I said, the quality is not the best you can get.
Usually this meant that if there was little time and/or only slight corrections required, I would do the grading in the editing software, and only if I had the time to re-set all the edit points would I go for the whole thing and do it in After Effects.
But, *drumroll* with this new script I just wrote, you can export your movie from your edit software of choice, then export an EDL (Edit Decision List) as well, import the movie into AE and then run my new script to split your layer at all edit points given in the EDL. Nice. If you want to give it a try, download the script here and let me know how it worked for you.
