Editing Smartphone or iPhone H.264 Video
If you are using your smartphone (iPhone) for making videos, you will most likely need to edit your video somehow. These days, there are many options when it comes to editing digital video. You can edit on a desktop, laptop, tablet, iPad or even directly on your smartphone.
The device you choose is determined by budget, convenience and how involved the editing process is going to be. Editing can use up a lot of processing power, depending on file format, number and type of effects and so on.
Let’s quickly look at the pros and cons of different options.
Desktop
Using a desktop editing system will give you a lot more options and potentially more editing power for your money. Additionally, a separate monitor (or even multiple monitors) provides you with more screen area for editing. Extra processing power generally comes cheaper via a desktop system.
The major downside is lack of mobility. Once you have set up your editing system, you won’t be going anywhere with it in a hurry.
Laptop
A laptop will provide you with a balance between mobility, screen space and processing power. A high-powered laptop will cost you more money than the equivalent desktop. However, you can fold it up in a minute or two and be on your way.
You also have a keyboard for using software shortcuts to speed up editing.
Tablet/iPad
Still more screen space than a smartphone or iPhone, but less than a laptop. A keyboard and a mouse will be optional extras, but for simple jobs you probably won’t miss them. Inevitably, there will be less processing power available.
Smartphone or iPhone
It’s possible to edit your smartphone-shot video without the files needing to leave your smartphone. There are a number of inbuilt or 3rd party editing apps available. For those using the iOS system, you have the fantastic LumaFusion app option.
However, performing complex editing work within the restricted space of your smartphone screen is going to be a challenge.
H.264 file format
Most smartphones including iPhones shoot video in a file format called H.264 (also known as MPEG-4 AVC). While this format is brilliant at keeping file sizes low, it comes with some drawbacks. This is especially evident when editing.
Even powerful editing systems can struggle when it comes to editing H.264 files, so what is going on? For those of us who are not professional editors, we might be surprised to find our computer’s system almost grinding to a halt as soon as we add effects to a video clip. The reason for this is to do with the way H.264 files work.
Compression
To fit high quality video into a smaller file, the information contained with an H.264 file is heavily compressed. The way it works is by reducing the information contained within each frame. Whereas a uncompressed video frame will contain every bit of information, H.264 frames only contain information which is different to the previous frame.
If a pixel within a frame is the same brightness and colour as the same pixel in the previous frame, the H.264 frame does not keep a record of that pixel, it rather refers the playback software to the original frame. If the same pixel of the next frame is still unchanged, the H.264 frame will keep referring the software back to the first frame.
So, using this system unnecessary information is not recorded, thereby saving file space. However, the downside of this system is that it places extra burden on the playback system. This is because it has to keep referring back to recreate a full frame’s worth of pixels.
Group of Pictures (GOP)
Highly compressed video codecs are only made for recording and final delivery on the web or they are optimised for playback systems. Now, the system used to create an h.264 file is called Group of Pictures (GOP). This means only every keyframe is saved as a full frame. And in practice, this means that normally these are between 30 to 90 frames between each keyframe.
Since the digital editing is frame based, each keyframe has to be extracted each time you scrub, play or render. In other words, a sequence has to be extracted in RGB space and held in RAM or cache. This process repeats itself over and over and keeps your CPU and GPU busy.
Adding Effects
While your system will probably cope when simply playing back the H.264 video, problems can arise as soon as you add more for the software to do. For example, adding a colour grading or correction process. Now, not only is your system having to recreate each from by referring to the original frame, but it the has to adjust that frame again, depending on what you are asking it to do.
VFR – Variable Frame Rate
In addition to the compressed H.264 file format, smartphones often shoot video with a variable frame rate. Like with the H.264 format, VFR is essentially a software corner cutting strategy to reduce file sizes. If a smartphone camera detects little or no movement in a video, it can slow the frame rate to reduce the total number of frames (and therefore file size).
Even worse news, VFR occurs during a shot. This means you can set your video to record at 30fps, then your smartphone camera might decide to drop the frame rate to 15fps for a few seconds, before going back up to 30fps.
Again, any editing software has to deal with this extra task when playing back the video. The software must be able to see a video has changed frame rate and adapt to maintain correct playback speed.
Double Trouble
The combination of missing frame information and varying frame rates can cause a real headache for your editing system. Here are some options to improve your editing experience using H.264 files:
1. More processing power
The simplest (most expensive) option is to upgrade your computer’s processing power. More power will mean your computer can keep up with all the extra tasks you have given it. That said, even the most powerful systems can struggle with higher bitrate H.264 video, so you might want to give your new system a trial before lashing out the extra $.
2) Transcode the H.264 files
Basically, this means changing the compressed files to uncompressed files. The missing frame information will be placed into each frame and 15fps will be converted to 30fps. Now your video file will require less processing power to play. On the downside, the files will be much larger.
You can transcode to an intermediate codec, like ProRes or DNx. In addition, (another downside) you’ll have to wait for your files to transcode before you can begin editing.
Two of the most popular software packages for transcoding are Adobe Media Encoder and FFmpeg. But there are other options available as well. If using Media Encoder, you can set up a watch folder to automatically transcode any footage dropped in the appropriate folder.
3) Proxy workflow
In recent years, proxies have become much easier to use. Essentially, proxies allow you to easily switch between the original footage and a light-weight version, optimized for editing. Now, you can use your original files when you need the best visual quality (color grading, masking, exporting, etc.), but switch to the proxy when trimming or assembling your timeline.
For example, Adobe Premiere can be set to automatically create proxies when importing videos files. The extra time required at the start might save you hours or days waiting for your system to grind through H.264 footage.
H.265
High Efficiency Video Coding (HEVC), also known as H.265 and MPEG-H Part 2 has been designed as a successor to H.264. The new system provides even more compression than H.264 and no longer uses the GOP system. However, the same issues arise when editing H.265 as with H.264. Your system will need to work harder to process a H.265 during the editing process, so the same tips apply.
Eager to learn more?
Join our weekly newsletter featuring inspiring stories, no-budget filmmaking tips and comprehensive equipment reviews to help you turn your film projects into reality!
Thank you!
You’re welcome, Daniel