What is a ToF camera in Smartphones?
ToF refers to “time-of-flight”.
Time of flight (ToF) is the measurement of the time taken by an object, particle or wave to travel a distance through a medium.
Remember those films about submarines in World War 2? Remember the sonar equipment used to try to detect the submarine? A device would send out a “ping” sound and wait for the reflections. Depending on the time it took to bounce back, the sonar user would try to detect the position of the submarine.
Just like the sonar does with sound, the ToF camera measures the time it takes for light to travel to and bounce off objects. The camera then uses the delay to judge distance. In this way, the ToF camera sensor builds up a 3D map of the world.
ToF cameras are also sometimes known as a “depth camera”.
These cameras are now starting to appear on smartphones. Here’s some smartphones which carry ToF sensors:
- Samsung Galaxy S10 5G
- LG G8 Thin Q
- Oppo RX17 Pro
- Honor View 20
- Huawei P30 Pro
- Oppo Reno
- Sony Xperia XZ4
Reports say Google is likely to bring ToF depth support to its ARCore augmented reality platform as part of Android 10 “Q”.
Coming to iPhones next year?
Analyst Ming-Chi Kuo said Apple might equip its 2020 iPhones with camera enhancements both in front and at the back. “We predict that three new 2H20 iPhone models will all be equipped with front Face ID, and two of the new models will provide rear ToF.”
Using ToF sensors, Apple will be able to boost a device’s Augmented Reality capabilities. It will allow future iPhones to map a room and create an accurate 3D rendering based on the acquired depth data. iPhone users can then maximize ARKit 3’s features such as motion capture and people occlusion. Users can create videos in real-time augmented with 3D objects, without needing to use a green screen.
What is the use of a ToF camera?
Photography and augmented reality are the two obvious uses for a ToF camera.
Gesture recognition
What if your smartphone could take a picture when you wave your hand in a certain way? The upcoming Google Pixel 4 is reported to have gesture recognition as one of it’s main selling points.
Blurry backgrounds
One use of the 3D imaging system might be to create a digital Depth of Field so that you can choose to add a blurry background to your photos or videos. At the moment, this seems to only work for photos and not video.
However, there’s disagreement on whether this system is in fact responsible for adding depth of field. Chris George of Digital Camera World says, “the depth information can help the AI software on the phone to defocus the background of an image – giving the bokeh blur loved by professional portrait photographers.”
However, Andrew Williams writing in Trusted Reviews says, “Huawei’s P30 Pro has some of the best shallow depth of field effects seen in a phone and does not appear to need its TOF camera at all for “Aperture” blur photos.” He also says the ToF sensor is not high enough quality to create nice looking blurred backgrounds.
Face Identification
Another use for ToF cameras is in the field of authentication and security. Unlocking your device using face recognition in 2D is obviously going to be less secure than using 3D mapping of your face.
Movement tracking
Williams suggests the main use of ToF in smartphones in the future will be for movement tracking.
How do Time-of-Flight cameras work?
A ToF camera comes in two main parts: an illuminator and the camera itself. The illuminator part floods the field of view with light which is out of the range of human vision. The sensor then measures the time it takes for the light to bounce back.
The process happens in fractions of a second, we’re talking nanoseconds here. Each pixel has its own time difference reading, so that all the pixels together create a 3D map of the image. And this is done for each frame, usually up to 60 frames per second.
This requires quite a bit of processing power, which is why it is only now becoming more common in devices such as smartphones. As mentioned by Williams, current ToF cameras tend to have quite low resolution. The ToF camera sensor in the Samsung’s S10 5G contains just 240×160 pixels.
Future applications for smartphone filmmakers
Anyone using their smartphone to shoot video will be keenly interested if this technology can be used to add shallow depth of field. Especially if this can be done in post production, as this allows greater flexibility.
This is also of interest to those adding computer generated effects to their videos. For example, when tracking movement in a video it’s better to have everything in focus as much as possible. Using a 3D map should allow us to shoot everything in focus, us this to add CGI, then add shallow depth of field after, if desired.
That’s just speculation on my part, but 3D mapping an image must surely have huge potential for filmmaking.
The reason why this isn’t yet seen in smartphone cameras when shooting video is probably down to the processing power needed. Reports are that when using a ToF sensor, smartphones have been heating up. As smartphones have no fan or other method of cooling, this can be a big problem.
But with increases in processor power, expect ToF cameras to become more used in photo and video applications.
Eager to learn more?
Join our weekly newsletter featuring inspiring stories, no-budget filmmaking tips and comprehensive equipment reviews to help you turn your film projects into reality!
