Saturday, October 6, 2012

SynthCam Review for iOS

Very few apps doing anything truly innovative with the camera on a modern smartphone. That is not to say that apps like Instagram are not useful, it's just that they don't do anything new.  Most photographic apps perform either some measure of in-camera alterations or have some kind of photo sharing capability. Released in early 2011, SynthCam does something entirely new. Very shallow depth of field shots can look beautiful coming out of a dSLR equiped with a fast aperture lens, but are impossible to replicate on devices with very small sensors, such as those found in camera phones. This is not a limitation of technology, but one of physics and geometry. In fact, as I pointed out in this post, sometimes the extreme amount of focus that you get in a small sensor device is actually useful.

Straight Capture from iPhone 4s
SynthCam takes the advantage of the video and gyroscope/motion sensing capabilities of a smart phone (iPhone 4s in this case) to simulate a larger light capturing area than what the phone actually has. It does this through continuous capture of a given scene, with synthetic computation to construct a shallower depth of field. Think of it as a physical-world ray tracing application; during the capture process, you have to keep the phone parallel to the scene; by slowly moving it in a circle, you are tracing out a virtual area that would be analogous to a larger sensor/lens system.
SynthCam Capture.
So how does it work? In practice...it takes, well, some practice. The key is to move the camera slowly, and accurately within a 2D plain; tilting or rotating the phone will produce suboptimal results. Accurate captures can produce results that look like they were downsized from a dSLR. Poor captures produce pictures that look like they suffer from excessive motion blur. The key is slow and deliberate hand motion during the capture process. A rule of thumb  to remember is that shallow depth of field is not a cure for poor composition. Even though a fast dSLR lens can blur out the background, you still wouldn't place your subject behind a busy background (like tree branches) because of the harsh bokeh that can result. The same applies to SynthCam; don't expect pretty results if the background is busy.

A side benefit to this application is that you can use it to generate low-light pictures with  less noise than what your camera phone is capable of doing.  Because the final image is reconstructed from multiple video-frame images, the application essentially functions as a automatic image-stacker. The results that you  get are very much cleaner than what the a phone camera would be capable of on a single shot. However, when I tested this, the results were inconclusive. Yes, there is less noise in the SynthCam frame, but the straight capture from iPhone is still better. Because it is relying on the video subsystem. SynthCam can only output 960 by 720 pixels, compared to 3264 by 2448 for the iPhone 4s.... more noise or not, the straightforward capture has more detail, and if you downsize the iPhone capture to SynthCam size, the noise becomes less of an issue.

iPhone 4s Straight Capture
SynthCam Capture.
In the example above with the yellow orchid, you can see that the motion capture path was actually fuller than in the example at the top of the page, but that the background blur looks noticeably worse. I think part of it comes down to the the fact that my hands aren't naturally steady, but also the the synthetic processing that the program uses to create the effect of good bokeh. You really have to be slow and deliberate. Besides, why blur out the picturesque cityscape of Ketchican, Alaska in the first place?

Version 2.0 of this application added the ability to do multi-point focus, which is something that you cannot do with a conventional image capture, as there can only be one plain of focus. Multi-point capture can produce images that simulate those that you would get with a tilt-shift lens. Personally, I think this is a fad, but again, this is a technically innovative use for the iPhone camera.
iPhone 4s Straight Capture
SynthCam One-Point Capture
SynthCam Two-Point Capture
Note that in the last picture of the figurines, a two-point capture was used to keep the figure in the background in relative focus compared to the foreground. The program has an option to do four point capture. Though try as I might, I can't can't get my samples to come out as crisp as those on the SynthCam website. One of these days it might be interesting to create a little rig to see what could be accomplished with a little bit of mechanical help.Note also, once again, how 'synthetic' the quality of the bokeh is.

The downside to this application is that the capture is based off of the iPhone's video system, not it's stills system, so the the output is limited in pixel size compared to a conventional shot; however the results are still great for web output. This app is not going to replace a dSLR with a portrait lens, but it does open up creative possibilities when only a camera phone is available. There is a large degree of patience and technique to using this app, though. If the staff at your local Starbucks knows you by name, your hands might be a little too jittery for this to work. However, the great virtue about it is that it is free, and that it points to some truly creative things that you can do with a smartphone. Say for example, you were writing an independent enthusiast blog, for... oh, I don't know... let's say cameras or some other sort of knick-knack... and you forgot to tote your real camera one day for your next blog post. SynthCam is actually a nice alternative for creating interesting looking pictures for blog posts that don't look like you snapped them from your camera phone.

SynthCam is the creation of Marc Levoy of Palo Alto, California and is available for iOS devices running version 4.2 through iTunes.

No comments:

Post a Comment