4K codecs - the good, the big and the ugly. Part 2

The first part of this blog looked at the file sizes of a 1 second 4K video clip using various QuickTime codecs. This part looks at the image quality of the codecs that use an efficient lossy compression system. Some codecs can compress an image and uncompress it without any loss of information (lossless compression), but these produce extremely large file sizes and I have left them out of this test as they should, in theory, show no difference to the original.   

The codecs compared here are GoPro's Cineform, Avid's DNxHD, MJPEG A, MJPEG B, MPEG 4 and Photo JPEG. Here is the table of file sizes for the various versions I made:

File sizes of 4K 1 second clip

Here is the link to the 1 second clip at HD 1920x1080 size in the MPEG 4 codec at 90%.

I made 2 JPEG (100%) stills of the first frame of the clip. One is the full size 4K image and the other is a 640x360 pixel crop with the image zoomed up to 200%. The cropped image is the top of the tower with the crane by the side. All the images are shown below, but I also made a stack of the crops in Photoshop and sliced out the layers for a comparison. 

Quality of various codecs 90% quality. 200% zoom and cropped

I haven't included MJPEGB here because it is no different from MJPEGA, but both MJPEGs show a large variation in the original colour, lifting the gamma significantly. All the other codecs were consistent and matched the uncompressed version.


The original purpose of this test was to see if the Avid DNxHD codec was suitable for 4K use but when I saw how poor the quality was I looked at all the other codecs to see what was wrong. You can see from the sliced picture how blurred the image of the crane has become, but the MPEG4 version is as sharp as the Photo JPEG image below. I was surprised when I looked at the file size how much DNxHD was compressing the image because I hadn't seen such degradation when editing high definition material and the file size is larger than MPEG4. I rendered out this clip in 1920x1080 with the codecs again set to 90% quality. The DNxHD clip was 44 MB in size and the MPEG4 was only 14 MB in size.

To be precise the HD version of the DNxHD codec was 44,808 KB in size which is exactly the size of the 4K version, which gave the solution. The DNxHD codec is only designed for the 1920x1080 video space and when the image dimension goes beyond that the pixels are duplicated (or more) to fill in the gaps, hence why it is blurred and the file size doesn't increase.

I think this is a mistake and Avid should limit the dimension the codec can produce, like H.264 does. Or it should ignore file size and produce a similar quality of image that the other codecs show here. I assume Avid will come up with a native 4K codec very soon which can be used in the next generation of cameras such as the Blackmagic Design Cinema Camera and their edit tools.

Meanwhile the best solution for moving 4K videos around the web appears to be MPEG 4 because it scales well, reproduces the original colour and is the smallest of all the options. However I hope that the GoPro Cineform codec becomes more widely used (and free) as it is robust and platform agnostic. And of course we wait for the launch of H.265 which is likely to be the choice of the broadcast world.

For videographers producing 4K clips the only solution is to use a lossless codec 

Here are the JPEG images of all the test clips. Please download them to compare.

Cineform 90%

Cineform 90% crop with 200% zoom

Avid DNxHD 90%

Avid DNxHD 90% crop with 200% zoom

MPEG A 90%

MPEG A 90% crop with 200% zoom

MPEG 4 90%

MPEG 4 90% crop with 200% zoom

Photo JPEG 90%

Photo JPEG 90% crop with 200% zoom

4K codecs - the good, the big and the ugly. Part 1

There can be little doubt that there is going to be a huge increase in the demand for 4K footage over the next few years and much of that early demand will have to satisfied by stock material. Currently broadcasters can turn to an enormous library of SD and HD material compiled over decades of shooting, but that won't cut the mustard when intercut with newly acquired 4K footage.

Which is good news for those of us who are planning on making our own library of high quality 4K stock shots. One question I would love to answer is which flavour of 4K will be the choice of stock libraries and I really hope that the big boys can come to some sort of agreement here, because uploading four different versions of a 4K clip is not a quick process.

Now H.265 has been announced and I think this will become the bookies favourite even though its younger cousin H.264 is not currently stock libraries first choice for progressive footage. But efficient as H.265 is, it's unlikely to be a good format to edit with for reasons I won't go into here. This means any stock footage will have to be transcoded to another format or codec before editing.

So I ran some tests on existing codecs using a 4K clip from a time lapse within Adobe After Effects to see if any of them would be good enough to host on the web and edit with natively. Here is a still from the sequence. 

The One Tower, St Georges Wharf, Vauxhall, London

The crane on the tall building behind Vauxhall Bridge was unfortunately hit by a helicopter in January 2013 in case it looks familiar - you can see a 1080p version of the time lapse here. I chose this clip because of the complexity of the image with lots of vertical and horizontal lines and a lot of detail in the sky and water.

Here are the codec candidates:

Avid DNxHD - works on Mac and PC and the codec is free.
BlackMagic codec - great quality but probably too big and not widely used.
Cineform - again works on Mac and PC but you need the Premium version of Cineform Studio to create files and this is $299.
Photo JPEG - An old codec, with good compatibility. Currently the choice of many stock companies but not the most efficient of compression choices.
TIFF - Using LZW compression should give a perfect image but with file sizes to match. Also tried a QT using TIFF.
Animation - Not used so much any more but moving between editing and graphics tools is used to be the king. Huge differences in file size depending on the compression ratio.
MPEG A & B - Had their day I think and largely used only for interlaced  
ProRes - A really great codec but will not play well (if at all) on PCs, which really is a ridiculous situation. If it did this would be numero uno. I'm working on a PC so couldn't produce any test results. Pah!

I created a 4K (4096x2304) 1 second clip in each format at two or three compression settings (75%, 90% & 100% where possible) to look at file size and quality. Here are the results:

File sizes of  1 second 4K clip in various codecs 

The TIFF sequence adds up to 288 MB for the 25 frames. 

Unsurprisingly the uncompressed or lossless compression codecs are significantly larger than the lossy compressed versions. The embedded TIFF QTs were significantly larger than the sequence of stills using the LZW lossless compression and the TIFF sequence would still be a good way to exchange between editing and graphic systems.

The lossy compression codecs start with Cineform (100% quality) at 126 MB for a 1 second clip. Bearing in mind that the maximum  length of a clip on iStock is 30 seconds and you could end up with a files of 3.78 GB! At 75% quality it would be a mere 1.8 GB

The tests showed that DNxHD 444 10 bit was almost the same size at 100%, 90% and 75%. Somewhat surprisingly it made little difference on MJPEG codecs whether 75% or 90% compression was chosen. But Cineform at 100% jumped up to 126 MB - almost double the 90% file size, which shows remarkably efficient compression if the quality stands up.

Photo JPEG and MPEG4 also showed a big difference between 75% and 90% and a 30 second clip at the higher compression (75% quality) would be just over 1 GB in size.

In Part 2 of this blog I look at the picture quality of the smaller files to see whether they have a future as a 4K stock format.

Another new video format - H.265

For stock footage and stills of London go to London Photography and Video. 

There are nearly as many video formats in existence as there are spoofs of the Gangham Style video and they make a video editor's life hell. The producer's simple question of "could you just export a file for me to send to the client" is the start of a round of "fifty questions", forty-nine of which the producer doesn't know the answer to and the one he guesses is usually wrong.

There is no point or indeed desire to explain things like codecs, wrappers, bit-rate and dimensions to a producer who wants a 23 minute long video in HD quality but small enough to e-mail. Unless you want them to go and make a cup of tea. So the editor (me) takes an educated guess and produces something that will play on the client's Blackberry.

So when a new format emerges from the ITU that will "ease the pressure on global networks" I am likely to crawl into a dark corner and order more tea. But this one could be a goer since it is the successor to H.264 the (not very snappily named) codec that is now the lifeblood of the internet video world (80% of all video streamed on web is in H.264) and is liked by Apple and Microsoft OS's alike. 

The recently announced, excitingly named Recommendation ITU-T H.265 or ISO/IEC 23008-2 or High Efficiency Video Coding [I think I'll stick to H.265] is twice as efficient as its numerically challenged cousin at crunching numbers which means similar quality files will be half the size. At the moment a 23 minute programme (which is all you get in a 30 minute slot on a commercial station) in H.264 at full HD resolution creates a file that's about 1.4 Gb in size. Download four of those and your 5Gb 4G limit is exceeded and that is going to cost you, but with H.265 you get twice as much for your money.

Now I am very interested in how 4K television develops (don't get me started at dinner parties) and this new codec is designed with 4K and even 8K in mind and will start to make it possible to watch online over a fibre connection. It will take a while to phase the new format in but once it is on my desktop and my client's tablet, it is one less question to ask when exporting a video.

4K Ultra-hd to be broadcast in July 2014 - time to shoot.

For stock footage and stills of London go to London Photography and Video. 

There has been quite a split amongst video, tv and film makers about the relative merits of 4K (now known as ultra-hd) television and whether it should be bypassed altogether in favour of 8K (whatever that will be called but super-ultra-hd sounds quite good in a Japanese accent).

Now it looks like 4K is going to get it's 15 minutes of fame although it may only last 2 years - I love a good cliché mixed with a tautology - as it appears Japan's Ministry of Internal Affairs and Communications has decided to launch its 4K broadcast in July 2014, two years ahead of predicted schedules, and 8K tv in 2016. The advanced 4K launch date could be to encourage the purchasing of (already available in Japan) ultra-hd televisions but more likely because the department's minister is a massive soccer fan and the 2014 World Cup final will be hosted by Rio de Janeiro that month.

What is more certain is that the demand for 4K footage will start to ramp up and so will the amount of cameras available to shoot it. Hopefully this will also lower the price since acquisition of 4K at the moment is quite high mainly because of the outboard recording unit. 

You can bare-bones it and shoot on the Canon DSLR EOS 1D C, which if you could buy for £6000 instead of nearer £9K would be snapped up faster than a sprat in a salmon farm. One of the reasons Canon have given for the high price of the 1D C is that there are only going to produce them in small numbers; I can see the price dropping as Canon decide to produce them in higher numbers when initial sales to early adopters tail off. CVP in London have already shown a £1340 drop in the initial price.

CVP selling the Canon EOS 1D C

My dream machine though is the Blackmagic Design Cinema Camera MKII with a larger sensor (so wide angles are possible) that can shoot 2.5K in RAW and 4K in GoPro's Cineform codec. Even if the price doubled for such a beast I doubt Blackmagic Design would have much trouble selling them.

Blackmagic Design Cinema Camera with important new logos!

But for now many people are going to shoot 4K on a GoPro at 12 frames per second and run the footage through Twixtor to produce full speed 4K footage. If you can handle the semi-fisheye look the quality is quite good. There is a big gap in the market just waiting to be filled and I think that Sony have the basics with the superb NEX-FS700 - just need a less pricey way of recording its 4K output.

Sony NEX-FS700

Certainly the Japan Government's announcement will accelerate the need for 4K footage and hopefully we will get the kit to shoot it at a price we can justify.

Useful software for videographers and photographers - Part 3

The two previous posts on software has concentrated on PC software since that is what I use most of the time, but a revisit to my Mac Pro reminded me of a couple of good solutions on iOS and also why one bundled app on Windows 7 is so much better than its equivalent on the Mac.

First a piece of software that works on both platforms. It's called Better File Rename and made by PublicSpace.net, is now in version 5 and costs $19.95. It is accessible by right-clicking in a Windows Explorer window which opens a new window offering a huge choice of sorting and renaming options. It is a simple way of renaming a large batch of files and I use it to name the hundreds of images I shoot during a time lapse so that when I open the image sequence in After Effects I see a name for the sequence not just letters and numbers.

Better File Rename

Two bits of advice about using BFR may help you out. First, if you have more than a hundred files, at first just choose one to rename, make the name change and perform rename. Then open all the files and perform the name change without changing parameters again. Because BFR updates the preview to all the files it can take a long time with a large batch. Second, don't try to rename more than 50 files on a network drive, it will fail. Move all the files to a local or USB drive, perform the rename and then move them to the network drive.

One of the ways I found to improve file handling on a Mac was to stop using Finder, which for someone moving around lots of files is clunky and frustrating. I spent a while looking for an alternative and came across Path Finder by Cocoatech. This can completely replace the OS Finder or can work alongside it and is a very powerful piece of software. Simply having two browser panels is a big improvement on Apple's own software, but it offers built in video and stills preview, excellent sorting and a much better control over application launch. I couldn't use my Mac without it and have not found anything quite as good for Windows yet. At $39.95 it is not cheap but you can have a 30 days trial and I think you will shell out after that.

Path Finder by Cocoatech

I upgraded all of my PCs to Windows 7 as soon as it was released and soon discovered the Snipping tool which I have permanently pinned to my taskbar. All it does is grab a still image of a window or a rectangular area of my desktop. I can then save the grab as a file or copy and paste it into any document. It is so easy to use and I am in control of where the files go - much quicker than Skitch. If you haven't used it and you have a Windows 7 PC, just try it. You'll find it in the Accessories folder in your Start Programmes.

Snipping tool

In Part 1 of this software blog series I wrote about Disk Aware and it's ability to find large hidden files using up valuable disk space. CC Cleaner by Piriform discovers all the little files that also use up a large amount of space. The software has been around for many years and I am always astonished by how much rubbish (CC used to be called Crap Cleaner) hangs around my PC. I just ran CC Cleaner on my laptop and this was the result:

CC Cleaner by Piriform

13,127MB of stuff I can happily delete. Notice that the analysis only took 103 seconds to run and I have deselected history and cookies as items I want to delete. At this point I haven't run cleaner so I can look through what it suggests deleting and modify the selection. I can also get it to look at the registry and delete orphan files from there. I wouldn't recommend doing this unless you are confident at fixing problems because the registry is a scary place, but I do use it and so far CC Cleaner has caused no problems.

I hope this series has unearthed a few hidden gems in the software world. Many of the tools are free and all bar one are under $100 dollars. There are some great people making this software and I would like to thank them here for their work. I would also like to ask everyone to use the software properly, not to distribute it and to pay the small amount asked to use it when appropriate. 

Please suggest any software in the comments as I know there must be loads of useful ones I have yet to discover.

  

Useful software for videographers and photographers - Part 2

Earlier this year I was presented with some footage shot in Cote d'Ivoire, Africa on a Canon 5D MKII. The pictures were OK but the sound was distorted when the levels peaked and a look at the waveform showed that everything was square off. After an appeal on Twitter (@pukkascott if you want to follow) one suggestion was to try iZotope's RX2 audio repair software. 

iZotope RX2 tools

This is an immensely powerful suite of tools including Denoise, Declip, Spectral Repair, Declick & Decrackle, Remove Hum and more. After a quick view of an instructional video I applied Declip to the exported WAV file and after a little bit of playing around the distortion was completely removed and the waveform peaks nicely rounded. At $349 for the standard version it is not cheap but could save your bacon. 
A much cheaper version called music and speech cleaner is available for $39 which improves noisy speech but is no good for declipping.
There a lots of sound capturing pieces of software available and after a lot of experimenting I plumped for SoundTap by NCH software. It produces very clean records from anything played on a computer or voice recordings from Skype. 
There a lots of sound capturing pieces of software available and after a lot of experimenting I plumped for SoundTap by NCH software. It produces very clean records from anything played on a computer or voice recordings from Skype. 

SoundTap

Backup is one of the most important day to day operations since tapeless production came along. It took me a while to get round the fact that my G-Tech Raid drives were not backed up despite having two drives in the case but my Netgear network drives were. So now I use Netgear network drives to backup the G-Tech ones using Netgear's own NTI Shadow software. It works OK but it's a resource hogger and I have to disable the software when editing.
Backup is one of the most important day to day operations since tapeless production came along. It took me a while to get round the fact that my G-Tech Raid drives were not backed up despite having two drives in the case but my Netgear network drives were. So now I use Netgear network drives to backup the G-Tech ones using Netgear's own NTI Shadow software. It works OK but it's a resource hogger and I have to disable the software when editing.

Deep Meta thumbnail selection

It's important to set up the right recording location and the file naming convention in options as it would be impossible to find the recordings later if you don't. One feature I really like about the software is that even when you click "start recording" the file only starts to be created when it senses an audio input. Until the end of December it is available for $19.99 so I recommend you to get it this week.

To backup my system drive I use Casper 7.0 by Future Systems which creates a clone of the drive. The software manages to copy every file on the drive so the backup is true replica of the primary drive and when that falls over you really can slot in the backup and everything boots up perfectly. 

It is light on computer resources and it uses SmartClone to only back up changes to the drive, but doesn't take hours checking the drive like other tools I have tried. I doesn't work with network drives or on a Mac unfortunately but should be on everybody's PC.

Getting more specialist I thought I would briefly mention two tools I use to upload and manage stock material. 

I recently discovered Deep Meta by Belgium company Eazign byba which is a great analysis tool for iStockphoto material. I won't go into detail about what the software can do because as it is free there is no reason not to try it, but one feature I discovered is a real bonus. If you control+click to select a number of connected files and then right click to open the dialogue window, click on "create thumbnail links...", save changes and a little while later the associated thumbnails will appear in the iStockphoto listing for each one giving a link to alternative shots for a purchaser. 

 Finally Send to Smugmug by Omar Shahine is a very useful piece of freeware for uploading photo and video files to your Smugmug account. Once account details have been entered it will list your albums (or you can create a new one) that you can upload to. Choose a folder containing the files and select which files you want to send and click upload. Very quick and easy.

Useful software for videographers and photographers - Part 1

Over the years I have purchased or downloaded for free many small pieces of software to achieve specific results or solutions. Some are used a couple of times and then deleted, others, like Christmas dogs, stay with me forever. I thought I would run through some of them here as they may prove useful to you one day.

MPEG Streamclip by Squared 5 is one of the best known video converters and despite owning powerful conversion software by Avid, Apple and Adobe, this is the one I use the most. Firstly it produces files that are universally playable and recommended by stock sites. Secondly it can convert sections of a DVD to a QT file that can be imported to Avid. Thirdly it is quick to use and the presets are logical. Finally it is very small programme that has little impact on computer resources.

MPEG Streamclip options

Most editors know that attaching Apple formatted drives to a Windows PC just doesn't work unless you install Mediafour's Mac Drive which allows you to be totally agnostic about the drive format. But another of the company's software, Disk Aware has been the find of the year for me. Disk Aware analyses every folder and file on a drive and presents it as a list arranged in size order and also as a colour-coded wheel which shows you the relative size of the files and folders. Simply click on a segment and analyse the next layer down.

DiskAware - The Wheel

The programme only costs $10 per computer but it can analyse any drive: internal, external, flash and even network drives. It enables easy clearing up of drives and lets you work out why all your storage has disappeared. 

One tool I used for a while, dropped and have returned to is IrfanView, the strangely named freeware viewer for PCs. One of the more recent improvements is IrfanView Thumbnails which you get when you install the complete set of plugins and I place as a shortcut on the desktop. It opens up an explorer window with a light grey background to distinguish it from Windows Explorer. Open up a folder containing mixed media and almost immediately it is populated with thumbnails which can be set to many sizes, not just Window's three. 

IrfanView Thumbnails of text, video and .psd

Thumbnails are shown for videos (with filmstrip icon), stills (even Photoshop) and text. PDFs are not shown however. One of the most useful tools is the ability to save those thumbnails as separate images for putting into a spreadsheet for example. Hovering over a thumbnail reveals information about the file itself such as dimensions, although for video only the dimensions of the thumbnail itself are shown. Like most features of IrfanView they are a work in progress, but the ability to see thumbnails from both videos and Photoshop files make it worth having.

In Part 2 I look at a a couple of sound tools, a backup programme and a couple of tools for uploading and managing material at iStock and Smugmug.