Storage Solution for Video Production Business - NAS?

[inhousecreate]

We are a team of creatives looking to solve our growing storage issues as I’m sure most businesses discover.
The majority of our work is video and now more often than not we filming in 4K.

On average a job/project will consume roughly 200GB in its entirety which includes the raw footage, proxies, revision exports, music, project files & final version outputs.

We are mainly using Adobe Creative Cloud (Premiere Pro CC) for our video editing. With that said, we also working with DaVinci Resolve.

From a storage perspective, we have two main 5.1 Mac Pro edit machines with M2 boot drives, a secondary SSD, and 2 x 4TB drives in Raid 0 Configuration.

In addition we have a Business Dropbox account where we keep our working files and final outputs shared to clients.

Our office is a complete Mac environment.


What are we hoping to achieve?

We are thinking of getting an 8-bay NAS (either Synology or QNAP - what are the main differences?).

We’re wanting to come back from a shoot and backup our footage from SD cards straight onto our NAS.

Our understanding is that this will enable us to work of any local machine to access our files and in theory work directly off the NAS. Is this possible? Are there speed limitations?

Does anyone have recommendations in this space? We think it’s better to get an 8-bay NAS over a smaller one to avoid the need to upgrade again in the immediate future.

Any advice would be greatly appreciated!

 

[techwarrior]

Can't go wrong with QNAP or Synology WRT NAS. For most, it comes down to a preference. I have used both, but Synology tends to be favored by more Mac power users from what I see on forums. Bottom line is to look for features and compare costs.

Some models can accommodate SSD for caching, speeding read\write operations considerably, that may be a worthwhile feature for an operation like yours.

Benefits of course with greater than 2-Bay NAS are RAID5 or 10 options. This gives you more resiliency should a drive fail.

Nothing will beat the speed of locally stored files when you are working on them. Between network latency and NAS I\O limits, local M2 storage will load and update much faster. However, with both NAS, the client Sync apps can selectively keep files in sync between the Macs and NAS, giving instant access with effortless backup. For example, you could have an Active Projects folder that the Macs sync to, giving all users a copy on their local system with a copy on NAS as well. This is similar to DropBox sync which you should already be familiar. When the project is completed, it could be moved to an Archive directory which does not sync, saving Mac storage space but keeping projects for later access.

RAID5 or 6 is a great way to go, giving you a lot of space with reasonable resiliency from 1 or 2 HDD failures. RAID10 adds redundancy, so if drives in one array fail, the secondary array keeps the data safe until the first array is repaired. If long term resiliency is critical, a secondary long term archive is probably justified.

 

[dwig]

...
Our understanding is that this will enable us to work of any local machine to access our files and in theory work directly off the NAS. Is this possible? ...

Possible, yes, but...

• NAS access will be slow (READ: painfully slow for large files, compared to local storage).
• When a NAS is used to work from directly it can't be considered a backup of any sort, which means you need another one for backup purposes.

The second pont, above, is a serious one to consider. Even if editors work locally and sync with the NAS, the NAS is no longer a reliable backup.

 

[MRrainer]

Budget?

I love these kind of question without a single hint of a budget....

Consider FreeNAS.
https://www.ixsystems.com/freenas-mini/

Also, don't use RAID5. RAID6 or forget about it.
RAID5 is dead and has been for as long as 1 TB drives have been available.
It can take days to re-silver a 4TB disk - at that point, another one (which usually came from the same batch) might fail and then your NAS is just an expensive piece of electronic waste.

Move to 10GBit Ethernet, if you haven't already. At least for the server.

Ideally, you'd get two and replicate one to the other, depending on how painful (and costly) it would be to wait for spare-parts to show up while it's not working.
Though I think iX-system will provide you with spares in advance (in exchange for money of course).

How do you plan on backing up your data - the replicated data on the 2nd unit can not really be described as "backup", it's more to increase the availability.

 

[inhousecreate]

Possible, yes, but...

• NAS access will be slow (READ: painfully slow for large files, compared to local storage).
• When a NAS is used to work from directly it can't be considered a backup of any sort, which means you need another one for backup purposes.

The second pont, above, is a serious one to consider. Even if editors work locally and sync with the NAS, the NAS is no longer a reliable backup.

Thanks for the reply... why is that so?

 

[inhousecreate]

Budget?

I love these kind of question without a single hint of a budget....

Consider FreeNAS.
https://www.ixsystems.com/freenas-mini/

Also, don't use RAID5. RAID6 or forget about it.
RAID5 is dead and has been for as long as 1 TB drives have been available.
It can take days to re-silver a 4TB disk - at that point, another one (which usually came from the same batch) might fail and then your NAS is just an expensive piece of electronic waste.

Move to 10GBit Ethernet, if you haven't already. At least for the server.

Ideally, you'd get two and replicate one to the other, depending on how painful (and costly) it would be to wait for spare-parts to show up while it's not working.
Though I think iX-system will provide you with spares in advance (in exchange for money of course).

How do you plan on backing up your data - the replicated data on the 2nd unit can not really be described as "backup", it's more to increase the availability.

Just exploring options at the moment and not too concerned with budget, it's more about finding the right solution without shooting ourselves in the foot right now if we make a decision.

10Gbe is definitely something we're looking at. Thanks for your comments!

 

[techwarrior]

Thanks for the reply... why is that so?

Mac and NAS Bus Speed for drives is either SATA III (6Gbps) or NVMe M.2 (32Gbps).

LAN speed is either 1Gbps (1000Mbps) or 10Gbps (10,000Mbps).

The drives may actually be slower than the bus, so read speeds would be roughly:

• 7200RPM HDD will typically max out around 100MBps (800Mbps).
• SSD might do 300-500MBps (2400-4000Mbps)
• M.2 NVME something like 3000-3500MBps (24,000 - 28,000Mbps),

Read and write will never be faster than the slowest point in the chain. so you can see the network, even at 10Gbps speeds slowing down M.2 Flash access speeds.

NAS drive speeds will be similar to those on the Macs, but may be slowed by multiple users accessing the NAS, by inadequate network bandwidth. Typically, NAS drives will be HDD. HDD are cheap (in comparison to flash), and lend themselves well to RAID. Regardless of HDD vs SSD, the main bus for mass storage in a NAS will be SATA.

NAS may speed up read and write operations of HDDs because RAID will slice up read\write tasks across 3 or more disks to reconstruct a file to send to the user. M.2 cache on NAS mainly helps with write speeds, they will work a little like a Mac fusion drive in that frequently used files may reside on the M.2 flash but if the files have to be pulled from the HDDs, things will slow down considerably.

Since video and photo editing typically demands fast drive access, local NVMe drives will offer the best performance for your users. But given cost, these local drives should be limited to the currently active projects, offloading less used files to NAS.

So, a NAS is probably best thought of as a massive storage\archive device rather than a hard drive to work off of. Using sync clients, they can effortlessly ensure working files are stored on both the Macs and NAS, and older files that are retained primarily for archive purposes need not be synced to the Macs.

Accessing files on a NAS can be done by web interface (slow), Finder (similar to any SMB share on a network), or sync client.

 

[monokakata]

Posters are correctly pointing out that a NAS isn't a backup. There may be reasons to back up a NAS with another NAS, but it's also possible (with Synology, anyway) to simply attach one or two USB3 disks to back up the NAS. I have a 4 disk/2 volume Synology NAS, 4 TB in each volume, and I have 2 4 TB disks attached. The backup is auto.

A pair of disks in a two-disk enclosure is a lot cheaper than another NAS.

 

[MRrainer]

It's not only about the bandwidth. 10GBit has lower latency than 1GBit.

This will be noticeable if you read data from the NAS and it's cached in its RAM.
But also, if it has a battery (or capacitor)-backed write-cache and you're writing data onto it.

We're at a point where you could actually equip a small NAS with enterprise SSDs.
You'll max out 10GBit with those.

 

[shaunp]

As a Mac user I would look at QNAP. Why? Because they have several models with Thunderbolt 3 and 10GigE support. Synology probably have better software, but QNAP have the widest choice of hardware. It's the ability to connect several users, at highspeed, to the same storage is what you are wanting.

Both QNAP and Synology also enable tiered storage - that is have a mixture of SSD and HDD that works a bit like a fusion drive in a Mac. There are limits to the performance, but overall I'm finding it okay. I have a TS-932x connected over 10GigE to a PC and the performance is pretty good, just make sure you change the smb.conf to enable smb multichannel if you are going over 10GigE.

You didn't say what your budget was, but QNAP will have something that's probably in your price range. The TS932x for example starts at under £500 for the NAS, you just then need to add your storage.

 

[phrehdd]

Your first stop should be smallnetbuilder site where you can see a bevvy of NAS units that have been tested (charts provided) and some subjective with a conclusion.

Both Synology and QNAP make fine small business NAS units. I happen to have been using QNAP for years while friends are as happy with their Synology 'appliance.'

To consider -
Real use (storage or work space)
NAS throughput vs network throughput including multiple users accessing
RAID options
Selection of drives, RAM size (and possible upgrade) and a "caching" scheme via added SSD in some instances.

There are various ways to set up a system and always make sure to include a backup schedule with perhaps a nod to disaster recovery.

 

[styymy]

Take a look at the Promise Pegasus 3 series. https://bit.ly/2VV54Hv
We're a creative team running our MAC environment within a Marketing dept. We're able to connect and share all of our creative and video files within our company's Windows server. We've been using the Pegasus3 series to store our files the past 6 years with no issues. Promise has provided great support whenever we've needed it.