11/25/2014 – With the upcoming release of Windows Storage Server 2012 R2 Essentials, Thecus will be introducing data deduplication into the SOHO/SMB segment – a first in this appropriately positioned price segment.
Data deduplication involves finding and removing duplication within data without compromising its fidelity or integrity. The goal is to store more data in less space by segmenting files into small variable-sized chunks (32–128 KB), identifying duplicate chunks, and maintaining a single copy of each chunk. Redundant copies of the chunk are replaced by a reference to the single copy. The chunks are compressed and then organized into special container files in the System Volume Information folder.
The result is an on-disk transformation of each file as shown in Figure 1. After deduplication, files are no longer stored as independent streams of data, and they are replaced with stubs that point to data blocks that are stored within a common chunk store. Because these files share blocks, those blocks are only stored once, which reduces the disk space needed to store all files. During file access, the correct blocks are transparently assembled to serve the data without calling the application or the user having any knowledge of the on-disk transformation to the file. This enables administrators to apply deduplication to files without having to worry about any change in behavior to the applications or impact to users who are accessing those files.
Figure 1: On-disk transformation of files during data deduplication
To cope with data storage growth in the enterprise, administrators are consolidating servers and making capacity scaling and data optimization key goals. Data deduplication provides practical ways to achieve these goals, including:
- Capacity Optimization: Data deduplication stores more data in less physical space. It achieves greater storage efficiency than was possible by using features such as Single Instance Storage (SIS) or NTFS compression. Data deduplication uses subfile variable-size chunking and compression, which deliver optimization ratios of 2:1 for general file servers and up to 20:1 for virtualization data.
- Scale and Performance: Data deduplication is highly scalable, resource efficient, and nonintrusive. It can process up to 50 MB per second in Windows Server 2012 R2, and about 20 MB of data per second in Windows Server 2012. It can run on multiple volumes simultaneously without affecting other workloads on the server. Low impact on the server workloads is maintained by throttling the CPU and memory resources that are consumed. If the server gets very busy, deduplication can stop completely. In addition, administrators have the flexibility to run data deduplication jobs at any time, set schedules for when data deduplication should run, and establish file selection policies.
- Reliability and Data Integrity: When data deduplication is applied, the integrity of the data is maintained. Data Deduplication uses checksum, consistency, and identity validation to ensure data integrity. For all metadata and the most frequently referenced data, data deduplication maintains redundancy to ensure that the data is recoverable in the event of data corruption.
- Bandwidth Efficiency with BranchCache: Through integration with BranchCache, the same optimization techniques are applied to data transferred over the WAN to a branch office. The result is faster file download times and reduced bandwidth consumption.
- Optimization Management with Familiar Tools: Data deduplication has optimization functionality built into Server Manager and Windows PowerShell. Default settings can provide savings immediately, or administrators can fine-tune the settings to see more gains. One can easily use Windows PowerShell cmdlets to start an optimization job or schedule one to run in the future. Installing the Data Deduplication feature and enabling deduplication on selected volumes can also be accomplished by using an Unattend.xml file that calls a Windows PowerShell script and can be used with Sysprep to deploy deduplication when a system first boots.
For more information on Thecus Windows Storage Server NAS, please visit:
For more information on where to buy Thecus WSS NAS, please visit:
Established in 2004, Thecus Technology Corp provides market leading network attached storage and network video recorder solutions, committed to revolutionize how everyone from home user to enterprise level business centrally stores, manages and accesses their digital data both onsite and cloud-based. Thecus strives to deliver continuous innovation through cutting edge technology and design to provide data storage with the most customer-friendly platform, rapid performance and robust security. In May 2016, Thecus was formally acquired by Ennoconn Corporation and thus became part of the Foxconn IPC Technology group. Partnered together this diverse group of companies work in tandem to provide a total IoT (Internet of Things) solution. Now with an unparalleled portfolio to work with, Thecus is collaborating to bring unprecedented change to the network storage industry.