Replication Information for NetSwap Plus

Executive Summary
Highly Reliable Systems (HRS), has added bi-directional incremental replication over the Internet to their NetSwap Plus removable drive Network Attached Storage (NAS) backup appliances.  The feature will also be available in certain network connected RAIDFrame models.  A Netswap Basic edition is available without replication.  A key differentiator of  these compared to typical NAS products is the inclusion of removable drive(s) as well as functionality to mirror internal storage to the removable drive so that 2 copies of data can be managed and maintained.  The removable drive can be transported for storage off-site or can be used to “seed” a remote location with up to 4TB of data per drive.  Once seeded, uni or bi-directional replication can then be used to keep the remote site(s) up to date.

Unlike backup appliances that require monthly fees, NetSwap products can be placed in any location, such in a second office, with a managed service provider, or even at an owner’s home.  Replication over the Internet is incremental  so only changes are replicated.  Integrated compression and encryption insure the Internet is efficiently used and that customer data is protected without requiring external VPN tunnels.  Encryption levels up to AES 256 are supported. Synchronization jobs can be throttled and scheduled so that Internet connectivity is not bottlenecked during working hours.

Simple port forwarding is used to allow Netswaps to see each other over the Internet and low cost dynamic DNS services are supported so that no static IP addresses are required. The NAS appliances support multiple replication jobs and can notify administrators with e-mail alerts upon the success or failure of a job.  The system detects open and changing files and makes sure replication occurs at a point in time using a snapshot to insure data integrity.

Netswaps groom off-site data on a set schedule, checking to make sure the data is readable and matches the source.  The appliance will join a Windows domain, and use NTFS formatting to allow removable drives to cross plug with the company’s line of USB3 products.  This allows simple emergency access to data from any Windows machine.

The replication features in the Netswap Plus product line utilize Rsync, which is a robust file replication engine that does intelligent block level incremental copies.

 Important Note:  In replication environments where files will be open and changing 24×7, the High-Rely “snapshot” feature can be used to track active file changes, providing integrity for the replication offsite.   When using this snapshot feature the worst case scenario would require having twice the space on the sending side of replication job as is required for your full data set.  For example, a very actively changing 1TB  data set might require up to a 2TB drive on the sending side of the replication to allow room for the snapshot. While doubling space is overly conservative in most cases, it should be considered in drive size planning.

Key Features

• NAS to NAS and Windows to NAS support. No requirement for proprietary cloud services vendors or monthly fees.  There is also no restriction on using approved and compatible Windows Rsync Client to send jobs to Highly Reliable System’s hardware appliances.
• Uni-directional or Bi-directional Sync jobs supported.
• Intelligent support for incremental changes insures only changed blocks are replicated to remote side.
• Replicate entire disks or do so by individual folders.
• Multiple replication jobs supported.
• Multiple locations supported.  Replication jobs can send data to several locations.
• Static IP address not required.  Support for inexpensive dynamic DNS services including  dyn.com, easydns.com, dnspark.com, namecheap.com, and no-ip.com
• Accept and arrange multiple incoming replication jobs into folders to allow multiple customers or locations to utilize one destination NAS appliance.
• Replication can be scheduled to occur as frequently as desired (choosing frequency by minutes, hours, days, weeks, months).
• Ability to replicate to additional local drives (used to support multi-bay devices), allowing copies on multiple media and supplementing the built in mirroring capability.
• Ability to optionally encrypt replication jobs (AES 128, 256, RC4, DES, Triple DES), eliminating the need for VPN tunnels or other outside encryption.
• Ability to optionally compress replication jobs (settable levels from 1-9)
• Ability to throttle the use of bandwidth, using 3 user defined start/stop times to insure that end user Internet functionality isn’t impacted during business hours.
• User adjustable retries – to manage and retry failures caused by Internet outages or other problems.
• Support for maintaining file integrity, even if a backup or other file write is in progress to while replication is occurring (Snapshot functionality to freeze directory)
• Push or pull replication jobs supported between local and remote sites.
• Option to leave files deleted from source on the destination.
• Log files kept on the appliance to allow administrator to monitor or troubleshoot replication jobs.
• Continuous data grooming.  Data integrity is checked during each replication pass to insure destination files have not become corrupt.
• Robust error recovery and reporting.
• Notify administrator via email upon beginning, completion, or failure of a replication job.
• User configurable connection ports (TCP 873, and 8873 are defaults) supporting all speeds of Internet connection.

Some key screen shots are shown below to demonstrate how some of these key features are configured.

Main Menu

Replication

Replication jobs can be set to do full disks or individual folders.  Multiple jobs can be setup on a single appliance.  “Snapshot storage” is available if there is a possibility that files on the NAS may be changing during a replication to insure a point in time snapshot is what is replicated.

Snapshots

If you anticipate doing backup jobs or writing to your NetSwap device while a replication is occurring simultaneously, you may wish to turn on the snapshot feature to help preserve file integrity of files being changed.

If selected, a read-only copy of the disk being replicated will be created that is frozen at the time the Replication Job starts using the Snapshot Storage to store any writes to the disk. The Snapshot Storage must be large enough to store all writes to the disk that will occur while the Replication Job is running or disk writes will fail once the storage is full. The safest way to know there is enough space on the Snapshot Storage disk is to use a disk with a capacity greater than or equal to the capacity of the disk being replicated or create a Snapshot Reserve that is equal in size to half the capacity of the disk.

The Snapshot Storage disk may either be an entire disk that is not shared or a disk that has been formatted with a Snapshot Reserve greater than 0. If an unshared disk is used, data on the disk will be destroyed.

Bandwidth Management

Compression levels of 1-9 are supported.  Compression is typically helpful for transferring larger files that are not already compressed (not video or mp3s, for example). Stay away from compression with things like .jpg files.  Also for small files you may find that compression slows down instead of speeding up transfers.  Most people find removing compression makes things faster on a LAN.  Also note that some ISPs, such as Comcast have features built into their services that provide an initial “power boost” for a few seconds to speed up file transfers, then are quickly throttled down to “normal” levels. In this case your bandwidth test with the ISP may not last long enough to show your “true” long, sustained available bandwidth.

 

Note: Global options allow you to control several advanced things such as whether replication will delete a file that has been removed from the source on the destination side (or keep it at the destination).

Scheduled Replication

Encryption
Many replication appliances and some software require a tunnel or VPN to be setup using expensive routers. With the NetSwap Plus, encryption can be done box to box, eliminating the need for expensive VPN routers.

Email Alerts
An email can be sent upon the beginning, completion, or failure of a replication job.  Many other options for emails exist, such as notifying when an end user does a drive swap or when the device is finished with a remirroring job.

Dynamic DNS

Each Netswap can be configured to register with popular Dynamic DNS services so that replication and other features can be used on a DSL or other inexpensive Internet connection that doesn’t have a fixed IP address.  Currently supported services include dyn.com, easydns.com, dnspark.com, namecheap.com, and no-ip.com