Contact us at 216.468.5200 or

White Papers

Optimization and De-duplication Technical Overview


Company networks are changing. Site and server consolidation is moving IT infrastructure from branch offices to central locations. Cloud computing is beginning to deliver IT services externally, and users need more flexibility to move between offices, work from home and work remotely while travelling. These changes mean that more and more users are working remotely from the applications they are accessing day-to-day.

The applications that users are accessing are also becoming more network dependent, providing richer experiences, providing advanced collaboration capabilities, or allowing users to participate in complex business processes.

More remote users and more content-heavy applications are pushing the requirement for Bandwidth Optimization technology. Acceleration Systems leverages the power of the cloud to provide Bandwidth Optimization any business can afford.

Acceleration_Systems_technical_overview_of_bandwidth optimization_technology

Cloud-Based Bandwidth Optimization

Traditional approaches to Bandwidth Optimization have made networks more effective by providing an expensive hardware device located at each side of the network. These hardware devices control the connection to the network and provide efficiencies. Hardware based solutions provide great optimization but introduce deployment problems – IT management required on each side of the network, costly purchase and upgrade processes, inability to support cloud scenarios or individual remote workers.

Acceleration Systems has been designed from the beginning to provide the capabilities of expensive hardware based Bandwidth Optimization entirely in a cloud-centric model – with algorithms appropriate to meet the requirements of low footprint clients and virtualized commodity servers.

As a cloud-based Bandwidth Optimization solution, Acceleration Systems has the following advantages:

  • Full optimization capabilities within an individual client or inexpensive commodity appliance
  • Easy to deploy within cloud infrastructures
  • Extensive software-based optimization capabilities (flexible deployment options)
  • Straight-forward deployment without changing the network
  • Targeted acceleration for problem applications or specific users
  • Scales up and down without friction as needs change

Acceleration Systems intercepts traffic destined for the public Internet or a secure corporate server, routing the traffic through the Remote Business Accelerator (RBA) on the client side of the network (or soft client running directly on the PC or laptop). The client side RBA routes traffic to the Acceleration Systems cloud-based servers. With traffic routed through Acceleration Systems’ technology on both sides of the network, our software can remove unnecessary redundant calls across the network, pre-fetch and bulk transfer data blocks, and further reduce the number of bytes required to deliver the requested data. All traffic to or from the public Internet is routed through the RBA, allowing the acceleration software to learn traffic patterns and provide the maximum optimization.

Acceleration_Systems_technical_overview_of_bandwidth optimization_technology_traffic_routing

Traffic routing is flexible. The preferred approach is dictated by the customer’s environment:

  • Individual software client intercepts traffic destined for the public Internet
  • The local RBA acts as the default gateway for the office
  • Traffic in a site is routed to the local RBA using policy based routing
  • Static or DHCP routing is configured between clients and the local RBA

Once the traffic is routed through Acceleration Systems software within the client network, it is then optimized and passed through the Acceleration Systems cloud-based server. The cloud-based server re-creates the original network request and sends it on to the public Internet resource being accelerated. Data from the Internet goes through the same process in reverse back to the user.

By understanding the traffic patterns and connections being made between the client and the Internet, Acceleration Systems can eliminate redundant calls, and minimize the number of bytes transferring the last mile to the customer’s office connection.


Acceleration Systems provides comprehensive Bandwidth Optimizations specifically designed for the requirements of today’s cloud-based business resources. Acceleration Systems provides three categories of optimization:

  • Protocol Optimization
  • In-line Stream De-duplication
  • Compression

Protocol optimization is the only optimization which is directly dependent on the actual protocol being used across the network. De-duplication and compression provide benefits regardless of the protocol or data being transferred.

Protocol Optimization

Protocol optimization looks at the message sequences for specific protocols, and ensures that redundant requests are removed and not passed over the Internet connection. The Acceleration Systems software client or client-side Remote Business Accelerator recognizes requests made to the Internet and identifies the protocols being used within the requests. Many protocols do not make the most efficient use of the network (making requests across the network which are not required or which ask for information in small, inefficient amounts).

Acceleration Systems provides a number of protocol optimizations to make protocols more efficient. These combine with the other optimizations to ensure that traffic to and from systems such as email (via MAPI or HTTPS), file shares, collaboration and ERP applications are fully optimized.

CIFS Optimization

A typical example of an inefficient protocol is the Common Internet File System (CIFS) protocol. CIFS provides the file share capabilities built into Windows based solutions. The CIFS protocol was not designed with wide area networks in mind – it transfers information across the network in small requests, resulting in very inefficient use of the network for large files. Small requests across the network are very sensitive to any latency on the connection which results in the end user experience suffering. Acceleration Systems intercepts CIFS requests to offsite resources and identifies when a large amount of content is about to be transferred. Instead of making small requests for the content across the Internet, Acceleration Systems moves bigger blocks of content into a pre-fetch buffer. Acceleration Systems then serves content from the pre-fetch buffer on the client side of the network instead of making further requests across the network for small amounts of data.

Acceleration Systems intercepts CIFS requests to offsite resources and identifies when a large amount of content is about to be transferred. Instead of making small requests for the content across the Internet, Acceleration Systems moves bigger blocks of content into a pre-fetch buffer. Acceleration Systems then serves content from the pre-fetch buffer on the client side of the network instead of making further requests across the network for small amounts of data.

Acceleration_Systems_technical_overview_of_bandwidth optimization_technology_CIFS_Optimization

Pre-fetching significantly decreases the number of requests across the network, and avoids the latency of the network affecting the end user experience. For example, the number of requests for a 2Mb file across a wide area network with 200ms latency decrease from 234 requests down to three requests with Acceleration Systems pre-fetching. This decreases a user’s wait time from over a minute to fifteen seconds.

HTTP Optimization

Acceleration Systems’ HTTP Optimization includes header identification and manipulation, NTLM authentication pooling, and HTTP de-chunking. These protocol optimizations focus on ensuring that the block caching and compression optimizations work effectively on the largest possible data within the network request.

HTTPS/SSL Optimization

Often connections across the Internet are encrypted using HTTPS/SSL encryption. Encryption ensures communication is private. To an observer it appears as if the requests contain random data even if the same requested document is being returned. This encryption can prevent pattern matching and compression optimizations built into a Bandwidth Optimization system from operating.

Acceleration Systems provides optimization for encrypted information – ensuring patterns are recognized between requests, and allowing compression to work effectively. Optimizing encrypted information involves providing the Acceleration Systems cloud server within the data center with the certificate of any encrypted application server. Certificates never move outside the data center. All data transferred from client to Acceleration Systems cloud servers on to the target application server will continue to be made over fully encrypted connections, ensuring the privacy of the communication is maintained.

Other Protocols

Acceleration Systems has various other optimizations that help identify and accelerate traffic for particular protocols. Other protocols currently supported include MAPI (via RPC or over HTTP), print servers, DoubleTake, iSCSI, RDP, SQL and VNC. If a protocol is not currently identified by Acceleration Systems, it will still be optimized through de-duplication and compression. Acceleration Systems is continuing to enhance our capabilities with new protocols at each release.

In-line Stream De-duplication

In-line Stream De-duplication (ILSDD) is a key technology of the Acceleration Systems product. ILSDD de-duplicates patterns of information as the information passes across the network. It is bi-directional, replacing patterns of data with cache references as information is sent to, or received from, cloud-based servers.

Acceleration_Systems_technical_overview_of_bandwidth optimization_technology_In-line_Stream-Deduplication

As Acceleration Systems is used to optimize connections to an application server, the ILSDD technology learns the patterns being transferred between users and the server. If any pattern is seen more than once, a reference to the pattern is sent across the network instead of the original block of data. This removes considerable traffic from the network for similar content.

The algorithms built into the ILSDD technology ensure that Acceleration Systems can quickly determine if a pattern within traffic passing across the network has previously been seen and can be replaced with a reference. The cache size available for ILSDD is configurable. The ILSDD technology ensures that patterns seen most frequently remain within the cache and patterns which are infrequently used are dropped from the cache. ILSDD can make use of inmemory and disk-based cache to ensure the maximum throughput possible.

In office scenarios, with a Remote Business Accelerator installed in the office, ILSDD provides shared caching for all users in the office, providing greater optimizations for all. Since the cache is based on content that would be sent or received by individual users, security of the content is always maintained. ILSDD works regardless of the type of traffic or application server, and detects patterns across multiple unrelated applications (e.g. if similar content is seen in an email attachment and a file share copy). ILSDD is extremely successful within collaboration scenarios where information is retrieved, changed and uploaded again. Since it is bi-directional, and protocol independent, ILSDD will replace data with references if, for example, a user downloads a file from a content management application and subsequently sends it within an email.


The final step in Acceleration Systems’ optimizations is to compress any remaining information which would be sent across the Internet. With modern data types like HTML and XML being text based – and with the frequent transfer of text-based content, compression plays an important part in removing bytes from the network by minimizing the data that needs to be transferred. Compression is also very effective on print traffic.

Optimization Effectiveness

Acceleration Systems’ various optimizations combine to remove a high percentage of bytes from the network, and improve the user experience to applications across remote connections. Acceleration Systems typically removes between 70 and 90% of information from the network, depending on the data and patterns of information passing across the network.

Acceleration_Systems_technical_overview_of_bandwidth optimization_benefits_by_protocol


Acceleration Systems’ unique cloud-based Bandwidth Optimization functionality provides significant acceleration of network connections, improving usability for users accessing applications remotely. The technology is designed to meet a very broad range of deployment needs including highly dispersed offices and large numbers of remote users. Protocol optimizations, bi-directional, block level de-duplication and compression combine to provide 70- 90% offload in typical scenarios. The algorithms are designed to provide optimizations in lowfootprint installations with commodity hardware such as our Remote Business Accelerator.