We are living in an age of unprecedented data growth. In fact, the IDC recently predicted that by 2025 the world’s data will reach a total volume of 175 zettabytes (ZB), up from its current size of 33ZB. That represents a compounded annual growth rate of 61 percent. Whereas much of the data generated in past decades has been structured – that is, information that conforms to a pre-defined model, such as what would be found in a spreadsheet – most of the data that will contribute towards the IDC’s growth predictions will be unstructured in nature. It will be information that does not conform to existing models. In fact, the world we are living in is already full of this unstructured data, including photos, videos, presentations, web copy, and other rich media.
Despite this vivid shift in both the type and amount of data generated, many individual enterprises still rely on conventional storage modes like tape libraries or Storage Area Network (SAN) and Network Attached Storage (NAS) systems. As the explosion of unstructured data only grows bigger, we believe it will become increasingly difficult to scale these older storage methods upwards in an affordable and efficient manner. In this blog, we examine why enterprises ought to consider object storage as a scalable, flexible way of modernizing their data storage to stay competitive while keeping up with the increased data demands of their staff and consumers.
Scalability Challenges of Traditional Storage Methods
For the last two decades, SAN (and to a lesser extent NAS) systems have been a go-to solution for enterprises requiring large, centralized databases for storing and retrieving structured data. Whereas SAN systems generally make use of block-based storage, NAS systems typically deploy file level storage. Briefly, with block storage, the SAN application divides files into individual blocks of data, assigns them identifiers and then decides where to store them on mediums like RAID (redundant array of independent disks) and virtual machines. With file storage, information is organized in a hierarchical structure and retrieved via different paths that are available to users with appropriate access rights. Unfortunately, the pace at which enterprises are generating data is beginning to cause problems with scaling conventional block and file-based storage systems.
According to 451 Research, a majority of modern enterprises already handle data stores beyond 1 Petabyte in size. Over time, as their IT teams add more blocks and further build upon already dense file hierarchies, database queries run slower and business operations begin to suffer. Some organizations attempt to mitigate these issues by deploying scale-out file systems or leveraging pure public cloud storage. However, when dealing with large data sets past the petabyte range, the first option can get expensive as organizations invest in the commodity drives needed both to store and backup their information. While pure public cloud storage may seem more attractive, enterprises following this route can be hit with ingress/egress fees, network usage costs, and data privacy issues.
Enter object storage. Somewhat ironically, major cloud providers like Amazon (S3 Storage Services) and Microsoft (Azure Blob Storage) have been making use of this technology for years. Now, as data volumes continue to grow, object storage’s ability to leverage a file’s metadata offers enterprises an efficient, flexible, affordable and highly available way to store and process data whenever and wherever it is needed.
Object Storage: The Scalable Solution
Object storage is a data management technique that stores files as objects, along with associated metadata, in a flat address space rather than as blocks or files in a hierarchical system. Every object gets a specific identifier (object ID) that can be used to retrieve it straight from its location in the storage pool. Unlike block storage, where data blocks are stored independently of each other or file storage where paths must be specified in order to sift through databases, object storage provides a fast, reliable way to place and retrieve unstructured data. As well, object storage’s use of a flat address space makes it inherently scalable – enterprises face virtually no limit on how much information can be stored.
Unlike conventional storage methods, object storage allows users to input and customize metadata, the information that describes a file’s characteristics, as desired. This ability to leverage a file’s metadata makes Object Storage is extremely useful for your administrators. Eliminating the need for databases makes it simple to tag, log and store files as well as sync and share them even across multiple geographic regions. From sharing medical imagery to integrating processed video content into content delivery networks (CDNs), object storage can suit a variety of purposes for enterprises across multiple verticals.
The eStruxture Advantage
At eStruxture, we offer a multi-purpose, distributed object storage system based on the Hitachi Content Platform. Our solution has the following significant benefits when compared to other popular products on the market:
- As a pan-Canadian data center provider, our solution is 100% data sovereign, eliminating any concerns over privacy.
- Our customers do not pay ingress/egress fees, enabling them to save money and pay only for the space they use.
- We offer true data shredding capabilities – something that is not fully possible with public cloud providers.
- We maintain full Amazon S3 compatibility so that any application capable of interacting with the Amazon RESTful storage API can be deployed on our object storage service.
In terms of data security and protection, our system makes use of firewalled service delivery perimeters with intrusion detection capabilities. With all data encrypted, immutable WORM policies applied to data buckets and all interactions logged and retained, you can securely share your data between application environments. In fact, our system generates unique keys for each data access client. With security and freedom in one solution, users can harness data growth and bridge traditional and emerging technologies to make their data available anywhere, anytime.
In short, as enterprises continue to generate increasing amounts of data, it is time to look beyond conventional block and file-based storage systems and the use of SAN, NAS, and tape. While older storage methods do still serve a purpose, the amount of unstructured data being produced by businesses today is pointing to a need for increased adoption of object storage as a secure, resilient and affordable approach to data storage and management.
Here at eStruxture, we encourage enterprises to take stock of the unstructured data they are producing, evaluate the costs of continuing to use current storage solutions and ask us about the savings they could achieve by leveraging our object storage system.