In 2012 I did a blog post on Facebook going public. The premise of the piece wasn’t about the fad that surrounds social media or that a social media company was going public, it was that the group of IT practitioners that manage Facebook’s storage infrastructure had deployed something pretty amazing.
In 2009 I quoted a few simple stats:
- Facebook had about 1 billion users - remember, each user has a mailbox for messages
- Users were uploading about 250M photos per day
Today,
- Facebook has 1.55 billion users - and IT is managing even more mailboxes
- They are uploading 900M photos per day
Recent data suggests that the Facebook data warehouse is approximately 300PB today.
Deploying and managing this much capacity is not a simple task. Yet in a 2012 quote on TechCrunch by VP of Engineering Jay Parikh explained why keeping your data is important, “Big data really is about having insights and making an impact on your business.” The challenge today is that the cost of storing that much data is impeding the competitive business process. In other words the storage costs more than the servers, applications, management and people who process the data all put together.
I believe most customers want to have the type of infrastructure to be able to manage this much capacity but it's just not practical. The notion of trying to save money and build your own storage infrastructure out of off the shelf hardware and glue it all together like the team at Facebook is an impossible task. The whole goal behind trying to do something like this would be to save money but in the end, not only would it cost more, the solution wouldn't meet the needs of the business.
The other major challenges is the fact that you cannot build this type of an infrastructure with the off-the-shelf hardware of today. In order to develop this type of an infrastructure you need a team of developers like the team Facebook has. However, even with all of the code that the Facebook developers have built to manage and store their data, The environment is highly custom and really only applicable to their type of work.
Most businesses can afford to have this type of staff, it detracts from their competitive advantage. Competitive advantage is more than just the results of the analytics that they can derive, it's also the cost of doing the analytics that plays a roll.
So the next step is for businesses to look to enterprise storage to solve their challenge, but which array do you choose? Inevitably, in order to balance your own costs with the required capacity, along with the performance necessary to perform the analytics fast enough for the business to be competitive, you have to make compromises. So how does one choose what storage characteristic they need to sacrifice? Capacity, reliability, performance? Something needs to give, and it finally has.
INFINIDAT has built a new storage array called InfiniBox that has over 100 new storage patents designed specifically to change the way in which data it stored and persisted. They have even gone so far is to rewrite RAID algorithms in order to create the most reliable, densest storage array available (2PB in a single floor tile). They have created the new mesh architecture for their array that combines DRAM, SSD and spinning media to deliver a million IOPS. They have also developed a management interface that allows the boxed to be deployed and managed more quickly and more simply than any other array on the market. It's innovative design boasts just 8Kw of power usage per array, even the environmentals help the business's bottom line and competitive advantage. Finally INFINIDAT has multiple support touch points to help ensure true client satisfaction.
Now businesses do not need a costly team of storage experts to help them develop a storage infrastructure. In addition businesses do not need to compromise on the different storage characteristics they require in order to drive value. Now businesses no longer need to hire a data hygienist in order to figure out what data to throw away, because throwing data away ruins the competitive advantage.