Menu
How big data is changing the game for backup and recovery

How big data is changing the game for backup and recovery

In today's distributed databases, getting a reliable snapshot of all those petabytes isn't easy

It's a well-known fact in the IT world: Change one part of the software stack, and there's a good chance you'll have to change another. For a shining example, look no further than big data.

First, big data shook up the database arena, ushering in a new class of "scale out" technologies. That's the model exemplified by products like Hadoop, MongoDB, and Cassandra, where data is distributed across multiple commodity servers rather than packed into one massive one. The beauty there, of course, is the flexibility: To accommodate more petabytes, you just add another inexpensive machine or two rather than "scaling up" and paying big bucks for a bigger mammoth.

That's all been great, but now there's a new sticking point: backup and recovery.

"Traditional backup products have challenges with very large amounts of data," said Dave Russell, a vice president with Gartner. "The scale-out nature of the architecture can also be difficult for traditional backup applications to handle."

Today's horizontally scalable databases do include some capabilities for availability and recovery, but typically they're not as robust as those IT users have become accustomed to, Russell added.

It's a problem that can leave large enterprises vulnerable when outages strike. But it's also where a new class of data-protection products is beginning to enter the picture.

Datos IO's RecoverX is one of those.

"If you have a traditional database like Oracle or MySQL, it's scale-up, and there's always the notion of a durable log," said Tarun Thakur, Datos IO's co-founder and CEO.

In such scenarios, a copy of that log is what constitutes a backup when problems arise.

In the world of today's next-generation databases -- where data is distributed across small machines -- it's not quite so simple.

"There is no concept of a durable log because there is no master -- each node is working on its own stuff," Thakur explained. "Different nodes could get different rights, and every node has a different view of an operation."

That's in part because of a trade-off that's been required to accommodate what's commonly referred to as the "three V's" of big data -- volume, velocity, and variety. Specifically, to offer scalability while accommodating the crazy amounts of diverse data flying at us at ever-more-alarming speeds, today's distributed databases have departed from the "ACID" criteria generally promised by traditional relational databases. Instead, they've adopted what are known as "BASE" principles.  

It's a critical distinction. Most pertinent is that where traditional databases promise strong consistency throughout -- that's the "C" in ACID -- distributed ones strive instead for what's called "eventual consistency." Updates will be reflected in all nodes of the database sooner or later, but there's a time lag.

"If you need scalability, you need to give up consistency -- you have to give up one or the other," Thakur said.

That makes it tough to get a reliable snapshot of the big picture for point-in-time recovery. Not only is it more difficult to track which data might have moved where in a distributed database at any given moment, but the resiliency features that often come "baked" into newer distributed databases -- replication, for example -- won't protect you if data gets corrupted, said Simon Robinson, a research vice president with 451 Research.

"You just replicate that corrupted data," he said.

Earlier this month, Datos IO launched RecoverX to address those concerns through features including what it calls scalable versioning and semantic deduplication. The result is cluster-consistent backups that are both space-efficient and available in native formats, the company says.

Souvik Das, who until recently was CTO and managing vice president of engineering with CapitalOne Auto Finance, has felt the backup crunch first-hand.

After years of using traditional databases, CapitalOne underwent a "massive transformation" a few years back that included rolling out new distributed technologies such as Cassandra, said Das, who is now senior vice president of engineering at healthcare-focused startup Grand Rounds.

That meant looking for a new strategy for backup and recovery.

"Most of the backup vendors and software are typically tuned to the type of systems that they're backing up," he explained.

Using an older-style backup product with a newer distributed database could spell trouble, he said.

"Either that software would completely fail because it has no idea how to back up the new data stores, or it would work in a very suboptimal way," Das said. "We knew going in that we would have to have different backup solutions."

CapitalOne has been evaluating Datos IO as well as Talena, another major player in the space, Das said.

Vendors of more traditional backup products are gradually adjusting their own technologies for big data as well.

"It usually takes the incumbent backup vendors some time to support the newer technologies," 451 Research's Robinson said.

"Rewind 10 years and it was very difficult initially to easily do backups for VMware virtual machines," he added. "This opened the door for players like Veeam to enter and steal the VM backup market from under the noses of the incumbents."

Follow Us

Join the New Zealand Reseller News newsletter!

Error: Please check your email address.

Featured

Slideshows

Reseller News launches inaugural Hall of Fame lunch

Reseller News launches inaugural Hall of Fame lunch

Reseller News welcomed 2015 and 2016 inductees - Darryl Swann, Dave Rosenberg, Gary Bigwood, Keith Watson, Mike Hill and Scott Green - to the inaugural Reseller News Hall of Fame lunch, held at the French Cafe in Auckland. The inductees discussed how the channel can collectively work together to benefit New Zealand, the Kiwi skills shortage and the future of the industry. Photos by Maria Stefina.

Reseller News launches inaugural Hall of Fame lunch
Educating from the epicentre - Why distributors are the pulse checkers of the channel

Educating from the epicentre - Why distributors are the pulse checkers of the channel

​As the channel changes and industry voices deepen, the need for clarity and insight heightens. Market misconceptions talk of an “under pressure” distribution space, with competitors in that fateful “race for relevance” across New Zealand. Amidst the cliched assumptions however, distribution is once again showing its strength, as a force to be listened to, rather than questioned. Traditionally, the role was born out of a need for vendors and resellers to find one another, acting as a bridge between the testing lab and the marketplace. Yet despite new technologies and business approaches shaking the channel to its very core, distributors remain tied to the epicentre - providing the voice of reason amidst a seismic industry shift. In looking across both sides of the vendor and partner fences, the middle concept of the three-tier chain remains centrally placed to understand the metrics of two differing worlds, as the continual pulse checkers of the local channel. This exclusive Reseller News Roundtable, in association with Dicker Data and rhipe, examined the pivotal role of distribution in understanding the health of the channel, educating from the epicentre as the market transforms at a rapid rate.

Educating from the epicentre - Why distributors are the pulse checkers of the channel
Kiwi channel reunites as After Hours kicks off 2017

Kiwi channel reunites as After Hours kicks off 2017

After Hours made a welcome return to the channel social calendar last night, with a bumper crowd of distributors, vendors and resellers descending on The Jefferson in Auckland to kickstart 2017. Photos by Maria Stefina.

Kiwi channel reunites as After Hours kicks off 2017
Show Comments