Check out our companion blog!
Sept. 6, 2022

What are RTO and RPO & how do they drive backup design?

What are RTO and RPO & how do they drive backup design?

If you don't meet your company's Recovery time objective (RTO) and Recovery Point Objective with your backup design, nothing else matters. Seriously. No one cares if you can back up – only that you can restore in a timeframe they consider reasonable. The only way that's going to happen is if you agree to these times UPFRONT. In this episode in our new back to basics series, we'll jump right into this extremely important topic. We'll explain what RTO & RPO are, what recovery time and recovery point actual are, and how they relate to RTO & RPO. We'll also explain how to get your company to agree to them.

Mentioned in this episode:

Interview ad

Transcript
W. Curtis Preston:

Hi, and welcome to Backup Central's Restore it all podcast.

W. Curtis Preston:

I'm your host w Curtis Preston, AKA Mr.

W. Curtis Preston:

Backup.

W. Curtis Preston:

And I have with me a guy that I am absolutely positive.

W. Curtis Preston:

My, my dad, when I was younger would definitely call a hippie.

W. Curtis Preston:

How's it going, prasanna?.

Prasanna Malaiyandi:

I was trying to figure out where you were taking that.

Prasanna Malaiyandi:

And I was very afraid where that was going.

W. Curtis Preston:

Yeah, my dad would've with that hair that you got

W. Curtis Preston:

going on, my dad would totally have called you a hippie back in the day.

W. Curtis Preston:

And this is well, actually pretty sure both of my parents and mind you,

W. Curtis Preston:

I don't know if I've ever told you.

W. Curtis Preston:

Did I tell you that?

W. Curtis Preston:

I, I wasn't allowed to wear jeans until like, I, I didn't own a pair of

W. Curtis Preston:

jeans until I was 18 years old and it's because genes were what hippies wore.

Prasanna Malaiyandi:

Interesting.

W. Curtis Preston:

That was.

W. Curtis Preston:

Corduroy was my, um, was my, uh, yeah, exactly.

W. Curtis Preston:

Um, you know, the, you know, you could start a fire down there.

W. Curtis Preston:

Um but yeah, you, you know, you,

Prasanna Malaiyandi:

more people don't wear corduroy.

Prasanna Malaiyandi:

Like they're super comfy.

Prasanna Malaiyandi:

They're nice material.

W. Curtis Preston:

what's amazing is they're pretty warm though.

W. Curtis Preston:

And I did this in Florida, right?

Prasanna Malaiyandi:

I don't know about Florida if you'd want that there.

Prasanna Malaiyandi:

Honestly though, I think even today, people would still call me a hippie.

Prasanna Malaiyandi:

So it's all good.

Prasanna Malaiyandi:

Curtis, you don't have to go back to your dad's generation back in the

Prasanna Malaiyandi:

day, calling me a hippie back then.

Prasanna Malaiyandi:

So,

Prasanna Malaiyandi:

how about this?

Prasanna Malaiyandi:

If we can get five comments on our podcast, right?

Prasanna Malaiyandi:

Positive comments in the next month.

Prasanna Malaiyandi:

No, sorry.

Prasanna Malaiyandi:

In the next two weeks from when this goes live,

W. Curtis Preston:

all right.

Prasanna Malaiyandi:

Curtis will grow a beard for the next

Prasanna Malaiyandi:

three months for the next three

W. Curtis Preston:

let's see apple,

Prasanna Malaiyandi:

Yeah.

W. Curtis Preston:

pulling up to see.

W. Curtis Preston:

All right.

W. Curtis Preston:

We have to be specific.

W. Curtis Preston:

Uh, alright.

W. Curtis Preston:

Right now we have 16.

W. Curtis Preston:

Ratings on, on the,

Prasanna Malaiyandi:

so I'll make it a little harder.

Prasanna Malaiyandi:

So if we get to TW hold on, I said five before.

W. Curtis Preston:

yeah.

Prasanna Malaiyandi:

Okay.

Prasanna Malaiyandi:

So, okay.

Prasanna Malaiyandi:

So if we get nine, how about that?

W. Curtis Preston:

Well, if we get nine, if we get nine new ratings and comments

W. Curtis Preston:

I'll grow a beard until Christmas.

Prasanna Malaiyandi:

Okay.

W. Curtis Preston:

I, I, I can't commit to after that, but, but I don't see

W. Curtis Preston:

that I don't see that happening.

Prasanna Malaiyandi:

And just, just, just, just watch Curtis, just be careful.

W. Curtis Preston:

uh,

Prasanna Malaiyandi:

this is, this is nine from when this episode goes live, correct.

W. Curtis Preston:

yeah, yeah.

W. Curtis Preston:

Which, which will be sometime in September or, well, it might be in August.

W. Curtis Preston:

We'll see.

W. Curtis Preston:

Um, yeah, so right now, there's right now there's 16 ratings on the, uh, on the, uh,

Prasanna Malaiyandi:

just, yep.

Prasanna Malaiyandi:

Just, just, just wait, just wait.

W. Curtis Preston:

All right.

W. Curtis Preston:

Well, now I'm scared, but, uh, but speaking of ratings, I'll throw

W. Curtis Preston:

out our, our podcast, our, our, our disclaimer, uh, Prasanna and

W. Curtis Preston:

I work for different companies.

W. Curtis Preston:

He works for Zoom.

W. Curtis Preston:

I work for Druva and this is not a podcast of either company.

W. Curtis Preston:

The opinions that you hear are ours.

W. Curtis Preston:

Also rate us at ratethispodcast.com/restore.

W. Curtis Preston:

And, uh, you know, if, if, if you like what we're talking

W. Curtis Preston:

about, if, if, you know, if you.

W. Curtis Preston:

Somebody who's been listening to the podcast.

W. Curtis Preston:

We know you're out there.

W. Curtis Preston:

Just reach out to me @wcpreston on Twitter, or w Curtis Preston at Gmail.

W. Curtis Preston:

And, um, you know, we'll, we'll get you on the podcast.

W. Curtis Preston:

We'll talk about your favorite subject.

W. Curtis Preston:

We'll even keep you anonymous if you want.

W. Curtis Preston:

Right.

W. Curtis Preston:

We'll give you a fake name.

W. Curtis Preston:

Like we've had Harry Potter and Ron Weasley on here.

W. Curtis Preston:

Um, you know, it's all.

Prasanna Malaiyandi:

couple mystery guests without any

W. Curtis Preston:

Yeah, yeah, yeah.

W. Curtis Preston:

We've had, yeah, we've had, yeah.

W. Curtis Preston:

Where we didn't even give him a name.

W. Curtis Preston:

Right.

W. Curtis Preston:

Um, so, uh, so that's all good, right?

W. Curtis Preston:

That way you can, you can speak to your heart's content and not think, not worry

W. Curtis Preston:

about what your employer thinks about.

W. Curtis Preston:

It we'll even disguise your voice.

W. Curtis Preston:

So, um, I thought this week we would kind of go back to.

W. Curtis Preston:

Basic, but incredibly important topic.

W. Curtis Preston:

And that is just to the concepts that are RTO and RPO, which of course

W. Curtis Preston:

for those who don't already know

Prasanna Malaiyandi:

or office RTO is a

W. Curtis Preston:

So recovery time, objective and recovery point objective.

W. Curtis Preston:

And then also we should talk about, um, you know, RTA and RPA and how those

W. Curtis Preston:

are related, but completely different.

W. Curtis Preston:

So let's first talk about, and, and, and I guess what I'm gonna make the

W. Curtis Preston:

title of this is why RTO and RPO are, you know, what, what did I say?

W. Curtis Preston:

I was gonna, what did I put here?

W. Curtis Preston:

Oh, I didn't put, what did I say in the message Prasanna?

W. Curtis Preston:

I was quite eloquent.

W. Curtis Preston:

What, what did I say the title should be

Prasanna Malaiyandi:

why RTO and RPO should drive all backup design.

W. Curtis Preston:

done?

W. Curtis Preston:

Right?

W. Curtis Preston:

Because let me, let me ask you a question Prasanna.

W. Curtis Preston:

Do backups matter.

Prasanna Malaiyandi:

No,

W. Curtis Preston:

Does anyone care if you back up?

W. Curtis Preston:

No one cares.

W. Curtis Preston:

If you back up, they only care.

W. Curtis Preston:

Yeah.

W. Curtis Preston:

The only thing is what

Prasanna Malaiyandi:

is restoring data, and if you fail to restore

Prasanna Malaiyandi:

data, there's a high likelihood.

Prasanna Malaiyandi:

Your job might be out gone.

Prasanna Malaiyandi:

So.

W. Curtis Preston:

Yeah, and it, and it, and I would say, and I, I feel

W. Curtis Preston:

so strongly about this and, and by the way, I'm, I'm speaking to the

W. Curtis Preston:

I'm, I'm not speaking to the choir.

W. Curtis Preston:

I'm speaking to old me.

W. Curtis Preston:

I spent the first, I don't know how many years of my backup career, not

W. Curtis Preston:

really knowing much about RTO and RPO.

W. Curtis Preston:

And I, I kind of used the concepts, I suppose.

W. Curtis Preston:

But I didn't use them to drive backup design.

W. Curtis Preston:

I didn't use them to set expectations with my, you know, with my customers.

W. Curtis Preston:

Right.

W. Curtis Preston:

I was at a very large bank and we had all kinds of expectations.

W. Curtis Preston:

Um, And, and, and I know, you know, and you know, you've worked at companies

W. Curtis Preston:

where you've got customers that have expectations and you're, well, you

W. Curtis Preston:

know, you, you and I are both married, so many arguments that you have as, as

W. Curtis Preston:

a marriage couple, as a married couple comes from what mismatched expectations.

W. Curtis Preston:

Yeah.

W. Curtis Preston:

Um, and, and I think that's, that's go ahead.

W. Curtis Preston:

Go ahead.

W. Curtis Preston:

What were you gonna

Prasanna Malaiyandi:

Now, and I was just thinking back to, I know we've

Prasanna Malaiyandi:

had Jeff Rocklin on this call or on the podcast and in your book, right.

Prasanna Malaiyandi:

That you wrote modern data protect.

W. Curtis Preston:

Yep.

Prasanna Malaiyandi:

by O'Reilly um, in that book also, right?

Prasanna Malaiyandi:

There's that entire chapter of working with your stakeholders,

Prasanna Malaiyandi:

setting expectations, right?

Prasanna Malaiyandi:

Understanding and getting agreement on, Hey, this is what it is, because

Prasanna Malaiyandi:

like you said, a lot of the time it comes down to expectations aren't

Prasanna Malaiyandi:

agreed to, and aren't set upfront and therefore, when something goes wrong,

Prasanna Malaiyandi:

Inevitably something does right.

Prasanna Malaiyandi:

Then everyone's like, oh, that's not what I thought.

Prasanna Malaiyandi:

And oh, I thought I would get my data back tomorrow and oh, why am I losing data?

Prasanna Malaiyandi:

Right.

Prasanna Malaiyandi:

And it's like, oh, because these things weren't clearly

Prasanna Malaiyandi:

documented, discussed upfront.

Prasanna Malaiyandi:

Like you said, Curtis, when designing those backup systems,

W. Curtis Preston:

Exactly.

W. Curtis Preston:

And, and so, you know, when you think about the ways that you can

W. Curtis Preston:

get in trouble as a backup admin, one of them is clearly you either

W. Curtis Preston:

the restore didn't complete the.

W. Curtis Preston:

In the expected amount of time and the restore lost more data than was expected.

W. Curtis Preston:

Now I'm just saying expected, I'm saying it that way specifically

W. Curtis Preston:

because it it's, it's what they were expecting, not what you were expecting.

W. Curtis Preston:

right.

W. Curtis Preston:

You probably always knew how long it was gonna take, but if the

W. Curtis Preston:

powers that be well, you may or may not know we did have an episode.

W. Curtis Preston:

Why, uh, why restore is often.

W. Curtis Preston:

Usually it takes longer than the backup.

W. Curtis Preston:

Right.

W. Curtis Preston:

Uh, and if you haven't listened to that episode, I would highly

W. Curtis Preston:

recommend it because it goes into it.

W. Curtis Preston:

Go ahead.

Prasanna Malaiyandi:

oh yeah, like you were saying, it goes into all the details

Prasanna Malaiyandi:

about why restores can be slower than your backups and issues around that.

Prasanna Malaiyandi:

Um, I was also going to add that.

Prasanna Malaiyandi:

Even another challenge that you see with restores is some people.

Prasanna Malaiyandi:

And I know Curtis, we always talk about this, verify your restores, right?

Prasanna Malaiyandi:

Verify that your backups are done.

Prasanna Malaiyandi:

Right.

Prasanna Malaiyandi:

But some people have never actually done a restore.

Prasanna Malaiyandi:

So they can't tell you, or they've done such a small restore that they don't

Prasanna Malaiyandi:

know how long in real life is it gonna take to bring my Oracle database back up

Prasanna Malaiyandi:

and running to the latest point in time.

W. Curtis Preston:

They've often done sort of functional restores, but not

W. Curtis Preston:

per, but not performance restore tests.

W. Curtis Preston:

Right.

W. Curtis Preston:

I, I, I remember when I was at the bank, we did something like.

W. Curtis Preston:

Like we did a handful of restores every single day because we didn't

W. Curtis Preston:

have snapshots back in the day.

W. Curtis Preston:

There were the number one reason for restores is still

W. Curtis Preston:

still human, human action.

W. Curtis Preston:

Right?

W. Curtis Preston:

Not in fact, I would say it's even more, uh, more so today than it was.

W. Curtis Preston:

20/30 years ago, because now we have raid and erasure coding and highly

W. Curtis Preston:

reliable drives like SSD drives versus the rotational drives that we were,

W. Curtis Preston:

uh, that we used for so many years.

W. Curtis Preston:

I would say that, that at this point, like 99% of the time that you're gonna

W. Curtis Preston:

do a restore is due to some kind of action of some kind of human, some

Prasanna Malaiyandi:

I was, I was just

W. Curtis Preston:

but.

Prasanna Malaiyandi:

you seen a study about that?

Prasanna Malaiyandi:

That'd be an interesting stat.

Prasanna Malaiyandi:

I wonder if there is an industry stat talking about what percentage

Prasanna Malaiyandi:

of restores is actually user

W. Curtis Preston:

Yeah, I, I just, I just think about like the,

Prasanna Malaiyandi:

Yeah.

Prasanna Malaiyandi:

Yeah.

Prasanna Malaiyandi:

The day to day

W. Curtis Preston:

center used to be.

W. Curtis Preston:

Right.

W. Curtis Preston:

We, we, when I started, we had servers running on a disk drive.

W. Curtis Preston:

A disk drive.

W. Curtis Preston:

You know, you had the OS disk drive, you had the application disk

W. Curtis Preston:

drive, and then you had one or more data disk drives, disk drives, not

W. Curtis Preston:

LUNs, not LUNs on a RAID array.

W. Curtis Preston:

What's a RAID array.

W. Curtis Preston:

Right.

W. Curtis Preston:

Um, and it was all obviously rotational discs.

W. Curtis Preston:

And we went through you.

W. Curtis Preston:

I'm sure you know, nothing of this, but there was.

W. Curtis Preston:

There was a big HP recall.

W. Curtis Preston:

We had HP, a lot of HP servers and it was an HP disc recall

W. Curtis Preston:

because it was, they were leaking.

W. Curtis Preston:

Swag oil.

W. Curtis Preston:

I don't even know what that means, but swag oil, they were leaking swag oil onto

W. Curtis Preston:

the platters and thus creating data loss.

W. Curtis Preston:

We, we called them the Valdees discs.

W. Curtis Preston:

Um, for,

Prasanna Malaiyandi:

on Exxon.

Prasanna Malaiyandi:

Valdis the oil

W. Curtis Preston:

based on Exxon Valdis apologies if anybody works

W. Curtis Preston:

in that, you know, industry, but, uh, that was what we had back then.

W. Curtis Preston:

That that just doesn't happen.

W. Curtis Preston:

Right.

W. Curtis Preston:

I mean, indiv, if an individual drive, whether it's first off SSDs fail way

W. Curtis Preston:

less often than, than rotational drives.

W. Curtis Preston:

Right.

W. Curtis Preston:

And, and if they do they're in an array and it's just replaced, it's,

W. Curtis Preston:

it's, it's like replaced right away.

W. Curtis Preston:

Right.

W. Curtis Preston:

You have hot swappable drives.

Prasanna Malaiyandi:

I think though the challenge is if you think about the

Prasanna Malaiyandi:

types of scenarios and use cases, right?

Prasanna Malaiyandi:

When I think about like a user accidentally deleted something or some,

Prasanna Malaiyandi:

a use case like that, the amount of data I'm restoring, isn't a large amount.

Prasanna Malaiyandi:

Right.

W. Curtis Preston:

well, but, but it's not just the, it's not just the user.

W. Curtis Preston:

Right?

W. Curtis Preston:

Notice the way I, where I said it, I said that the action of some human, right.

W. Curtis Preston:

That could be an admin dropping a VM, it could be a hacker.

Prasanna Malaiyandi:

That's okay.

Prasanna Malaiyandi:

That's what I was gonna get to right.

Prasanna Malaiyandi:

Is yeah.

Prasanna Malaiyandi:

The ransomware style use cases where yes, that is a smaller percentage

Prasanna Malaiyandi:

of probably overall restores.

Prasanna Malaiyandi:

But if I look at the amount of data recalled during those scenarios versus

Prasanna Malaiyandi:

typical user restore behaviors, right.

Prasanna Malaiyandi:

Um,

W. Curtis Preston:

Oh, I see.

W. Curtis Preston:

Oh, that

Prasanna Malaiyandi:

do on the much larger end.

Prasanna Malaiyandi:

Right.

Prasanna Malaiyandi:

Which are the RTOs that you need to be considering, right?

W. Curtis Preston:

So you're saying that you think that if you look at

W. Curtis Preston:

data per reason, the amount of data of restored versus the number of restores.

W. Curtis Preston:

You're saying, if you look at the amount of data restored versus the reasons that

W. Curtis Preston:

you're restored, you think that the vast majority will be, uh, ransomware attacks.

W. Curtis Preston:

I, I, I can't, I can't dispute that.

W. Curtis Preston:

I would say ransomware attacks and disasters.

W. Curtis Preston:

Um, but, uh, interesting.

W. Curtis Preston:

Yeah.

W. Curtis Preston:

So, so let's go.

Prasanna Malaiyandi:

of things, oh, sorry.

Prasanna Malaiyandi:

Just quickly on that from the restore time objective, that's why it's important

Prasanna Malaiyandi:

to understand and try to figure out a way to extrapolate, to get to that

Prasanna Malaiyandi:

sort of full RTO restore scenario.

Prasanna Malaiyandi:

So you understand the performance there as well.

W. Curtis Preston:

Yeah.

W. Curtis Preston:

You

Prasanna Malaiyandi:

it an application or a bunch of VMs or whatever else it is

W. Curtis Preston:

you know, this is gonna sound like a non sequitor,

W. Curtis Preston:

but, but, um, I'm pulling up a, I'm pulling up a scene from the west wing.

W. Curtis Preston:

I don't know.

W. Curtis Preston:

Did you ever watch the west wing?

Prasanna Malaiyandi:

Nope.

W. Curtis Preston:

The west wing is an amazing show.

W. Curtis Preston:

Um, and, um, you know, in this house we've seen, my wife has

W. Curtis Preston:

seen the entire west wing at least four times the entire series.

W. Curtis Preston:

And there's this scene in there where the president played by Martin.

Prasanna Malaiyandi:

short.

W. Curtis Preston:

was gonna say Martin short, it's not Martin short Martin.

W. Curtis Preston:

Martin she, okay.

W. Curtis Preston:

Yeah.

W. Curtis Preston:

Um, yeah, so the president played by Martin sheen, uh, decides on

W. Curtis Preston:

his next Supreme court justice, who was Edward James Almos.

W. Curtis Preston:

And there's.

W. Curtis Preston:

There's this moment, he goes, so when's he gonna get here?

W. Curtis Preston:

And he goes well in a couple of days, what a couple of days, like, normally

W. Curtis Preston:

again, this is the expectation thing.

W. Curtis Preston:

Normally when somebody's nominated for a position like Supreme court,

W. Curtis Preston:

they hop on the plane that moment.

Prasanna Malaiyandi:

Yeah.

W. Curtis Preston:

But, but the, the, the Almos character decides to drive down.

W. Curtis Preston:

Uh, through he lives in Maine and he's gonna drive down and stop in

W. Curtis Preston:

Connecticut for some, for some antiquing.

W. Curtis Preston:

Um, so there's just, this, this is the thing it's like, what's the

W. Curtis Preston:

expectation versus what actually happens.

Prasanna Malaiyandi:

Yep.

W. Curtis Preston:

So, so, and, and I, and I would say that the bigger, the

W. Curtis Preston:

restore, the greater the expectations.

Prasanna Malaiyandi:

Yep.

W. Curtis Preston:

And so what you have to do, what you must do, and if you have

W. Curtis Preston:

not done this yet, you must do this.

W. Curtis Preston:

Now that is to decide per application on an RTO and an RPO.

W. Curtis Preston:

And, and then, and by the way, this is an ITER, an iterative

W. Curtis Preston:

process, which I think we'll, we'll probably talk about at the end here.

W. Curtis Preston:

How, how do you do this?

W. Curtis Preston:

Because, and I know we've talked about this on the, on the podcast

W. Curtis Preston:

before is if you ask the typical business unit, what RTO do they want?

W. Curtis Preston:

They will say zero, right?

W. Curtis Preston:

How fast do you wanna restore it immediately?

W. Curtis Preston:

How much data do you wanna lose?

W. Curtis Preston:

None

Prasanna Malaiyandi:

How much are you willing to spend?

W. Curtis Preston:

none , their answers are always the same.

Prasanna Malaiyandi:

Yep.

Prasanna Malaiyandi:

And I think though, going back to what you said, right?

Prasanna Malaiyandi:

This is where.

Prasanna Malaiyandi:

It's not so much that as a backup person, you decide what the RTO and RPOs are.

Prasanna Malaiyandi:

I think

W. Curtis Preston:

You absolutely do not do

Prasanna Malaiyandi:

yeah.

Prasanna Malaiyandi:

You have to have that discussion with the business stakeholders to be

Prasanna Malaiyandi:

like, okay, what are you expecting?

Prasanna Malaiyandi:

And like you said, right, you ask them and it comes back to sort of dollars, right?

Prasanna Malaiyandi:

Because Hey, if you want that zero RTO, zero RPO, zero data loss, right.

Prasanna Malaiyandi:

That is going to be a pretty penny.

Prasanna Malaiyandi:

And is that really needed by your application or.

Prasanna Malaiyandi:

Can you

W. Curtis Preston:

It.

W. Curtis Preston:

Yeah.

Prasanna Malaiyandi:

yeah, I'm okay.

Prasanna Malaiyandi:

If it takes a weekend to bring back up, it's not mission critical.

Prasanna Malaiyandi:

And so that's fine.

Prasanna Malaiyandi:

And we're not losing a lot of downtime.

W. Curtis Preston:

Yeah.

W. Curtis Preston:

The answer to your last question is almost never, right.

W. Curtis Preston:

Meaning almost never does the application need zero and zero, right?

W. Curtis Preston:

Unless it's like a financial trading firm or

Prasanna Malaiyandi:

Exactly.

Prasanna Malaiyandi:

Yeah.

W. Curtis Preston:

Yeah.

W. Curtis Preston:

They are there right on the opposite end.

W. Curtis Preston:

I've be I've.

W. Curtis Preston:

I worked at a paper mill.

W. Curtis Preston:

Their, their RTO was two weeks

Prasanna Malaiyandi:

Yeah.

Prasanna Malaiyandi:

And

W. Curtis Preston:

their RPO was two weeks as well.

Prasanna Malaiyandi:

And I would say for the companies that have

Prasanna Malaiyandi:

a zero RTO, zero RPO, they're not talking to the backup team.

Prasanna Malaiyandi:

They're probably talking to the storage infrastructure team.

Prasanna Malaiyandi:

The compute team, right.

Prasanna Malaiyandi:

Backup is just sort of like a, okay.

Prasanna Malaiyandi:

If everything else fails, it's the last line of defense.

Prasanna Malaiyandi:

It's not the first place I go in order to recover.

W. Curtis Preston:

Yeah.

W. Curtis Preston:

And, and by the way, that brings up a topic, which we

W. Curtis Preston:

should cover in this episode.

W. Curtis Preston:

And that is, should there be different RTOs and RPOs based on what happened?

W. Curtis Preston:

Okay.

W. Curtis Preston:

I, I would argue that it, it depends it depends.

W. Curtis Preston:

Okay.

W. Curtis Preston:

So, so let's talk about RTO and RPO.

W. Curtis Preston:

What are they?

W. Curtis Preston:

So Prasanna, what is R T O

Prasanna Malaiyandi:

So RTO is basically recovery time objective.

Prasanna Malaiyandi:

It's basically an objective telling you if you needed to recover a data set, how

Prasanna Malaiyandi:

long will it take you to bring it back?

W. Curtis Preston:

Right.

Prasanna Malaiyandi:

And the key here is it's not in, this is where

Prasanna Malaiyandi:

I like to differentiate versus what a lot of other people, it's

Prasanna Malaiyandi:

not just bringing back your data.

Prasanna Malaiyandi:

It's actually bringing back your application to a good known state.

W. Curtis Preston:

Bingo.

Prasanna Malaiyandi:

See, I didn't forget everything.

W. Curtis Preston:

Yeah, no good job that wasn't a test by the way.

W. Curtis Preston:

Uh, but the, yeah, I I'd say mistake number one, that, that a

W. Curtis Preston:

lot of people make is that they think it means to restore time.

W. Curtis Preston:

It doesn't, it means from the moment the outage happened to the

W. Curtis Preston:

moment the application and any related applications are back up and

W. Curtis Preston:

running in fully functional state.

W. Curtis Preston:

Right.

W. Curtis Preston:

That, that is, that is the objective.

W. Curtis Preston:

We're gonna, we're gonna talk about the reality in a minute, but

W. Curtis Preston:

that, but that is the objective.

W. Curtis Preston:

That's what you've agreed.

W. Curtis Preston:

You say, listen, another way to call this is, is an SLA, right?

W. Curtis Preston:

A service level agreement.

W. Curtis Preston:

You have an SLA with your stakeholders.

W. Curtis Preston:

That, what, what is that?

W. Curtis Preston:

What was that?

Prasanna Malaiyandi:

that, that was me being like, I

Prasanna Malaiyandi:

don't like calling them SLAs.

Prasanna Malaiyandi:

I like calling them SLOs.

W. Curtis Preston:

Okay.

W. Curtis Preston:

All right.

W. Curtis Preston:

All right.

W. Curtis Preston:

I'll I'll I'll let that slide . So what would make it an SLA to

W. Curtis Preston:

you if you just agreed to that?

W. Curtis Preston:

Archie R RPO.

Prasanna Malaiyandi:

Or the RTO?

Prasanna Malaiyandi:

Well, I think it's, you need to be able to provably show that you are

Prasanna Malaiyandi:

hitting that every single time, right.

Prasanna Malaiyandi:

Rather than here's objective, right.

Prasanna Malaiyandi:

Because these are.

W. Curtis Preston:

But the agreement.

W. Curtis Preston:

I mean, yeah.

W. Curtis Preston:

I mean, we may, we may be, um, I don't know if we're, I mean,

W. Curtis Preston:

what an SLA is, it's an agreement between two groups of people, maybe

W. Curtis Preston:

more than two groups of people.

W. Curtis Preston:

This is the agreement that we have made.

W. Curtis Preston:

This is what, this is the objective we're gonna meet.

W. Curtis Preston:

So maybe the RPO and RTO is the, the metric an objective

W. Curtis Preston:

upon which you create an SLA.

W. Curtis Preston:

All right.

W. Curtis Preston:

I, I, I won't sure.

W. Curtis Preston:

Okay.

W. Curtis Preston:

I'll, I'll be fine with that.

W. Curtis Preston:

So, uh, so an RTO is essentially how long it takes to bring, you know,

W. Curtis Preston:

how long it should take to bring the application back up online.

W. Curtis Preston:

Right?

W. Curtis Preston:

So RPO recovery point objective is basically how much data you have

W. Curtis Preston:

agreed you're allowed to lose it.

Prasanna Malaiyandi:

data.

Prasanna Malaiyandi:

How you're.

W. Curtis Preston:

So a as expressed by a matter of time, meaning you

W. Curtis Preston:

agree that you will allow you, you will allow a loss of one hour's worth

W. Curtis Preston:

of data or 24 hours worth of data.

W. Curtis Preston:

And, and I would say at least going back in, back in back my day, um, RPO was the

W. Curtis Preston:

one we talked about the least, at least it's the one we talked about the least.

W. Curtis Preston:

Frankly, did that come out in English?

W. Curtis Preston:

the less, it was less frankly discussed because we all knew that we only

W. Curtis Preston:

backed up once a day and that all the backups didn't work every day.

W. Curtis Preston:

And so we knew that the best we could do was a 24 hour RPO and that maybe it might

W. Curtis Preston:

be 48 or 72, depending on what day of the week it was and all that kind of stuff.

W. Curtis Preston:

But nobody wanted to talk about.

Prasanna Malaiyandi:

That's interesting being on the, from the vendor side, I

Prasanna Malaiyandi:

remember focusing so much on the RPO side of things and less on the RTO side.

W. Curtis Preston:

Interesting.

W. Curtis Preston:

Interesting.

W. Curtis Preston:

Now, is that because.

Prasanna Malaiyandi:

a lot of it was really around replication, right.

Prasanna Malaiyandi:

Where people do care more about the RPOs and the fact that, because a lot of it

Prasanna Malaiyandi:

was disk based systems, storage appliances replicating from one to another, right.

Prasanna Malaiyandi:

Typically your RTO was minutes or

W. Curtis Preston:

Yeah, the RTO was minutes.

W. Curtis Preston:

So that was easy peasy.

W. Curtis Preston:

That was way better than anything it had before.

W. Curtis Preston:

So then they're like, okay, now let's talk about the amount of data we're gonna lose.

W. Curtis Preston:

Right.

W. Curtis Preston:

Ex exactly.

W. Curtis Preston:

So the,

Prasanna Malaiyandi:

But,

W. Curtis Preston:

thing is that sure.

Prasanna Malaiyandi:

one last thing in terms of the data you lose, right?

Prasanna Malaiyandi:

That's from the time a disaster strikes to going backwards in time.

Prasanna Malaiyandi:

Right.

W. Curtis Preston:

Correct.

W. Curtis Preston:

It is the disaster happened.

W. Curtis Preston:

And then, you know, the, the amount of data that we, that we transactions,

W. Curtis Preston:

whatever it is that we put into the system that we're agreeing we can lose

W. Curtis Preston:

because we had to restore from a backup that is 10 hours old or whatever it is.

W. Curtis Preston:

Right.

W. Curtis Preston:

We're agree.

W. Curtis Preston:

We're agreeing in advance that, you know, we need to.

W. Curtis Preston:

We need to lose less than four hours worth of data.

W. Curtis Preston:

Um, and then, you know, as we talked before, we have an SLA based around that.

W. Curtis Preston:

So what I, what I having spent so much time in the backup side, the,

W. Curtis Preston:

there was R there was RTO and RPO, but then there was some people call RPA.

W. Curtis Preston:

RTA others call RPR and RTR so that's recovery point actual or

W. Curtis Preston:

recovery point reality, right?

W. Curtis Preston:

One is one is recovery point objective.

W. Curtis Preston:

The other is reality, right?

W. Curtis Preston:

So that's, this is something that you, as a backup person, and again, I

W. Curtis Preston:

use the term backup to be, to include any kind of recovery mechanism.

W. Curtis Preston:

This is something that you, as a backup person should know.

W. Curtis Preston:

You, you should be aware for every type of system that you have.

W. Curtis Preston:

You should be aware of what the actual recovery time, at least your portion

W. Curtis Preston:

of the recovery time, you should know that the recovery time actual and

W. Curtis Preston:

the recovery point actual is X number of hours, and you should be able to

W. Curtis Preston:

communicate that effectively by say.

W. Curtis Preston:

And, and how would you know that.

Prasanna Malaiyandi:

Actually doing it and trying it out,

W. Curtis Preston:

Yes.

W. Curtis Preston:

There's a word for that.

Prasanna Malaiyandi:

restore validation.

W. Curtis Preston:

Another word

Prasanna Malaiyandi:

Uh, Verify

W. Curtis Preston:

starts with a T

Prasanna Malaiyandi:

test.

Prasanna Malaiyandi:

There we go.

Prasanna Malaiyandi:

Test your backups.

W. Curtis Preston:

yeah, it's a bit.

W. Curtis Preston:

Yeah, but all those words you said were all valid.

W. Curtis Preston:

Those were all very valid.

W. Curtis Preston:

It's just the only way you're gonna do this is to test it.

W. Curtis Preston:

You've got to, you've got to test your restores in order to

W. Curtis Preston:

know what your RTA and RPA are

Prasanna Malaiyandi:

Uh, one thing,

W. Curtis Preston:

go ahead.

Prasanna Malaiyandi:

no finish.

W. Curtis Preston:

Well, well be because the next phase we're, we're

W. Curtis Preston:

gonna talk about is how to come to some sort of agreement, right?

Prasanna Malaiyandi:

So

W. Curtis Preston:

like you're in a meeting and they're like, I want,

W. Curtis Preston:

I want an RTO and an RPO of zero.

W. Curtis Preston:

and then, then you should be able to say immediately, well,

W. Curtis Preston:

currently we could do 24 hours and 16 hours, whatever the number is.

W. Curtis Preston:

And then, and then you have a discussion right.

W. Curtis Preston:

But if you don't know that you, you know, your Sol.

Prasanna Malaiyandi:

I think the one thing to also consider is like we had

Prasanna Malaiyandi:

talked about recovery versus re recovery of an application versus restoring data.

Prasanna Malaiyandi:

You may not be responsible for the end to end recovery of that application.

Prasanna Malaiyandi:

You might only be responsible for a part.

Prasanna Malaiyandi:

So just because that application team says, oh, I have four hours

Prasanna Malaiyandi:

to recover my applications, don't think that you have all four

Prasanna Malaiyandi:

hours to get the data back, right?

Prasanna Malaiyandi:

Because you might only be a small percentage of bringing

Prasanna Malaiyandi:

up the entire application.

W. Curtis Preston:

Yeah.

W. Curtis Preston:

And part of that four hours may be equipment PR procurement.

W. Curtis Preston:

I don't know why that was so hard for me to get out equipment procurement, right.

W. Curtis Preston:

It may be, there may be repair.

W. Curtis Preston:

There may be bringing in a vendor and, um, you know, all of

W. Curtis Preston:

that has to be figured into it.

W. Curtis Preston:

I I'll, I'll say this when you have a major outage, unless

W. Curtis Preston:

you've planned really well for it.

W. Curtis Preston:

Like, like you, you have to be able to plan, like you have to

W. Curtis Preston:

have spare equipment available.

W. Curtis Preston:

You have to have spare storage capacity.

W. Curtis Preston:

You have to have spare computing capacity and you have to have

W. Curtis Preston:

a, some sort of recovery system rocking and rolling and ready to go.

W. Curtis Preston:

That's the only way you're gonna meet most modern RTOs and RPOs.

W. Curtis Preston:

If you're gonna wait to call a vendor to come replace a disc drive or a server,

W. Curtis Preston:

before you start your restore, you're never gonna meet your RTO and RPO.

Prasanna Malaiyandi:

Well, and especially right now, because it's still the,

Prasanna Malaiyandi:

I guess technically the pandemic's over we're now in an endemic stage,

Prasanna Malaiyandi:

but during the COVID pandemic, right.

Prasanna Malaiyandi:

It was hard to get equipment, right.

Prasanna Malaiyandi:

Supply chains, people showing up in offices.

Prasanna Malaiyandi:

Right.

Prasanna Malaiyandi:

So if you needed to add a server in order to be able to do the restores

Prasanna Malaiyandi:

good luck trying to hit your normal RTOs

W. Curtis Preston:

Yeah, we, we may be.

W. Curtis Preston:

We may be in the endemic phase, but trust me, the supply chain problem is not over.

W. Curtis Preston:

Right.

W. Curtis Preston:

Um, the, I am aware of competitors of Druva's that have several month lead

W. Curtis Preston:

times on their, on their new systems.

W. Curtis Preston:

So it, it, it, you know, it, it, the problem isn't over.

W. Curtis Preston:

So the, you know, and of course we think of that as a competitive

W. Curtis Preston:

differentiator, of course.

W. Curtis Preston:

Right.

W. Curtis Preston:

Cuz we don't, we don't have that issue cuz we're, we're a service.

Prasanna Malaiyandi:

One other thing I wanted to add about

Prasanna Malaiyandi:

equipment is in the, this is where you can go to the extreme, right?

Prasanna Malaiyandi:

You could say, okay, for every single system I have, I'm

Prasanna Malaiyandi:

gonna double the capacity.

Prasanna Malaiyandi:

Right.

Prasanna Malaiyandi:

That way I never have to worry about bringing in

Prasanna Malaiyandi:

equipment in case a site fails.

Prasanna Malaiyandi:

Right.

Prasanna Malaiyandi:

The thing though, you have to worry about is backup.

Prasanna Malaiyandi:

People aren't spending a whole lot of their budget on making sure there's

Prasanna Malaiyandi:

infrastructure ready for backup.

Prasanna Malaiyandi:

So as someone working in backup, you need to make sure you figure out what are those

Prasanna Malaiyandi:

mission critical applications that need to be immediately up and running, right?

Prasanna Malaiyandi:

Where maybe I need to keep some percentage extra capacity in order to support that.

Prasanna Malaiyandi:

Right.

Prasanna Malaiyandi:

What are sort of the things that, eh, if something happens, it might.

Prasanna Malaiyandi:

Say a week to bring these back up.

Prasanna Malaiyandi:

Maybe I don't actually have equipment for that, for those things.

Prasanna Malaiyandi:

Right.

Prasanna Malaiyandi:

And that's okay.

Prasanna Malaiyandi:

But I think going and telling someone, oh yeah, your production budget.

Prasanna Malaiyandi:

I need the exact same amount for backup.

Prasanna Malaiyandi:

Right?

Prasanna Malaiyandi:

Usually doesn't fly in a lot of corporations.

W. Curtis Preston:

Yeah.

W. Curtis Preston:

I mean, you know, and I'm talking about additional, like, I think this

W. Curtis Preston:

is about virtualization and cloud.

W. Curtis Preston:

The more virtuals I, the more virtualized you are, the more cloud

W. Curtis Preston:

focused you are, the easier this particular issue becomes, right?

W. Curtis Preston:

You just need one or two extra servers ready to go.

W. Curtis Preston:

Um, not an entire, I'm just saying, I'm just saying you can,

W. Curtis Preston:

you can deal with a lot more

Prasanna Malaiyandi:

That's true.

Prasanna Malaiyandi:

Then

W. Curtis Preston:

by, you know, how much you need is, is, is.

W. Curtis Preston:

Gonna be up to you, but I'm just saying, if you, I'm just saying it's easier if

W. Curtis Preston:

you're virtualized, because if you're virtualized and you have an application

W. Curtis Preston:

goes down because of some sort of data issue, you can easily restore that VM in

W. Curtis Preston:

another server without acquiring anything.

W. Curtis Preston:

I guess that's, that's sort of what it, where I was going

Prasanna Malaiyandi:

which works.

Prasanna Malaiyandi:

But the one thing I would caution is cloud is great.

Prasanna Malaiyandi:

But if you have an entire disaster that strikes a geographic region, right.

Prasanna Malaiyandi:

And everyone is trying to spin up in the cloud at the same

Prasanna Malaiyandi:

time, cloud is still servers.

Prasanna Malaiyandi:

So don't think it's something magical, right.

W. Curtis Preston:

No, it's not magical, but you could, you could

W. Curtis Preston:

prepare for a multi, you could prepare for a different region cloud

Prasanna Malaiyandi:

Yep.

W. Curtis Preston:

recovery,

Prasanna Malaiyandi:

Exactly.

Prasanna Malaiyandi:

Just make sure you look at your options,

W. Curtis Preston:

yeah.

W. Curtis Preston:

Yeah.

W. Curtis Preston:

Um, why, why we just gotta have a big butt man.

Prasanna Malaiyandi:

but I, I just, but I I'm just saying, I just

Prasanna Malaiyandi:

wanna make sure people don't think the cloud is something magical.

Prasanna Malaiyandi:

Yes.

W. Curtis Preston:

cloud is not magic.

Prasanna Malaiyandi:

There are lots of

W. Curtis Preston:

It is not magical.

Prasanna Malaiyandi:

Yeah, there are lots of great benefits to it,

Prasanna Malaiyandi:

but you just need to make sure you understand the limitations as well.

W. Curtis Preston:

So, so the title RTO and RPO are what drives backup design.

W. Curtis Preston:

So RTO drives the power, the, the speed of the system, because the, the beefier,

W. Curtis Preston:

the system, the, the quicker it's able to restore, um, you know, the, the, the, the.

W. Curtis Preston:

Easier.

W. Curtis Preston:

You're gonna be able to meet a tighter recovery time objective, right?

W. Curtis Preston:

The RPO is what's going to drive your backup frequency.

W. Curtis Preston:

If you have a one hour RPO and you're backing up once a day, you are in trouble.

W. Curtis Preston:

Right.

W. Curtis Preston:

Um, so that, that's what I meant B because all your backup decisions or

W. Curtis Preston:

your backup design decisions should be based on how they affect RTO and RPO.

Prasanna Malaiyandi:

Yep.

W. Curtis Preston:

And if, if you're not, if you're not doing that, then

W. Curtis Preston:

you are doing yourself a disservice.

Prasanna Malaiyandi:

And it's also not to say that you will never be able to

Prasanna Malaiyandi:

meet a one hour RPO with a backup system.

Prasanna Malaiyandi:

Right.

Prasanna Malaiyandi:

Once again, it also has to take into account not only the speed, but

Prasanna Malaiyandi:

also the amount of data you have.

Prasanna Malaiyandi:

Right.

Prasanna Malaiyandi:

So take that into consideration as you're looking at it, because maybe you have a

Prasanna Malaiyandi:

database with not a lot of change rate.

Prasanna Malaiyandi:

That's fairly small that yeah.

Prasanna Malaiyandi:

One hour RPO.

Prasanna Malaiyandi:

You can hit and probably like a 15 minute RTO, right.

Prasanna Malaiyandi:

Perfectly well suited for that.

Prasanna Malaiyandi:

But if it say grows from a small database to say 20 terabyte database, yeah.

Prasanna Malaiyandi:

Maybe you're not able to hit those same RPOs and RTOs.

Prasanna Malaiyandi:

Right.

Prasanna Malaiyandi:

I think that was one of the points you wanna make earlier, Curtis.

Prasanna Malaiyandi:

Right?

Prasanna Malaiyandi:

Is it depends on not only the size, but I think you also wanna talk

Prasanna Malaiyandi:

about the types of failures too.

W. Curtis Preston:

So I, I don't think there's any RTO or RPO you can't meet.

W. Curtis Preston:

And I'm not.

W. Curtis Preston:

So I'm not sure I agree with what you said just a few minutes ago.

W. Curtis Preston:

I, I think I understand what you were trying to say, but it, it

W. Curtis Preston:

it's, well, first off there is no RTO on RPO you can't meet.

W. Curtis Preston:

With money, right.

W. Curtis Preston:

Regardless of the size of the database,

Prasanna Malaiyandi:

I should say picking a certain technology to use.

W. Curtis Preston:

Okay.

W. Curtis Preston:

Yeah.

W. Curtis Preston:

Yeah.

W. Curtis Preston:

Different technologies enable different RTOs and RPOs replication, and, you know,

W. Curtis Preston:

uh, CDP, continuous data protection.

W. Curtis Preston:

These are technologies that can, that can beat both a zero, uh, RTO and a zero RPO.

W. Curtis Preston:

They are expensive, you know, they are expensive.

W. Curtis Preston:

Uh, but the question is how much money are you losing when you're down?

Prasanna Malaiyandi:

Yep.

W. Curtis Preston:

There's that.

W. Curtis Preston:

So, okay.

W. Curtis Preston:

So if that's how we decide on backup design and, and RTO and RPO are really

W. Curtis Preston:

important and I've never done that.

W. Curtis Preston:

How do I do that?

Prasanna Malaiyandi:

How

W. Curtis Preston:

you know, so yeah,

Prasanna Malaiyandi:

well, I think the starting point is go talk

Prasanna Malaiyandi:

to your business stakeholders.

Prasanna Malaiyandi:

right.

Prasanna Malaiyandi:

I think, understand what they need, what are their requirements?

Prasanna Malaiyandi:

And not just, oh, what do you want from RTO and RPO, but ask them the

Prasanna Malaiyandi:

questions of what would the impact be if this application was down

Prasanna Malaiyandi:

for a day, because that'll change the answer they give you back.

W. Curtis Preston:

yeah, what is the financial impact?

W. Curtis Preston:

Do this app being down for a day or an hour, et cetera.

W. Curtis Preston:

And if they don't have that data, then honestly they don't deserve to be in their

Prasanna Malaiyandi:

Speaker:

Tier three, tier three.

W. Curtis Preston:

Yeah.

W. Curtis Preston:

Tier three.

W. Curtis Preston:

Yeah.

W. Curtis Preston:

So you get an RTO on an RPO of a week.

W. Curtis Preston:

If they don't have that data, then I don't know what to say.

W. Curtis Preston:

Right.

W. Curtis Preston:

Um, but if they have that data and they, and they know that it's a

W. Curtis Preston:

million dollars an hour, well, that helps you go and justify the amount

W. Curtis Preston:

of money that you need to spend.

W. Curtis Preston:

So you ask for that RTO and RPO, and then you say, well, our current system.

W. Curtis Preston:

As designed and as budgeted has an RTO, an RTA, an RPA, uh, you know, you could

W. Curtis Preston:

say that like in plain English, you could say it has the ability to meet an RTO of

W. Curtis Preston:

an hour, has an ability to meet an RPO of 12 hours, whatever the number is for you.

W. Curtis Preston:

And, um, so.

W. Curtis Preston:

The, and then they were like, what, you know, and that's

W. Curtis Preston:

when the conversation begins

Prasanna Malaiyandi:

then it's a negotiation like at a car dealership.

W. Curtis Preston:

it's absolute why'd you have to bring up car dealership.

Prasanna Malaiyandi:

Sorry.

Prasanna Malaiyandi:

It's not, maybe it's not as painful as a car dealership.

W. Curtis Preston:

I had such a non-fun experience getting my

W. Curtis Preston:

wife, her, her new car, and it was.

W. Curtis Preston:

Why you gotta bring that up.

W. Curtis Preston:

Um, but yeah, it's a, it's a conversa, it's a business

W. Curtis Preston:

discussion back and forth, right?

W. Curtis Preston:

They could, they say, well, we want, we want, you know, an RTO of, of one hour.

W. Curtis Preston:

And you're like, well, that's, I'm sorry.

W. Curtis Preston:

That's not possible.

W. Curtis Preston:

It's totally possible.

W. Curtis Preston:

It's just that it costs a, you know, a ton of money.

W. Curtis Preston:

So you, you have to, you have to come to a point where.

W. Curtis Preston:

You're like I could.

W. Curtis Preston:

And, and then, and you know, it, it's almost always starts like this.

W. Curtis Preston:

They want an RTO of this, and you're able to do an RTO of that and you

W. Curtis Preston:

need to go, you need to, right.

W. Curtis Preston:

You need to meet in the middle.

W. Curtis Preston:

Almost always.

W. Curtis Preston:

You're going to need to make some technological changes

W. Curtis Preston:

in order to, to do that.

W. Curtis Preston:

Those technological changes have costs.

W. Curtis Preston:

You can go back to that business shooting and say, we can get to here.

W. Curtis Preston:

It's going to.

W. Curtis Preston:

1 million.

W. Curtis Preston:

Right?

W. Curtis Preston:

And then they go, what?

W. Curtis Preston:

And then you go, well, we can get to, you know, we can get to here for a million.

W. Curtis Preston:

We can get to here for 25,000, right.

W. Curtis Preston:

Somewhere in there.

W. Curtis Preston:

There's a, there's a point of decreasing marginal returns.

W. Curtis Preston:

Right.

W. Curtis Preston:

Um, and you need to find that spot and get them to agree to that spot.

Prasanna Malaiyandi:

the one question you brought up earlier, Curtis, which

Prasanna Malaiyandi:

might be worthwhile thinking about is, or discussing is you mentioned that it

Prasanna Malaiyandi:

depends on the type of failure, right?

Prasanna Malaiyandi:

When you're talking about RTO and RPO, right.

Prasanna Malaiyandi:

Does that come into this discussion as you're talking to the

W. Curtis Preston:

I think it should.

W. Curtis Preston:

I think it should.

W. Curtis Preston:

This, this is a, this is a, um, a philosophical discussion.

W. Curtis Preston:

There are those who feel that all RTA, all RTOs and RPOs should be the same.

W. Curtis Preston:

Whether you deleted a file.

W. Curtis Preston:

Or, you know, you had a natural disaster take out your entire state.

W. Curtis Preston:

Um, I don't personally feel that way.

W. Curtis Preston:

I, I, I feel that for, for the most common type of, of things that happen,

W. Curtis Preston:

you should be able to, to have a pretty short RTO and RPO, right.

W. Curtis Preston:

You, you should be able, you know, I lost a file.

W. Curtis Preston:

Boom, boom.

W. Curtis Preston:

And that should be like a minute.

W. Curtis Preston:

Right.

W. Curtis Preston:

You know, it should not take a long.

W. Curtis Preston:

And I think that if there's a major disaster, I think

W. Curtis Preston:

you will get some, some, um,

Prasanna Malaiyandi:

Leeway.

W. Curtis Preston:

understanding.

W. Curtis Preston:

Yeah.

W. Curtis Preston:

Some leeway.

W. Curtis Preston:

Thank you.

W. Curtis Preston:

That's a perfect word.

W. Curtis Preston:

But, but again, all that really matters is that this is just my opinion.

W. Curtis Preston:

It's what, it's what your company will, you know, pay for.

W. Curtis Preston:

If you're going to have, if you're, if you're gonna have a,

W. Curtis Preston:

you know, the best RTO and RPO.

W. Curtis Preston:

For every kind of outage, then it's just gonna cost you a whole lot of money.

Prasanna Malaiyandi:

yeah.

W. Curtis Preston:

As long as they're willing to pay that money,

W. Curtis Preston:

then, you know, we're all happy.

Prasanna Malaiyandi:

And also on the flip side, right?

Prasanna Malaiyandi:

If your RTO and RPOs are short for the most common ones, and they're

Prasanna Malaiyandi:

long for these critical or for these unexpected outages set that expectation.

Prasanna Malaiyandi:

So people aren't yelling at you later, right.

Prasanna Malaiyandi:

Set the expectation with the business and say, look, I will save you money.

Prasanna Malaiyandi:

Right.

Prasanna Malaiyandi:

And here's what it is in the most general cases.

Prasanna Malaiyandi:

And yes, something catastrophic happens, then yes, here is now my new RPO or

Prasanna Malaiyandi:

my RTO is gonna be, say three days.

Prasanna Malaiyandi:

And as long as everyone's okay with that, and it's understood and documented, right?

Prasanna Malaiyandi:

It's something you can go forward with, cuz you're saving a bunch of

Prasanna Malaiyandi:

money because it's all about risk.

Prasanna Malaiyandi:

Right?

Prasanna Malaiyandi:

How often is that catastrophic event going to happen?

Prasanna Malaiyandi:

Right.

Prasanna Malaiyandi:

And is say three to five days to recover.

Prasanna Malaiyandi:

Is that.

W. Curtis Preston:

Yeah, that's it.

W. Curtis Preston:

That's all we're saying right.

W. Curtis Preston:

Is get the RTO and RPO decided upon and agreed upon beforehand

W. Curtis Preston:

and get the RTA and RPA.

W. Curtis Preston:

Hopefully the two should match, get them to match, but if they don't match, by

W. Curtis Preston:

the way, that's another scenario is we all know we should have a better RTO

W. Curtis Preston:

and RPO, but this is what our budget currently will allow due to market

W. Curtis Preston:

conditions, the condition of the company, whatever, as long as we all know that

W. Curtis Preston:

now, so that when something bad happens and then you go to do this large

W. Curtis Preston:

restore and it takes a really long time.

W. Curtis Preston:

The, you know, they will, they will know that that's the case.

W. Curtis Preston:

Right.

Prasanna Malaiyandi:

don't wanna be left holding the bag

W. Curtis Preston:

You do not want to be, you do not want to be the one

W. Curtis Preston:

blamed for the long restore or the restore that lost an acceptable or an

W. Curtis Preston:

unacceptable amount of, uh, of data.

Prasanna Malaiyandi:

Yep.

W. Curtis Preston:

All right.

W. Curtis Preston:

Well, um,

Prasanna Malaiyandi:

You think we talked about,

W. Curtis Preston:

that time.

Prasanna Malaiyandi:

I, I gotta try to mix it up every once in a while, you

W. Curtis Preston:

You had, you had to, you had to argue with me, man.

W. Curtis Preston:

So you hurt, you hurt my feelings.

W. Curtis Preston:

all right.

W. Curtis Preston:

Well, well, it was good.

W. Curtis Preston:

Good stuff.

W. Curtis Preston:

Um, I hope you guys, um, hope you folks out there.

W. Curtis Preston:

Learned a thing or two, and maybe, maybe you didn't agree.

W. Curtis Preston:

You know what?

W. Curtis Preston:

Come on, come on the podcast.

W. Curtis Preston:

Uh, we don't even agree with each other sometimes.

W. Curtis Preston:

So, you know, we'd be, we'd be happy to have you on give us a comment.

W. Curtis Preston:

And apparently if you make more than it has to be at least nine comments more

W. Curtis Preston:

than we have today on the apple podcast.

W. Curtis Preston:

Apparently I have to grow a beard nine or more.

Prasanna Malaiyandi:

Yep.

W. Curtis Preston:

Apparently I have to grow a beard for Christmas.

W. Curtis Preston:

Um, That'll be, that'll be something.

W. Curtis Preston:

All right.

W. Curtis Preston:

Well, and remember, of course, to subscribe so that you can restore it all.