If you liked last week’s episode where we talked about this “so let’s talk about ransomware” series on reddit, you’ll love this week. We have the author, Snorkel42, to talk about the origins behind the security cadence series, and why he decided to finally write some on ransomware. (He explains that everything he talks about his ransomware, but he admits he’s been “Mr. Myagi’ing” it for a while.). This guy knows his stuff, and this is the second time he has been on the podcast. He’s knowledgeable and entertaining. One of those rare combinations. This is a great episode you will not want to miss.
Here are the three posts:
https://www.reddit.com/r/sysadmin/comments/tdvbp4/security_cadence_okay_fine_lets_talk_ransomware/
https://www.reddit.com/r/SecurityCadence/comments/tedapy/security_cadence_ransomware_part_2_actions_on/
https://www.reddit.com/r/SecurityCadence/comments/tfm927/security_cadence_ransomware_part_3_the_worst_case/
Transcript
[00:00:00] W. Curtis Preston: I want to say thank you again for coming on the podcast and talking to these really important things,
[00:00:06] Snorkel42: No, it was a pleasure to be here. I really, I have to say when I got, when I got your message, like, wait, is that Mr. Backup? That is Mr. Backup! I was really excited.
[00:00:35] W. Curtis Preston: Hi, and welcome to Backup Central’s Restore it All podcast. I’m your host, W. Curtis Preston, AKA Mr. Backup.
I have with me, my ghee deployment consultant, Prasanna Malaiyandi.
[00:00:47] Prasanna Malaiyandi: uh, Curtis, how’s your ghee going?
[00:00:51] W. Curtis Preston: You know, you may recall it a month or so ago. I got ghee for the first time. And, you know, for those that don’t know what ghee is, it’s clarified butter specifically. It’s apparently an Indian thing, right? we did learn that ghee is from Sanskrit. That means sprinkled, which is interesting. But the, um, and the thing about it is that it can sit on the counter.
Um, like it’s shelf so it the counter and you made a comment by the way. I know this because I’m literally editing this episode right now. And you made a comment that, you know, it can sit there probably for a couple of months or until you run out. So I’ll just say this, the, uh, the jar, I thought that I was going to be the only weirdo using the ghee. Yeah, I am not the weirdo using ghee. In fact, if anything, the rest of the house is used the ghee much more than I have. And that jar that we bought is close to gone. So
[00:01:52] Prasanna Malaiyandi: it’s convenient. It’s super easy. You just keep it out. You warm it up a little. It becomes really liquidy. Right? You put it on warm bread. You toast that. super good. You can also put it in rice. Warm rice. It’s really good.
[00:02:05] W. Curtis Preston: Yeah, well, I mean, it’s like it’s butter, right? you know, all the places you can put butter, you can put ghee. Um, it’s just, it, it, it is interesting for those that have never had it. It has a slightly different flavor than butter, but it, you know, so there is a, there is a, I didn’t know what my mouth was going, what was going to happen when I put it in there,
[00:02:28] Prasanna Malaiyandi: That’s funny because I never think about that. Like to me, like butter and ghee it’s like, ah, yeah.
[00:02:33] W. Curtis Preston: Yeah. The first time I had, I had ghee, I remember going, huh, this tastes different, you know? Um, but what it was, but it was, but it was yummy, you know, so we continue to, but yeah, I think that jar is almost gone. So we’re going to have to, we’re going to have to find the ghee at Costco, which is
[00:02:54] Prasanna Malaiyandi: They actually, I think they sell like one pound. Yeah, one pound jars or something.
[00:02:59] W. Curtis Preston: Well we have a, a back by popular demand guest here. Uh, he was on the podcast before and. Is the author of the security cadence series on Reddit, been in IT for about 25 years and in InfoSec about 20 years. And he is quite the celebrity over there on Reddit because I, you know, his posts have been incredibly popular with, uh, uh, he’s got a, uh, a, uh, what a, what do they call it over there? The karma of 35,000, which, you know, if you don’t know anything about Reddit, that’s a, BFD, I’ll just say that right now. And we had him previously on the podcast so if you haven’t heard that podcast, you should totally listen to that.
Welcome back to the podcast. snorkel42.
[00:03:50] Snorkel42: Good to be back. And I tell you, so last time I learned what karma meant on reddit. This time I learned what ghee is so good for me.
[00:04:00] W. Curtis Preston: I forgot to put our disclaimer in here. Prasanna works for zoom. I work for Druva. This is not a podcast of either company and the opinions that you hear are ours. And please rate this podcast and ratethispodcast.com/restore
or if you’re listening to, you know, to us on the apple podcast, just scroll down to the bottom and click five stars. Five stars is five stars. Yeah. Not
[00:04:23] Prasanna Malaiyandi: Curtis loves a comment
[00:04:24] W. Curtis Preston: about just how much you enjoyed this podcast. And, uh, also, you know, if you want to talk, we want to listen. We want to hear your, your backup woes your security woes.
I don’t know, whatever, just your woes. We’ll be here. We’ll be a friendly ear. And, um, reach out to me. It wcurtispreston@gmail, or @wcpreston on Twitter and we’ll get you on here. So, um,
I want to hear about this thing. You mentioned about security cadence. What, what started, because that was, that was the, um, that was the, what, what do you call that? Uh, the thing before the thing, the, the precursor, the preamble, the, in the title, um, uh, to the post that we saw that.
So where did that term come from?
[00:05:13] Snorkel42: So the term came from a previous employer. I worked at where I was a network engineer, um, and it was a large company that did not have an InfoSec presence. There was no InfoSec team. It was just kind of considered, Hey, all engineers are responsible for security. Um, and you know, we would, we’d have our occasional shots off the bow in terms of security, you know, problems or issues.
You know, I, I hesitate to say breaches, but, you know, incidents and when they would occur and we would all pile into a conference room and we would talk about what happened and what we should have done to prevent it. And it would always come down to, well, if we just bought six figure dollar product X, this would not have occurred.
Meanwhile, in the data center, there were piles of six-figure products that were completely ignored because those products never run themselves. And if you’re complaining of not having resources to run those products, what makes you think you’re going to have resources to buy and run new products? So I finally got fed up one day and just challenged the team to, you know what?
We’ve got plenty of tooling. That’s not the issue. We just don’t have the oomph. We don’t have the motivation to actually use it. So why don’t we set ourselves a goal for six months, we’re going to implement a security change every week. It could be a big change, could be a really minor change, but there’s going to be something.
Move the ball forward. And also during that six months, we are going to just put the kibosh on talking to any vendors. So no products are allowed, no solutions are allowed that we don’t already own. Um, and you know, it took some, took some talking and wrangling people, but eventually folks fell in line and we sat down.
We’d wrote out a list of, you know, here’s some things just off the top of our head that we know we could implement with, you know, very little roadblocks, no impact. And we just started and we called it our security cadence. Once a week, we have a security cadence of releasing a security update and it, you know, it ended around six months in our CIO came to us and said, listen to what you guys are doing is fantastic, but please don’t restrict yourself to only free solutions.
Like we have money. If there’s stuff you need to keep this ball moving, please ask. Um, so, you know, it kind of turned the whole thing on its head of, you know, securing the company. But also we were no longer begging for resources. The, you know, the executive leadership was asking us, was begging us to start spending resources because they saw what we were doing and saw the value.
[00:07:28] Prasanna Malaiyandi: So, could you talk a little bit, I’m sure at the very beginning as you’re starting this right. With any new process or new, any new endeavor, it’s a little difficult, right. Sort of getting into what does it mean? And trying to figure things out. So what were some of the challenges you guys went through and how did you address.
[00:07:45] Snorkel42: Yeah. So I would say the biggest challenge with any sort of security changes, especially in a large company is just the unknown of what will this break. Um, because quite often, especially in those early days, what you’re changing are out of the box configurations. So there’s this kind of mentality of, well, it’s probably an out of the box configuration for some reason, or, you know, we don’t know what legacy, I mean, this company that I was working at at the time was started in the 1930s.
Now we don’t know what kind of legacy applications are relying on this technology. Um, so I would say the first, the biggest thing was to just start easy, take the really easy ones to get as much buy-in as you can, you know, sit down to the engineers. Can anyone think of anything that would break by doing this?
And you will get some feedback. Yeah. But who cares? It’s not going to fix anything. Okay. That’s fine. Let’s just do it anyways. Um, and then start, you know, slowly ramping it up and taking little bite-size chunks. And if you look at the security CA cadence, Reddit posts, that’s exactly how I’ve been approaching them.
You know, I’ve started off with just really easy things to do and things I would not expect, um, to break many enterprises, you know, I tried to make it very clear in those posts of, you know, everyone’s environment is different and be careful, but you know, I’ve called out certain items of this. Isn’t going to break anything, just do it, you know, please just, just do it.
[00:09:08] W. Curtis Preston: I do remember in your, the trio of posts that you did that were around ransomware, you, you had a, there was a phrase that came up a lot. It’s escaping me at the moment, but it was like, turn this on and then customize as necessary. Right. That, that, that, you know, that you can’t, that no one solution does, uh, you, you can make a general rule for example, and then you’re going to find somebody that needs, that thing turned on the thing you just turned off, you’re going to find somebody that needs, that turned on and then you can turn it on for them.
Right. Um, and, and that’s okay.
[00:09:46] Snorkel42: Yeah. So one of the catchphrases are one of my guiding lights in InfoSec is to never let perfect get in the way of being good.
I call it out a lot. And part of the reason why I lean on it so heavily is it’s often a. Uh, voice of dissension that you get from folks when you’re trying to talk them into things like, oh, well that won’t solve this one edge case, so let’s not do it at all. Um, and you know, when it comes to security, security, it’s all about layers and it’s all about catching the attacker and yeah, there, this may not solve all of your problems, but it might be the alert that gets generated that tells you that they’re there. Um, you know, and so it is definitely a strong, um, demand I make a people of, you know, if you can only do this for one system, great, it’s better than none. Um, and yeah, so there there’s something to be said about going slow and implementing slowly, but there’s also something that I said about implementing broad and then backing off where you need to.
[00:10:44] W. Curtis Preston: Hmm. Yeah. Yeah, exactly. I would say that. When you, when you try the latter, when it, when I was thinking about your, your initial, this, the six months program that you had, the farther you got into that six months, and the more complicated things that you were doing that were potentially riskier, if you will, to the environment that you could potentially impact someone’s ability to do their job, the more you’re going to need support from above.
Right? Like, I, I, I told, you know, I told them to do this. We’re sorry that it broke, you know, we’ll
[00:11:21] Prasanna Malaiyandi: Yeah.
[00:11:22] W. Curtis Preston: we turned it off for now. We didn’t realize that by pushing this one button was going to make everyone in the company not be able to log in ever. Uh, we’ve turned it off until we figured that out. Right. Don’t don’t go, don’t go beat, snorkel out.
[00:11:36] Prasanna Malaiyandi: And maybe that’s also where you get some of those early wins before you take on those. So you get sort of the buy-in from upper management that, Hey, they are doing the right things. They are making improvements.
[00:11:47] Snorkel42: Yeah, absolutely.
[00:11:49] W. Curtis Preston: Yeah,
[00:11:50] Snorkel42: One of the, the biggest allies of InfoSec people that they forget about is the CFO. The CFO is the person when you’re doing these sorts of things that you want to have in your back pocket to be able to go have that conversation of, Hey, where is our money actually made?
Because I want to know, Hey, what, what divisions of this company aren’t really contributing that much to the bottom line. Cause those just became my test case the things that I’m really
not sure about. Let’s take them down because that’s not going to, you know, that’s not going to ruin our end of quarter numbers.
So they’re going to start this off as a retailer. So that was obvious. Don’t take down the stores under no circumstances do you take down the stores.
[00:12:26] W. Curtis Preston: Right.
Right.
[00:12:27] Snorkel42: but legal? Go for for it,
[00:12:29] Prasanna Malaiyandi: Yep.
[00:12:30] W. Curtis Preston: I don’t know how legal would feel about that. But yeah, no, I understand what you’re saying basically. So w every change that you made, you don’t have to roll it out. Company-wide you, you put it into places where you felt that it would do, you know, hopefully the change would have a minimal impact, but if it did have an impact, it would have a minimal impact to the company, because it only made legal, not be able to do something for a day or two,
[00:12:58] Snorkel42: Absolutely.
[00:12:59] W. Curtis Preston: which is a very different thing than no one can log into the cash registers for a day or two.
[00:13:03] Snorkel42: Absolutely. In retail, taking down the chain is the worst thing you could possibly do.
[00:13:08] W. Curtis Preston: Yeah, exactly. So for the record, I actually started in retail. I, I worked a hundred years ago. I was a shoe salesman at a, a chain called Kenny shoes, which no one. Under 25 even know exists, but you know, it used to, it was the parent company that created Footlocker. So Footlocker is still around, but Kenny shoes was its own store.
And I worked in, uh, retail. So I know I also worked at some, some what we now call big box stores. So I know what it’s like to be at the receiving end of that. And when, uh, when corporate, when corporation changes things and then poof, you know, you, you suddenly can’t do your job. That’s unacceptable.
[00:13:53] Prasanna Malaiyandi: So you went through this exercise, you had this process of, um, going for six months, doing a security update or roll out a week. And then you started writing about this small things that people can do to improve their security posture. And where did you go from there? Like, did you think you would keep writing this long because how long have you been posting on Reddit for your security cadence?
[00:14:21] Snorkel42: So I, I only started it in January. I did it as a new year’s resolution. Um, and it, the idea came to me. I was on a, on a different podcast. Um, and we were talking about InfoSec and we were talking about a term that I believe Wendy Nader from duo security, uh, coined, which is InfoSec poverty. Um, and it’s basically in reference to companies that just don’t have the resources to have dedicated InfoSec people or InfoSec tooling, and how, you know, it’s not really fair to expect these companies that just don’t have those resources to really be able to stand up against you know, the modern era of security threats. Um, so on this podcast we were discussing, you know, what do, what can we as InfoSec professionals do to help those companies? Um, and it’s been kind of living rent free in the back of my brain since I was on that podcast. Um, so as I was approaching the new year, I was like, you know what, I’m just going to hop on Reddit, starting in January and make that weekly post and see if I cant’ help um, you know, some of the folks in the sysadmin sub Reddit, which, you know, the sysadmin subreddit, they have the, um, the flares for everyone. And there’s a lot of them that list themselves as Jack of all trades. And those are those sysadmins that are working in smaller companies. And they’re, you know, if it plugs into the wall, that’s their job.
Um, and you know, those companies are exactly what InfoSec poverty is calling out of. You know, you have these brilliant sysadmins who are heavily overburdened, and they just don’t have the time to focus on this. Um, and they just kind of need someone to say, Hey, you know, this week, why don’t you disable this one thing that comes out of the box in windows and you do not need, and it creates a massive security risk.
Um, yeah, so I started in January and I have a nice long list, uh, in one note of post to make. And you know, every every week around Wednesday night, I just pull one up and I write a quick blog post.
[00:16:08] W. Curtis Preston: Yeah. Cause you can’t, you can’t schedule Reddit posts, Right. I don’t
[00:16:12] Snorkel42: Right. You can, uh, you can put them in drafts. So I write them
Wednesday night and Monday morning I remove it from draft,
[00:16:18] W. Curtis Preston: got it. Yeah.
that was Paul’s InfoSec weekly, I believe. Was it the podcast where you were, right, right. Yeah, Shout out to them. Um, so, uh, so you said you started in January, So, you’re what, uh, like eight or nine posts in on that. And did this, this, uh, this, what do you call it? Um, or maybe like 10. I don’t know. I can’t do math. Anyway. This is, um, the ransomware posts were, where did that fall into that? You know, the
[00:16:54] Snorkel42: yeah, it’s a fun question because literally, since I started this since post one, I’ve had people messaging me on Reddit saying, Hey, could you do something about ransomware? Um, cause it’s you know, it’s a top of mind topic, especially for the smaller orgs, that’s the big boogeyman. Um, and I’ve been honestly kind of Mr. Miyagi’ing it in terms of, well, everything I’m posting really has to do with ransomware. You just don’t realize it. Um, but when.
[00:17:20] W. Curtis Preston: reference by the way not sure if everybody listening will understand that reference, But, very nice reference.
[00:17:26] Snorkel42: But, yeah, so my expectation was I haven’t, you know, a list of posts that eventually I was going to say, Hey, you know, if you’ve been messaging me about ransomware, go read these posts. This is what I was driving at. Um, but then, uh, when Russia invaded Ukraine and the Conti ransomware groups, uh, came out and said that anyone that takes up arms or, you know, it goes against Russia.
We’re going to come after I got flooded with people saying, no, really, please. We need something. So hence the title, the, okay, fine. Let’s talk about
[00:17:54] W. Curtis Preston: Yeah,
[00:17:55] Snorkel42: Um, so I decided, yeah, it was time to just at least take a truncated approach to it.
[00:18:00] Prasanna Malaiyandi: How did you approach because ransomware is such a huge topic, right? I know Curtis, you and I, we talked about it, but just sort of your thought process behind like the series that you wrote and how do you get such a dense topic out there? Because there are so many different ways that ransomware can attack you and so many different, uh, crews out there with different methods.
So how do you sort of generalize it, especially, like you said, for those people who don’t have the time to research and follow up everything related to InfoSec, right?
[00:18:32] Snorkel42: So I giggle when you say dense. Cause one of the other pieces of feedback I get quite frequently is that my posts are too long. Um, but yeah, so my take on ransomware is that companies tend to focus on the exact wrong spot. Um, and I apologize for coming on to a backup, um, related podcast and say that most companies focus on backup and that’s, that’s
[00:18:57] Prasanna Malaiyandi: No, no.
[00:18:58] Snorkel42: effort.
That’s the, that’s the thing that
hopefully saves the company when everything else has just gone poorly for you.
[00:19:04] Prasanna Malaiyandi: And don’t Don’t worry, snorkel. We have the same opinion as well, or at least I do, right. That
it’s just a last resort, but you should really be protecting yourself upfront.
[00:19:14] Snorkel42: Um, and so the thing that comes, that happens every time, there’s a major ransomware breach, um, is, you know, it hits the media and everyone starts talking about the indicators of compromise. It loves talking about indicators of compromise because it’s easy to deal with, you know, how did they get in?
I was an email. Well, where did the email come from? What was the subject? Did it link to something, where did the link go to? What did it download? What was the hash of that downloaded and on and on and on and on. And because it’s easy then to go into your controls and, oh, we’re going to put in our spam filters to block that address.
We’re going to block that domain. We’re going to put in our, uh, endpoint security tools to block that hash, but it’s all pointless, right? Because that breach is done. That entire infrastructure has been burned. There was nothing left of it. So, you know, you’re, you’re reacting to something that’s no longer exist.
But when no one ever asks is, wait a minute, how did you know Susie in accounting downloading this attachment lead to their entire VMware infrastructure getting encrypted. And that’s, that’s the real takeaway from every single ransomware breach of, you know, it’s one thing to come in and, you know, the accounting system, one person in the accounting system is encrypted.
It’s another thing entirely to come in. And yeah, the entire network has gone now and we don’t have any data. Um, and that’s really where the security cadence posts come in. And w what I try to focus on, especially in the first post of this is what I would be doing right now. If you are waking up to a world where Russia has invaded Ukraine, and you’re all of a sudden, greatly concerned that ransomware group’s going to come after you, these are the things to start off with, and it isn’t necessarily preventing ransomware.
It’s preventing ransomware from being able to do anything significant. Um, and the nice thing about those controls is it translates to way more than just ransomware. Which is another issue that I think smaller companies particularly have when they’re dealing with InfoSec, as they put their blinders on and very specific attack types, like how do we protect against ransomware?
Oh, we get good backups. Well, how about, how do we protect against any sort of extortion attempt? You know, we, we had the lupus group or excuse me, Lapsis
[00:21:24] Prasanna Malaiyandi: Yeah, I think it was locked system. Yep.
[00:21:27] Snorkel42: I had been talking about all week and I just blanked on there anyways, you know, going after Nvidia and Okta and Microsoft and LG, really their playbook is the exact same as a ransomware group.
You know, ransomware only exists, not because they care about encrypting your data. They’ve went to extort you for money. Ransomware shifted, shifted recently to exfiltrating data because people had good backups or had restoration methods. We’ll find let’s steal the data. Cause we never really cared about encrypting the data.
We just needed that incentive to get you to pay. Um, so when you take a step back and look at how attacks function from the ground up and started going at the common denominators. You, you really don’t care about what the actual end objective is any longer because the controls are there to make sure that they never made it past sending that initial email, um, or, you know, tacking this particular vulnerability you had exposed to the perimeter for a week.
[00:22:23] Prasanna Malaiyandi: Because those vulnerabilities will constantly be evolving. Right. And so you kind of need a generic. Protection scheme, if you will, rather than something tailored for a particular ransom group, but that comes after you.
[00:22:36] W. Curtis Preston: Yeah, the rent
A ransomware attack is it’s the conclusion of, you know, what, like you got infected, but the ranch, I don’t know if I’m saying, I’m not saying this right, but It’s I want to say it’s the symptom, but it is actually the infection. Right. But that
[00:22:57] Prasanna Malaiyandi: not the cause.
[00:22:58] W. Curtis Preston: The problem is what allowed them to get there in the first place.
[00:23:01] Snorkel42: You’re absolutely right. And it’s, it’s interesting that it is a multi-tiered product at this point. There are, there are groups out there that sell you the initial breach. Um, so if you and Conti is one of the groups that are suspected of doing this, that they don’t do the initial breach, they buy the breach, they buy someone who already has the foothold and then use that foothold to do the actual encryption.
[00:23:24] W. Curtis Preston: Hmm.
[00:23:25] Snorkel42: And so you have this entire life cycle of, you know, third-party vendors that lead up to the final breach. Right. Um, you know, you know, Conti may be purchasing someone else’s exploit kits, um, someone else’s encryption kits. So you’re exactly right. That there’s the initial breach. That’s really your first opportunity.
And when you get to the point where things are encrypting, so many other things have been missed by that. That at least if you get to that point, you know, you have a lot of great opportunities to prevent it from happening again. Cause you should have learned so much up to that point of, oh my gosh, they got the phishing email through my end user is able to download this thing.
They were able to click this link to this weird domain that was stood up yesterday. Um, they’re able to execute a program after they downloaded it off the internet. I mean all these different controls that had to go poorly just to get to that point of encryption.
[00:24:18] W. Curtis Preston: Uh, speaking of phishing, I did see something like this is just a couple of days ago and they were, and, and again, I, I don’t remember exactly where I saw it, but it was like, it was saying that phishing had surpassed, uh,, that it now become the number one method of attacking companies versus I guess, uh, a standard exploit, I guess, would be number two, right?
A standard sort of direct hacking attempt the phishing had become the number one. I don’t know if you,
[00:24:50] Snorkel42: No,
[00:24:50] W. Curtis Preston: sounds like you saw that as the number one.
[00:24:53] Snorkel42: Yeah. Actually someone that, one of the things I really liked with the security cadence post is when people get in and correct me, or, you know, point out other things, because I’m certainly not an expert in all things. Uh, but in the first ransomware post, I made a person who works for a cyber insurance policy holder actually called me out and said, yo, phishing is the number one for sure.
But right close on its heels is the proxy shell exchange vulnerability, uh, which is a vulnerability from last year and, you know, impacting on-prem Exchange, uh, deployments and, you know, still plenty of unpatched boxes out there. But yeah, you know, you get these massive blips, right. You know, a log4j S sort of thing.
That is a crazy large vulnerability. That attackers jump on quickly. But the internal one is always fishing. There’s always social engineering is the quickest path to get past your perimeter.
[00:25:44] Prasanna Malaiyandi: Yeah. And especially with some of these large spikes, you also have the long tails, right. In terms of how long it takes to get every single system out there patched. And you’ll always have systems out there which don’t go patched for so long and still continues to be an attack vector. Right?
[00:26:01] Snorkel42: Right. And it’s that InfoSec debt that, that again, of, you know, a company that hired someone else that comes stand up their IT infrastructure one time and it’s been neglected ever since. And there’s no one patching those systems that are running their exchange 2003 deployments. I mean, they’re out there.
[00:26:20] W. Curtis Preston: the way. Here’s what I want to say. Who the hell is still running on prem Exchange. That’s all I want to say about that. And why aren’t you using 365? That’s all I’m saying Microsoft. You’re welcome. I’m just saying I don’t, it’s just, it’s just
[00:26:36] Prasanna Malaiyandi: Wait, wait, you forgot to add one thing to that, Curtis,
[00:26:39] W. Curtis Preston: what’s that? What’s that.
[00:26:40] Prasanna Malaiyandi: if you are using Microsoft 365, make sure to back it up.
[00:26:44] W. Curtis Preston: Yeah, absolutely. Yes. Thank you. Because Microsoft isn’t doing it for you. Yeah. Yeah. It’s a standard thing. We have to mention here on, on the podcast,
[00:26:53] Snorkel42: So a dropper is typically the initial thing that gets downloaded. So if you look through a normal, any sort of malware campaign, we’ll keep it as ransomware that, you know, I sent an email, uh, as a, as an attacker, that’s a phishing email. And then the whole point is to try to trick someone into clicking a link and downloading the program.
Um, or maybe it’s attached, uh, maybe it’s a word document or something like that it’s attached, but it’s something small and. Typically, you’re going to see it as a document macro. Um, and the whole point of it is that’s the simple, easy thing that’s going to slip through, you know, your, your various defenses, because it’s just a word document, but you enable the macro in the macros, what reaches out and downloads the current malware.
And there there’s a few reasons for that. A big one is that malware could potentially being, be being generated on the fly, meaning that the definition that’s behind that, the hash for it, or, you know, the, the detection mechanisms that more traditional antivirus is looking at won’t have those definitions.
Cause it was generated at the moment of downloads. Um, you know, we’ve just minor changes, but just to throw off that hash, um, but then that’s the thing that actually gets downloaded and executed and, um, you know, causes you all your problems. And, you know, from there it could be any number of things. So as we were saying that there are people who would just tell you that footprint, right. Or that foothold.
[00:28:11] W. Curtis Preston: Right.
[00:28:12] Snorkel42: That dropper could download just seed, too. Just something that’s calling back saying. Yep. I got something running on this computer and that’s it.
[00:28:19] W. Curtis Preston: Hmm. All it could literally. Oh, okay. So he could just literally sit there and wait for the second group. That’s going to purchase that. And then they download the malware that they want to download.
[00:28:31] Snorkel42: Right. So.
[00:28:31] W. Curtis Preston: That’s what you were referring to earlier. And so it looked like the, and again, this is common sense to you, but not necessarily to everybody, it looked like, you know, your best advice was to, to stop. Ransomware is to just think about how ransomware works when it gets in, when that dropper gets in. You’re not going to, I mean, yes, you should do user training and yes, you should do, you know, you should do all those things. And, but you should just assume that at least one of them is going to get it wrong.
I mean, I remember back when I was, uh, you know, 25 years ago when I was at a bank, we did regular InfoSec training with every new employee. And one of the things we constantly said, well, Uh, no one in it will ever call and ask you for your password ever, ever, ever. And then we would, and then immediately after the training, we would call them and ask them for their password and still a percentage of them would give it to us. Right. Um, So. you, you do the training, but then you just sort of assume that that’s going to, um, you know, um, that th that somebody is going to click on the wrong link. And so then you just think about stopping that malware at that point, you know, stopping them from accessing a command and control server, looking for, you know, this, this weird, you know, domains that stood up yesterday, domains that were stood up a long time ago, but just suddenly when active, um, you know, the limiting lateral movement inside the company, all of these things, uh, what, what did I miss.
[00:30:18] Snorkel42: A big thing that a lot of ransomware particularly will do. First thing is start deleting, shadow copies as a quick restoration point. So that is a pretty dead giveaway of, you know, you get the event ID that shadow copy was just deleted.
That’s
[00:30:31] W. Curtis Preston: And you’re referring to VSS there, right? The windows shadow copy. Yeah.
[00:30:35] Prasanna Malaiyandi: So I had a question for you snorkel about that one. Is, does that prevent backup apps from actually running that might leverage VSS and shadow copies?
[00:30:46] Snorkel42: Maybe it is the short answer, but depending how you attack this, if you’re, if you’re attacked for this is just, I want an alert on anything that delete shadow copies. Well, you know, if you have a backup solution that makes use of shadow copies and deletes shadow copies, then you know, that’s something that you tune out, right? So you need to know the source of what deleted then that should be your event ID and you just tune that one out.
[00:31:08] Prasanna Malaiyandi: Gotcha. So then you should only look for anomalous events that happen.
[00:31:13] W. Curtis Preston: Typically backup apps are going to, um, create a, create a shadow copy just to have a stable frame of reference and then delete it when they’re done.
[00:31:24] Snorkel42: Your backup app is probably running as a service. So that’s going to run a system or whatever your backup, um, username is, or a user account is, whereas your ransomware is likely going to be running as that end user.
[00:31:34] Prasanna Malaiyandi: Hm.
[00:31:35] Snorkel42: So when you ask yourself, does an accountant have reason to be deleting, shadow copies? Probably not.
[00:31:42] Prasanna Malaiyandi: So you can look for all these patterns and determine what’s real versus what’s not because I guess that’s the other hard part in InfoSec is like tuning out the noise or the normal behavior versus what’s anomalous.
[00:31:52] Snorkel42: Right. Um, and you know, it all starts with really, really good logs. you need to have that log information and then what’s going on in your systems. But honestly, going back to deleting shadow copies, of the other call-outs I made from a higher level, it’s just looking at what would, you would expect an end user to run, especially from the command prompt.
Right. You know, do you expect someone in legal to ever open a command prompt, let alone, you know, run whoami or run nets, you know, and start looking around your network. Probably not. So if someone in legal opens up a command prompt, that right there, it might be enough for you to go well, that’s weird. Start running, you know, typical attack commands, or, you know, living off the land commands. Now it’s real weird.
[00:32:35] W. Curtis Preston: And what would you use to watch for. Th there was a tool. I forgot its name that you mentioned about that.
[00:32:44] Snorkel42: Uh, so the tool I mentioned in one of my blog posts was raccine, which is so vaccine with an R, um, which is a tool that just monitors for shadow copy deletion, and just kills any process that does it. Um, the problem is it doesn’t discriminate. So again, if you do have a backup tool that does make use of deleting shadow copies, it’s going to kill that process for you.
Um, but if you don’t have that limitation, it’s a really handy, little quick, simple solution.
[00:33:12] W. Curtis Preston: but can you tune that or do you need another tool that’s tuneable.
[00:33:17] Snorkel42: Uh, well, it’s, it’s open source, so you can certainly modify the code, but no, uh, the current version that exists does not have any sort of options for that. It is, uh, a one and done sort of thing.
[00:33:28] W. Curtis Preston: Gotcha. Okay. yeah. So that would be, that’d be a perfect example of, like you said, when we were talking earlier, let’s try this. Right. Hopefully it doesn’t kill the backups, but if it does kill the backups, it would be pretty obvious because all the backups will fail because they’re unable to create, uh, the shadow copies.
[00:33:47] Prasanna Malaiyandi: I think one of the ones, and I don’t know which article number was from that I thought was very unique that you brought up was a different way to sort of trick the ransomware, um, into sort of not destroying your entire infrastructure. I think one of the examples you brought up is sort of creating hidden drives and book-ending normal drives available on that system. So ransomware kind of get stuck, or you can monitor for that.
[00:34:14] Snorkel42: Yeah. So, I mean, it’s part two, in case you’re wondering, um, so the, the actions on objectives posts, so, you know, we have the, the initial infection, you know, they mapped your network. They’re starting to spread out. Now they’re actually going to start trying to attack, you know, at this point. 1, you you have to call out that in 2022, one hopes that your endpoint security software, you know, whatever anti-virus, anti-malware, you’re running sees process X is encrypting word document Y. I’m going to kill it. If it doesn’t, you really need to have a come to Jesus moment with your endpoint protection vendor at this point. But you know, if you have something that’s running, that’s actively doing that. Um, at that point, I think one of the best controls you can possibly have. Is feeding it data that you don’t care about and putting alerts on it.
So, you know, as you were saying, Prasanna. One of the things I do is I create, um, deceptive file shares on my network. So just file servers that, you know, on their own separate windows box, doesn’t don’t have any useful data on them. I actually just clone my actual production file names and structures and just put random data in them.
Uh, but then I make drive mappings to my end points. Um, hidden drive mapping said, you know, from the windows GUI, you can’t see them, but, you know, from command prompt, you can. Um, and I just book in my valid drives. So, you know, you have a home drive at H. So put something before that and put something after that.
And, you know, hopefully the ransomware will go after those first. Um, and then I put just, you know, files on those servers that I monitor. If anything gets changed for them, no reason for anyone to ever touch these servers. No reason for anyone to touch these files. So, if anything gets modified, then it sets off alerts and I know something weird is going on.
Um, and hopefully it buys you enough time to, you know, to remote in at three in the morning and down whatever’s going on. Um, and yeah, the other thing I, I offered up in that it was what I coined as a ransomware tar pit of an actual service that’s just running or monitoring that server. And it just starts seeing files getting modified. It starts generating more. Um, so, you know, Hey, the ransomware, it hit my fake file share. It file one. Well, here’s four more files for you and it’ll just keep going and just keep going. And you know, it may not be foolproof, but it might just depending on how the ransomware is written, it might just put it in the loop that it will never escape from.
[00:36:37] W. Curtis Preston: Yeah. I mean, uh, the concept of honeypots is not new, but I like to sort of modifying it to, you know, the world of, of, uh, ransomware. I agree with everything you said about stopping it in the first place. What about, um, detecting data exfiltration? What, what do you think, um, companies can do there?
[00:37:03] Snorkel42: Yeah, so it’s a definitely a trickier process. Um, mainly from a tooling standpoint, at this point, you’re probably going to have to open up the wallet. Um, but you know, there there’s one, there’s just a basic security controls of content filtering and making sure that your end users don’t have a path out to the internet for mass file transfers. So for most enterprises, you probably don’t need more than HTTP and HTTPS from your workstations. Um, so you know, a lot of those transfers are trying to transfer out via FTP or, you know, a more traditional file transfer method that shouldn’t be allowed. Um, but going further, you need to look at, you know, what sites are out there for allowing mass transfers, you know, to, to keep it simple, to do all your users need to be able to reach Dropbox, um, or box or Google drive or any of that stuff. If the answer’s no, prevent it because that’s an exfiltration method. Um,
[00:37:59] W. Curtis Preston: Let me ask you about that. I guess I had this, this apparently misconception that they would be sending these exfiltrated files to something that they owned and controlled.
[00:38:15] Snorkel42: Right. So, and that’s, so I’m trying to build this out from, what’s easier to, harder to implement, um, going to that point. Yes. So when you get to the point of we’re going to just transfer to something we own. And honestly, at that point you’re probably hitting up AWS or Azure or something like that. And that’s where it gets really messy.
Unfortunately, in the cloud world, we live in, you can’t exactly block Azure. Right. Um, but that’s where I started looking at DNS security. Um, and one of my absolute favorite controls is just blocking newly registered domains or domains that have been parked for years that have all of a sudden gone live.
Um, cause a lot of the attack infrastructure, and this is honestly a great way of also stopping the initial drop or download because there’s a good chance that that’s going to go somewhere that was just newly stood up. Um, but yeah, so that, that is another control where you might be able to just stop them from being able to get to whatever destination there they stood up to accept these files.
Another point is always just a basic security, um, control of proper ACL’s. Um, you know, when it gets to data exfiltration again, you know, keep picking on poor Susie in accounting, but Susie and in accounting shouldn’t be able to get to your HR documents. She shouldn’t be able to get to your operations documents.
Yeah. They might be able to export out your payroll, which that’s terrible, but you know, your payroll showing up on pay spend tomorrow is a bad day. It’s not an end of the company sort of day though. Right? Like that’s not trade secrets going out. Um, you know, so, so that’s another control. Another thing that I would absolutely call out though is still honey documents.
You know, having documents for them to interact and transfer out that as soon as you see somebody interact with that, you’re, you’re figuring out what process it was and figuring out where system that came from, just stopping that information. Um, and then going into the, probably the more logical solution, but definitely at a cost it’s just the behavioral controls because in that exfiltration attempt, there’s going to be an end point that’s all of a sudden transferring a lot of data out to a new source that has never been seen before. And if you really know what normal looks like in your network, that should stick out like a big, big red flag. Um, and it’s a very hard thing to do with free solutions, but there are plenty of security products out there that are all about mapping, how your end points interact with each other on the network and what looks like normal, what isn’t normal. Um, I think it’s money well spent for those types of controls.
[00:40:43] Prasanna Malaiyandi: Can I ask a question about an earlier topic you brought up around EDR. So if most, or if EDRs are useful for detecting when encryption is happening and killing processes, et cetera, if that’s the case, would a lot of these ransomware attacks be prevented to start with. Or, and is it that companies who’ve been hit with ransomware have not deployed EDR solutions in their environments?
If EDRs can detect when encryption is happening on endpoint devices, if a company has deployed EDRs, does that mean they’d be able to stop ransomware? And so a lot of companies who got hit with ransomware. Didn’t have EDRs deployed.
[00:41:29] Snorkel42: Endpoint detection response is what we’re talking about here. So first information security is all about layered defenses, and you never rely on a single defense. Um, in my mind when your antivirus, your EDR, WM, whatever your endpoint security tool is, if that’s the thing that stops the malware, thank God it was there, but that’s still a, oh my goodness.
How many different things failed before it got to the point where that had to step up? Like, I never want to see anything out of that system. Um, that’s not false positives. Cause that’s the fun with EDR is they are a pile of false positive. Um, you know, to your question, did they not have it maybe, um, you know, EDRs are expensive, they are expensive tools. They are great tools, but they’re expensive.
And like so many other expensive security tools that are rarely set it and forget it tools. They are tools that require a lot of tuning and a lot of the finding of what’s right. Excuse me, within your enterprise. Um, but you certainly have a lot of companies out there that are still, you know, with prop with old definition based antivirus, that’s just scanning files and doing your nightly full scans.
You know, like we did back in the nineties, um, and companies that have been very happy to embrace Microsoft defender as their only antivirus. And, you know, there there’s some logic to it. It’s a great solution and it’s free depending on what your office 365 licensing looks like. Um, but I think the number of customers out there that have these large EDR solutions are few and far between. Uh, definitely the companies I’m targeting with my security cadence posts are the companies that probably don’t have that kind of assessment.
[00:43:06] Prasanna Malaiyandi: Yeah. Or even if they did sort of managing it on a daily basis, becomes difficult, especially with everything else they have to do, because it’s not a one and done sort of a deal.
[00:43:16] Snorkel42: Right. And you know, honestly, if you think about it as a, the evolution of antivirus, antivirus was a one and done for the most part, you know, you deployed Symantec back in the day, next next finish. And you never touched it again. Right.
EDRs aren’t that. EDR is, are constant tuning and definitions and breaking things in your environment and having to tune them back out.
And I think that’s also in the recall that I know of a number of companies have thrown their EDRs out because, oh, it just broke everything. That product was terrible. It wasn’t, you just needed the resources to handle it properly.
[00:43:48] Prasanna Malaiyandi: Yeah.
[00:43:48] W. Curtis Preston: I want to sort of round out things here. There’s some things that we haven’t talked about that were obvious ones that were. You mentioned in your first post that, you know, you talked about, you know, you password security, you talked about MFA.
Um, these are things that just everybody should be doing. Uh, my, my personal opinion at this point, you know, th th you know, if you’re not doing MFA on anything that matters, uh, you know, you’re not doing your job and, uh, and you know, that’s my opinion for what it’s worth, but MFA stops so many things.
[00:44:23] Snorkel42: Yeah, I If you were waiting for me to disagree. I wasn’t going to.
[00:44:28] Prasanna Malaiyandi: I have a question for you though. So I dunno if we’re going to talk about it now or later, but I’ve read recently with a lot of the Lapsis attacks as well as other gangs, right. There is a notion of SIM swapping attacks, right. Which sort of hurt some of the MFA approaches taken.
[00:44:51] Snorkel42: So Prasanna, don’t let perfect get in the way being good,
[00:44:54] W. Curtis Preston: I I was gonna I was going to say that that goes right to that, right? Yeah.
Just because it won’t fix everything doesn’t mean you shouldn’t do it, right.
[00:45:02] Snorkel42: but no.
[00:45:03] W. Curtis Preston: not, there is no silver bullet. Right. Um, and you know, a good backup person would never say don’t do InfoSec and a good InfoSec person would never say, do backup or not do backup.
Um, but I think that MFA is just so, and by the way, I, I finally ate my own dog food maybe about two years ago where I just realized that there were a lot of vendors that I personally interacted with, banks and things, that offered MFA as an option. And I finally said, look, I know it’s going to make it harder for me to access my bank account and my, you know, my PayPal and, you know, I, there, there’s only like, I dunno, there’s only like 20 accounts that I felt had that level of information that I needed MFA on. Um, and then, and then, and then I became like this like MFA Nazi, where I was like, I’m mad at them if they don’t offer MFA.
[00:46:05] Prasanna Malaiyandi: I remember what happened when you traded in your phone and you lost access to your MFA.
[00:46:11] W. Curtis Preston: Yeah. So I was using a Google authenticator, not realizing that when I traded in my phone that I lost all of my MFA tokens. And so I switched actually to authy, so that I can, I don’t have that problem. But, um, and now, and now I’m actually looking at a password manager. I think it’s one password that manages both your passwords and your MFA stuff. That sounds nice. So I’m already a big password manager, uh, fan, I just, um, didn’t, you know, I currently have to use two solutions, but Yeah.
[00:46:46] Snorkel42: Yeah. And to your point, Prasanna, about, you know, SIM jacking you know, and not being the silver bullet, the one thing I’d say is not all MFA’s are created equal. Any MFA is better than no MFA, um, but you know, it doesn’t have to be a roadblock in your organization, you know, Fido keys, Titan keys, things like that, that are literally you get the prompt and you tap a, the thing hanging out of your USB port or taps onto your phone is a really, really nice MFA solution.
That’s really easy for your users. And they require something that you have that’s truly physical. Um, and rather than, you know, replying to a text message, you know, there’s, uh, one of the big debates going on in the InfoSec world right now is the, the push notifications of yes, that was me. Um, and it’s right out of, uh, the playbook of lapsis of just that they actually had a picture on Twitter, um, yesterday of one of their chat things.
Yeah. Just spam them a hundred times. Eventually they’ll get mad and hit yes. Yeah, absolutely. And it’s funny, cause I actually had, um, our MFA at work today went a little bit sideways and they started pinging me repeatedly for something I had just tried to sign into it had in the back of my mind, like, Hmm, check the logs to make sure like, this is kind of weird.
Um, but still it’s, it’s better than nothing and yeah. Your end user may fail you and hit. Yes, that was me. Cause I got tired of getting this prompt a hundred times. at three in the morning, but still, um, but as InfoSec practitioners, that’s where we need to step in and go don’t we think it was abnormal that they got a hundred prompts, like did that not set off an alarm right there?
Why is it if we’re getting pinged over and over and over and over again? Um, cause I guarantee you that created a log somewhere.
[00:48:24] Prasanna Malaiyandi: Yeah.
[00:48:25] W. Curtis Preston: All right. All right. Well, the summary statement of your, of your blog series or your post series, whatever you going to call this, and by the way, your posts are long. I, uh, after you made the comment I went and while you were talking, I copied and pasted the three posts into a, uh, Google docs. Uh, one of them is 4,500 words long my friend. I mean, that’s long, even for me, I’m just saying
[00:48:50] Snorkel42: I got to tell you someone the other day commented about how much they liked my writing style. And it was the first time ever in my 42 years of life that anyone has ever said such a thing
[00:49:00] Prasanna Malaiyandi: I actually liked your writing style. I thought it was good
[00:49:02] W. Curtis Preston: This is a reason, this is a reason you’re here is you. Well, it’s a complicated issue. So, you know, I, I jab, but you know, 4,500 words is not that much for, for an issue of this magnitude. Right. But, uh, the first was 2,500. The second one was 2000, but the last one was 4,500. I was like, yeah, the boy could talk.
[00:49:23] Snorkel42: Um, I would just reiterate what I said of so many companies focus on the recovery side. Um, and that focus comes from focusing on what the actual objective was of the ransomware group to begin with. What was it? They were trying to do, whether they were going to encrypt your stuff. So you start there and it’s the wrong place to start.
Start at the beginning, start with how they’re going to compromise your first endpoint. And if you start looking at InfoSec from that perspective, what you’ll end up finding, and it’s a really gratifying feeling is you would turn on the news one day and there will be the latest, massive vulnerability or exploit being discussed.
And you’ll look at it and go, oh, my controls account for that. Not because I built controls around that particular exploit or vulnerability, but because I just built my controls around, how do I walk an attacker through initial foothold, moving laterally throughout my environment, and then acting on their objectives and then how do I stop them at each one of those steps?
Um, so I guess the, the, the too long, didn’t read it to use Reddit terminology. Don’t focus on ransomware, just focus on how attacks function and you’ll get ransomware taken care of by default.
[00:50:40] W. Curtis Preston: Right in the, the only major one that we didn’t discuss, which we really should have is the whole patching thing. Right. I go back to, I’m pretty sure if my, if my memory serves correctly, Wannacry. Was, you know, it was one of the. first big ones that really went, you know, it went haywire and everywhere. If I recall correctly, it was an exploit that had been patched a year prior in a windows. And if you had just been anywhere near up to date, then you’d have been fine.
[00:51:13] Snorkel42: Yeah. And Microsoft is, they are both great in how they maintain with a lion’s grip to a, or with an iron grip to backwards compatibility. And they are also just ridiculous. in they’re lions grip, iron grip on backwards compatibility, but I mean, Wannacry. The other side of Wannacry is exposure to the internet, um, which, and this kind of goes back to the proxy shell thing.
You know, Wannacry has went gangbusters because of all the companies that have Samba, SMB exposed to the internet, which again is you take a step back and go good god, why? But then you go jump on something like showdown and you look like, oh yeah, there’s tons. There’s tons of vCenters exposed to the internet and you go good god, why? Um, and again, it just kind of comes down to, well, I had this one sysadmin who was overworked and doing what they could, and we had this use case and he stood it up, but he wasn’t InfoSec focused. He didn’t know what he was doing. And from that regards and didn’t see the problem with it. Um, so yeah, I mean, to your point of patching.
Absolutely. Uh, but I guess my takeaway is focusing on the big things. And don’t worry about the latest zero day, quite as much. Um, cause I tell you attackers rarely are focused on that. Cause there’s so much low-hanging fruit of the stuff that has been patched since 2018. Anyways.
[00:52:34] W. Curtis Preston: Yeah. you, you mentioned SMB. My other big one is RDP RDP to the internet is just, I just want to slap you. Well, listen, we could talk all day. Uh, I just, I want to say thank you again, and, you know, for coming on the podcast and talking to these really important things,
[00:52:54] Snorkel42: No, it was a pleasure to be here. I really, I have to say when I got, when I got your message, like, wait, is that Mr. Backup? That is Mr. Backup! I was really excited.
[00:53:04] W. Curtis Preston: this is me. I, well, and I’m honored that you, that you knew who I was, so, Hey, you know, we’re, we’re members of the mutual admiration society, Prasanna?
[00:53:13] Prasanna Malaiyandi: Yeah, no, it’s been great snorkel having you on and yeah, great articles I’m will continue to read. I hope you keep doing your weekly posts on security cadence, because I’m sure a lot of people learn a lot of things from that. So
[00:53:25] Snorkel42: I’ll try to make a cliff notes version.
[00:53:27] Prasanna Malaiyandi: yeah, I keep it long. I
[00:53:28] W. Curtis Preston: Yeah. Good, good luck with that. Uh, yeah, he’s easy to find on Reddit is snorkel 42, make sure to check them. out and make sure to subscribe so that you can restore it all.
