SDP 4 COMPROMISE RECORDING

Kip Boyle:
Hi everybody. Welcome. This is Your Cyber Path. We’re the podcast that helps you get your cybersecurity career started or if you’re already working in the cybersecurity career field, we’re going to help you accelerate your career and get better positions, more compensation, all the stuff we want for you. And I’m here with Jason Dion, of course. Jason, it’s great to see you again. Welcome back.

Jason Dion:
Hey, it’s great to see you again in this [inaudible], Kip.

Kip Boyle:
Okay, so if you’re listening to this episode, just as it dropped, it’s August. It’s mid-August. You should be thinking about kids going back to school. You should be thinking about the fall and what’s unique about the fall? Well, in North America, one of the things that’s unique about the fall is hiring season begins again. Nobody’s really hiring over the summer. We’ve talked about this before. There are distinct hiring seasons here in the United States and North America.

That’s probably true in a lot of different places, just that Jason and I haven’t been hiring managers in those other places. So if you’re outside of the US, I want you to reflect on whether the patterns that we talk about here match up with the patterns that you’re observing there. And if they do, then pay attention to this stuff because it matters.

So hiring season’s about to start again and in the private sector, that’s driven more just from a perspective of what are people paying attention to, right? They’re on vacation over the summer just like you are. So they’re really not thinking about, “I’m going to bring a new person in the middle of July,” because that’s a really terrible time to bring somebody on board. Everybody’s gone.

This person comes into the office, there’s nobody there to help them get oriented, and it’s just not a very good experience. So in the private sector, we like to wait until after the high holiday season’s over with, but in the government sector, that’s a player. But there’s something else going on there. Right, Jason? Tell everybody… You know, there’s another thing going on there. What is it?

Jason Dion:
Yeah, so the big thing with the government sector is that our fiscal year in the government goes from October 1st through September 30th. So what ends up happening is all the new contracts get let out on October 1st. And so, if I’m a company and I am responsible for hiring, I don’t know, 50 people to work in the help desk for instance, or 50 people to work in a SOC for a government organization, my new funding starts October 1st.

And the other thing that happens and the reason why August and September is important is, these contracts, they already know it’s happening before October 1st. So for example, let’s say that Kip’s company had the contract for the last five years and he was the person who was running the SOC. Well, if Dion Consulting just came in and knocked out Boyle Consulting, then we are going to have the contract starting on October 1st.

Generally what’ll happen is, we’ll go to Kip’s current employees and say, “Hey, if you guys want to stay here, we’d love to keep you here and you’ll just change from being a Kip employee to a Dion employee, and you’ll be able to keep your job.” But notoriously what happens is, the new company coming in usually will offer the same or less than the current company, and therefore a lot of the people will end up leaving.

And what I’ve seen is generally around 20 to 25% of the people will change over during those contract negotiation cycles, which happens about every three to five years. So if the contract is [inaudible] over, which again, 20% of every government contract pretty much renews every five years, that is a time where there’s a lot of change in the hiring.

And so, a lot of times if I’m the new company coming in October 1st, here in August, I’m putting out those new job postings and I’m doing my interviews August and September so I can pick somebody, I can say, “Okay, you’re going to start on my contract on October 1st.” And so that’s why this August, September time period tends to be kind of a bump. And even sometimes into October, you’ll see that bump as well in the government and consulting sector because those new jobs now have funding.

The other place that you’ll see a lot of jobs that happens, this happens October and November, is the government itself. So if you’re going to usajobs.gov and you’re trying to get a job working as a government civilian, not as a contractor, those tend to be October and November because again, new fiscal year, new money. We now have money for that job and we can then advertise and hire for that job starting October 1st.

So that’s kind of why we have this kind of three to four month season between August and September for the government contractors and October and November for the government employees themselves.

Kip Boyle:
Thank you for unpacking that. And so we sometimes hear from people who are saying, “Hey, I’m trying to get into a defense contractor working world and I’m frustrated because nobody’s responding to me.” And we’re like, “It’s July.” Nobody is going to… I mean, just don’t expect something different than what you’re getting. Pay attention to the seasonality. So-

Jason Dion:
Well, it’s just like if you wait until December 24th and drop some applications, nobody’s going to call you because they’re all on Christmas holiday.

Kip Boyle:
That’s right. That’s right. So we really want you to pay attention to this because listen, if you’re out there killing yourself, submitting applications, and it’s the wrong time of the year, all you’re going to do is frustrate yourself. And job hunting is a frustrating enough thing, right? Let’s remove as many of the irritants as possible, right? That’s what we want for you, is to just kind of optimize your job search.

Well, today what we’re going to talk about in this episode is we’re going to talk about another principle, a security design and architecture principle that’s actually going to help remove frustration for you on the job. And the reason why is because so much in our worlds change all the time. We have new technologies coming along all the time, new products, new versions of products. We are shifting from traditional tech stack architectures to cloud architectures. Everything’s changing.

Isn’t it nice to know that there’s a couple of things in our working world that don’t change very often, that we can actually use in order to learn how to deal with new technologies and new changes. And that’s what these design and architecture principles will do for you. And so we’ve reviewed several already.

And today, we’re going to take a look at one called Compromise Recording and we’re going to unpack that a little bit, make sure you understand what it is and also how to use it on the job because when you use these things on the job, you’re going to produce better work. You’re going to sound a lot smarter than everybody who’s not using these principles. So, okay. Now, having said that-

Jason Dion:
Speaking of sounding smart, the name Compromise Recording, that just sounds like a big fluffy way of saying what this is.

Kip Boyle:
You know, most of these design principles from Saltzer and Schroeder do kind of have strange terminology and I think that’s for a couple of reasons. One is, well, welcome to 1975. This is how people thought back then, okay? This was all emergent and this is the first time that anybody had ever published anything like this. So that’s one reason.

I think the other reason is because these guys were academics and they were coming from that world. And of course, Jason and I, and most of the people listening here, we don’t work in that world. We work in the world where we got to get stuff done and we don’t have a lot of time to sit around, think about stuff, and we got to do things.

And so I think that’s maybe a couple of the reasons, right, why… But that’s also why we have to do these episodes because we don’t want you to go, “Compromise Recording? That sounds like a bunch of gobbledygook junk and then you ignore it. Oh, don’t ignore it. Got a weird name, but it’s cool.

Jason Dion:
And so let’s dive into what it actually is, right? So Saltzer and Schroeder said, in their paper, “In situations in which preventative controls are unlikely to be sufficient, consider deploying detective controls so that if security is breached, A, the damage might be able to be contained or limited by prompt, instant response. And B, evidence of the perpetrator’s identity might be captured.” So really Kip, what is the short way of saying all of those words? What do we need to do?

Kip Boyle:
Well, I like logging and alerting.

Jason Dion:
Yeah.

Kip Boyle:
Right? It’s kind of how people talk about it now. Event monitoring is another way people talk about it. And of course, we’re all very much sensitized to the fact that we’re absolutely drowning in logs and alerts and events. I mean, we have more than we know what to do with these days. And so, while this is a great idea, we’re not quite where we want to be yet. Right? So-

Jason Dion:
Well, I think it was also a little easier back in 1975 to do this, right? Because you could turn on system logs on your UNIX system, for example, right?

Kip Boyle:
Yep.

Jason Dion:
And you’d have one consolidated place to look at your log.

Kip Boyle:
That’s right.

Jason Dion:
And you had one person using a machine, maybe a couple, if you’re using a mainframe at a given time. But the volume was so much less than what we see nowadays in a networked environment.

Kip Boyle:
And you’re just talking about one enclosed system, right?

Jason Dion:
Right.

Kip Boyle:
In 1975, there really wasn’t machines talking to machines in real time. You had to load something on a tape, walk that tape over to another building, then that’s how you did data transfer back then, right? So the world was way more simple.

Jason Dion:
Yep. And you mentioned logging and alerting. You also mentioned event monitoring. The other thing that I think about with this is, when I think about the three A’s of security, right? So we talk about authorization, authentication, and then the third one, which is accounting. And accounting really is this thing of logging that we’re talking about, right? So if you could do accounting or auditing, that is all about logs and alerts and events. And as you said, we are drowning in logs, right? So we need all the help we can get. And one of the first things that I saw that kind of helped this about 20 years ago was a SIEM, right? And a SIEM is a security information and event management system.

It allows you to take all those logs and use something like syslog to gather the logs from all your network devices, all your hosts, put them into a centralized server that we can then query and look at and try to correlate the different logs against each other. Because as you just said, when you start networking systems together, it’s not just important to know what’s happening on your system, but it’s also important to know what’s happening on the database server down the hall and the web server over there, and the client down in the other part of the network. So that’s where I think the SIEMs kind of come in. What are your thoughts on that?

Kip Boyle:
I think that’s a great idea in abstract. So I remember when these systems first showed up about 20 years ago, and I was so excited because I was so frustrated by the fact that the best I had at that time was syslog, right? I could do log forwarding at a syslog, but then to…

Okay, so great. So now I’ve done it all in one place, but how do I search it? How do I analyze it? It was so tedious and difficult. I had to write regular expressions and I was limited by my own imagination because there were no canned search queries, there was no automation. I had to build it all myself. So I was really excited when these systems came along.

The problem that I’ve had ever since, is that it’s still a bear to wrestle these things to the ground and make them do stuff. It just requires… First of all, they’re expensive. They’re super expensive. None of my mid-market customers can afford these things. So you have to pay license fees, you have to deploy these things. Even in the cloud, they’re expensive.

The other thing is that they have to be operated by people who’ve been trained and dedicated to this. And my mid-market customer can’t afford that either, right? They rarely even have a single person dedicated to cybersecurity, let alone a single person who’s doing this kind of log analysis. Okay, so then you might say-

Jason Dion:
And just for the benefit of the audience, when you say mid-market, can you define that for them? Because I know a lot of people may be like, “What do you mean by that?”

Kip Boyle:
Right. Okay, thank you. So mid-market is defined in a number of different ways depending on who you talk to. So if you talk to the Small Business Association, they have one definition. Talk to the US Chamber of Commerce, they have a different definition. I’m going to give you my definition.

So generally speaking, mid-market is companies under a billion dollars of annual revenue, but who have about 50 to a hundred million dollars. So there’s a range there, okay? And that’s the range that I’m talking about. If you have more than a billion dollars of annual revenue, you generally can afford to hire dedicated cybersecurity people.

If you’re under 50 million to a hundred million dollars of annual revenue, you’re still worried about product market fit, right? You’re still trying to get legs under your business. And so cybersecurity just often doesn’t rise to the C-suite as an issue, unless it’s something that’s keeping them from closing deals, in which case it’s all they think about. But that’s mid-market.

Jason Dion:
And so when you’re dealing with a company, generally we’re talking something like 50 to 150 employees, I would guess?

Kip Boyle:
Something like that.

Jason Dion:
Yeah.

Kip Boyle:
Yeah.

Jason Dion:
So having a dedicated… You may be lucky to have one or two IT people, but there’s probably not going to be any cybersecurity people.

Kip Boyle:
Right. And IT people, you might have a few on your team, but most of your IT heavy lifting is probably being done by a managed service provider. You’ve probably outsourced that or you’re just doing everything on a shoestring and you’re having operators do most of the IT work also. I can think of a customer of ours that brings fruit in from the fields, cleans it, washes it, sorts it, puts it into boxes, and sends it wherever it needs to go.

And they have the thinnest IT crew you can imagine. And most of the people on the packing line are actually kind of keeping the systems running and the dedicated specialists are just running around with their heads cut off fixing stuff that breaks. That’s all they do is whack-a-mole all day long. So they don’t really have the time and the talent to really focus on something like a security information and event management system.

Jason Dion:
Yeah, that makes sense. In my time in the government and working at different places, some of the places I worked, even though we don’t do revenue, because you said a billion dollars in revenue, the government as a whole, yeah, we’re definitely more than a billion dollars of revenue. We’re a very large organization, right? But we break it up into very small organizations.

And one of the organizations I was working in, we had about 200 people on staff. We had five IT people, and most of our IT was done through a managed service provider. And our job was really just acting as the conduit of, we did touch labor if there was issues, and if there wasn’t, we had to call in to the MSP who [inaudible] into the systems and did all the hard work there, right? And that was 10, 15 years ago before MSPs were really as huge as they are now. But nowadays, everybody uses MSPs for that kind of stuff.

Kip Boyle:
That’s right. And as I’ve said on another podcast that I am on, your IT person is not your cybersecurity person. And this is a fallacy that I see senior decision makers in the mid-market say all the time, right? Like, “Oh, yeah. We got cybersecurity covered. We have this fantastic IT team.” Not exactly.

Jason Dion:
Yep. Now that being said, your IT team is part of your cybersecurity team though, right?

Kip Boyle:
Yeah, yeah.

Jason Dion:
So I think it’s really important that, and again, for listeners, if you’re looking for a way to break into cybersecurity, one of the best ways still, is to work in IT. Because when you work in IT, you’re doing things like, doing all your security, you’re doing all your logging, you’re actually looking through some of those logs, but you’re not necessarily a cybersecurity person, but you’re starting to touch that stuff.

Kip Boyle:
Right.

Jason Dion:
And that builds the experience that you can move into one of those cybersecurity roles. And then-

Kip Boyle:
Absolutely.

Jason Dion:
The other thing you’ve mentioned, MSP, right? We mentioned all this massive amount of SIEM data that gets consolidated. One of the other things that’s out there, one of the managed service providers you can hire is a cybersecurity organization or a virtual SOC.

Kip Boyle:
Right.

Jason Dion:
And they can do a lot of this for you, but at the end of the day when they find something, they’re still going to have to come to you and say, “Look, I found this. This looks suspicious. Go check it out.”

Kip Boyle:
Yeah, absolutely. And I’ll be also be honest with everybody about outsourcing your security operations center. I haven’t had very good luck with that and I won’t unpack why during our conversation today. If you want to know more, just send me a message. I’ll tell you all about it. It’s just a difficult thing to get right and I have a lot of customers that are frustrated by it. I’ve been frustrated by it when I’ve tried to do it.

But with the point you’re making is a good one, which is in the mid-market, the security information and event management solution that we’re talking about is just typically something you’re not going to do on your own. And even if you’re a giant organization, it still might make sense to outsource it. And so, that’s a very different way of operating. You’re using contracts to hold people accountable instead of directly interacting with staff. It’s just a very different way of getting things done.

Jason Dion:
And the other thing with that is if you’re in a very large organization, for instance, I worked with the US Navy for years. We had a single SOC that ran the entire Navy network, and it was located in Suffolk, Virginia at a place called NCDOC, the Navy Cyber Defense Operations Command. And so even though we didn’t outsource it from the Navy out to contractors, there was a lot of contractors working in that place. But all of the other commands across the Navy, there’s 2000 plus commands, all the ships, all the bases, all that stuff, they didn’t have their own SOCs. All their stuff got fed back into NCDOC and NCDOC worked as a centralized SOC around the world.

Now, there was people in those areas, so if you’re like, “Oh, look, we found there’s an issue over in our Yokohama Naval Base, right? We need somebody to go over there and fix that.” We wouldn’t necessarily have to send somebody from Virginia all the way out to Tokyo, but we had people in Tokyo in a small footprint that could then work back with the folks in Virginia to do that. And so that’s another thing you’ll see in these larger organizations. If you have a global presence, there’s probably going to be a single SOC for your company. And then they would have their tentacles reached out into all the local branches and sites.

Kip Boyle:
Right. And so they’re all doing Compromise Recording, but of course, they’re doing more than just recording compromises. They’re actually detecting compromises. And then of course, if they detect something, then that kicks off the incident response functionality. And now we’re trying to contain damage. We’re trying to eradicate the intruders of the malicious code. We’re trying to recover…

So Compromise Recording is just really the first step in a larger sequence of steps, which you’re all familiar with, but Compromise Recording is kind of where it all began, right? This idea that we should have a log that’s trustworthy that we can go to when we want to know, is somebody trying to do something that they’re not supposed to do? Or maybe I saw something bad happen and I’m trying to figure out, “Well, who did that?”

And if you’ve ever read a book by Cliff Stoll called The Cuckoo’s Egg, which was published in 1989, that whole book is about Compromise Recording. And an astronomer, Cliff Stoll, who didn’t have funding to do astronomy and ended in the backwaters of a data center and he spent months and months and months tracking down an intruder in his network [inaudible].

And the only reason he knew that that person was there is because they found a 50 cent error in their accounting system and that just kind of led the whole thing. It is a fantastic book, highly readable. If you’ve never read it before, I recommend it. So what else is there to say about Compromise Recording, Jason?

Jason Dion:
Yeah, I got a couple of things.

Kip Boyle:
All right.

Jason Dion:
So one thing is, it’s great to log things and we can log whatever you want and you’ll meet Compromise Recording. But if you’re never looking at those logs, they do you absolutely no good. The other thing is, you want to make sure your logs are logging the things you care about, right? Because you can log as much or as little as you want depending on your systems, depending on your capacity.

And this is something I talk about a lot in my Security+ course when we start talking about how do you do auditing and accounting? I can log every single event on your system, but do I really care about every event? Well, no. And if I do, then I’m going to bury myself in this mountain of data and now it becomes even harder to find what really is bad.

Kip Boyle:
Yeah, [inaudible].

Jason Dion:
Also, on the other side, I’ve seen people do the other thing where they don’t log enough, and then when it comes time to look at a problem, you don’t have the data you need. Yesterday actually, I was working with my CTO and we were going through an issue we were having with OpenAI, which is… Who runs ChatGPT. We have API based access. We have some software we’ve built, and we use their software to run a lot of the AI queries to help us with some of our work.

Kip Boyle:
Cool.

Jason Dion:
Well, the issue we’re having is, it appears there’s something wrong on their side. And when we’re sending a prompt like, what version of ChatGPT are you using? Which is very, very few words, it should be 10 tokens. It’s registering as 10 or 15 or 20,000 tokens. And we’re like, “What’s going on?”

And we went through about $500 in about 18 hours and normally, we do 20 bucks a day. So we’re like, “Something is wrong.” And we saw that based on the utilization going up, right? And we went back in to look at our logging to say, “Well, what’s going on?” I’m like, “Okay, show me the prompts people are using,” because we have our own custom system that then grabs all the prompts and sends it out.

Well, apparently, we weren’t logging people’s prompts or the responses back. So all we know is, a request was made and then we got a bill for 20,000 tokens and we’re like, “Something’s wrong,” because you usually get charged… Every 1000 tokens is like 2 cents, right? And every word tends to be a token essentially in AI. And so we started looking into it and it’s like, well, our logging was insufficient for this purpose of troubleshooting, right? We had all the security logs. We just didn’t know what people are doing with it.

So now, we’re adding in logging where we can say, “Okay. I want to see the prompt,” and then I want to know how many tokens each prompt was so we can see if it’s an error on our side or an error on their side, and be able to troubleshoot those issues. So that’s another reason for logging.

And then the other thing I would bring up is now we’re talking AI and ML. A lot of people ask me, “Are our jobs as people who do log analysis in a SOC, going to go away?” No, they’re not. Right? But there may be less of you and you’re actually going to be happy about that because the most boring work I’ve ever done in cybersecurity is being the person looking at logs and going benign, suspicious, malicious, benign, benign, suspicious, benign. Right? And you’re just going through good, bad, maybe, good, bad, maybe. Right? And that is a boring job to do 12 hours a day, four days a week, or whatever it is, when you’re working rotating hours in a SOC.

So I love the fact that we have the ability to set up alerts. I love the ability that we have the ability to set up rules. If you’ve ever worked with SNORT, which is an intrusion detection, intrusion protection system, you can create your own rules and it’ll actually integrate in with your SIEMs as well. And so as it’s seeing the traffic, it can go, “Ah, this looks suspicious. Let’s take a look at it.” And one of the interesting things, as we were looking through the server logs yesterday as we’re troubleshooting this issue with the API, we were seeing attacks against our server, right?

Kip Boyle:
Wow.

Jason Dion:
And we’re seeing in hex code the different stuff in our logs because we were logging that and we’re like, “Oh, what is that?” And we actually, there’s something you guys can do as well or those who are listening, you can look at your logs, grab that code, put it in ChatGPT and say, “Explain to me what this hex code is trying to do,” and it will go in and convert it from the hex code to the assembly, from the assembly into what it means. And then it said, “Oh, this is a remote shell attack.” They’re trying to get a Windows RDP session on your server, in which case we know, “Okay, these people obviously didn’t do any reconnaissance because we’re running a Linux server.” And so, it didn’t work, right?

Kip Boyle:
Right. Yeah. They were just spraying and praying.

Jason Dion:
Right. It was spray and pray. And we saw three different attacks over the last 24 hours that were all spray and pray. There was one for the Ghost attack, which was popular back in 2017. There’s still bots out there trying that on every new server. There’s people trying to get an RDP shell on every new server out there and they literally just go and touch every server and say, “Will this work? Nope. Okay, I’ll go to the next one.” Right?

And that’s where you want to make sure you’re hardening your systems… And you’ll see that in your logs, that people are trying to attack your systems all day every day, but most of the time, you don’t even see it because it’s just happening in the background. And if it wasn’t successful, it just didn’t even get logged and it just moves on.

Kip Boyle:
Right. And it’s automated, so it costs them nothing to attack you 24/7 until they get a hit. These are great examples, Jason, of the practical value of Compromise Recording as we experience it today. Some of the other things that you’re going to want to be looking for is impossible travel. So if you’ve got somebody who typically logs in from home because they’re doing remote work, and then shortly after they log in, you have another login to that account from 8,000 miles away.

Okay, well, that doesn’t seem reasonable. Right? Even though that person could be using a VPN to make that happen, it’s unlikely because they’re already logged in. So it’s like why would they double log in once locally and once for a VPN? So you want to get indicators of compromised stuff that is highly likely to be an attack. Because like you were saying, if you’re just sitting there just running through logs and you’re like, “Yes, no, yes, no, yes, no,” I’d rather just look at a list of highly suspect issues like impossible travel, and-

Jason Dion:
And I think that’s where we’re going to see more of the AI and ML being used, right? I know for… Since 2012, 2015, we’ve been using ML a lot, at least in the government side, to help us with that curation of logs and saying, “This is a known good, this is a known bad.” And then all this middle stuff, this yellow stuff, this suspicious stuff, that’s where I’m going to spend my analyst time to actually look through and put the human mind behind.

Kip Boyle:
Right.

Jason Dion:
And now with AI, it even has the ability to think about it in a different way where machine learning was just very much based on signatures and behavior and categories. With AI, it really can almost think for itself and make logical assumptions on what’s happening and did it make sense that Jason logged in from Orlando and then five minutes later, he logged in from Norway? Well-

Kip Boyle:
Yeah.

Jason Dion:
Yeah, actually it did because a couple weeks ago, I was in Norway and I was using a VPN to log into… Through my property in Orlando, before I [inaudible] the internet so that my traffic was protected. But then when my VPN went down, then it saw that I was in Norway and it was like, “Ah, okay, well, there’s the issue.”

Kip Boyle:
Yeah, so sometimes it makes sense and it’s a false positive, right? But false positives of false negatives are an enormous issue in all of this, right?

Jason Dion:
Oh, yeah.

Kip Boyle:
And it’s one of the things that makes the whole idea very fatiguing is, how do I know it’s a false positive? Well, I’m going to spend time checking it out and how do I know that it’s a false negative? Well, I don’t because I never even saw it, but something bad happened anyway. And so yeah, I’m looking forward to the machine learning to take this over for me because it is just so dang tedious and error-prone.

Jason Dion:
Yeah, and you know, what you just said, it rings so true. I’ve had that discussion so many times with senior executives, especially after an instant response. Like, “Well, we know we’ve been had, we see it in the logs.” I’m like, “Yes, we know it’s malicious. We went in, we cleaned it up, and we think we got it all.” And they go, “What do you mean you think you got it all?”

And I’m like, “Well, you never know 100% for sure, unless I remove all your systems and put in all brand new systems.” We will never know if the person is still hiding somewhere because you can detect that they’re there, but you can’t detect that they’re not there. They just may be really good at hiding and we see that a lot.

Kip Boyle:
And there’s nothing in the world quite so fun as your boss saying, “Not good enough, Boyle. I want them out.” And then you say, “Great. Just sign here and I’ll buy all new gear and then I can guarantee they’re not here.” “Whoa, whoa, whoa, whoa, whoa. Slow down, Speedy.”

Jason Dion:
Yeah, because even if you wipe a machine, right? If you have a Windows desktop and it’s been infected, yes, I can go and take out the hard drive and replace the hard drive or a solid state device, put on the operating system, get you back on the network, blah, blah, blah, blah, blah. But there are things that can affect your firmware and your BIOS and your UEFI, and all the other things inside the computer that don’t get changed out. And the only way to change that out is to replace the motherboard, in which case you might as well just buy a new computer. Right?

Kip Boyle:
Right. Yeah, absolutely. They’re not called advanced persistent threats for nothing.

Jason Dion:
Yes.

Kip Boyle:
Absolutely. Okay, well, anything else in Compromise Recording? I want to say one more thing, which is, in the beginning of this episode, I said that it’s great to have security and architecture design principles because they don’t change very often. And one of the things that you can do is you can use them to evaluate new products, right?

So if you’re evaluating a new product and you’re asking yourself, “Okay, well, let me just pass this new product through these 10 architecture design principles,” one of the things you’re going to ask yourself is, how easy is it going to be for us to do a Compromise Recording with this new product?”

And you may find out that, “Oh, my God, it’s impossible,” or we can do it, but “Oh, my gosh. We’re going to have to do so much custom coding and so on and so on and so on.” And so it’s just a great way to help you get oriented into what you’re about to get yourself into.

Jason Dion:
Yeah, definitely. And this is one of those critical things, right? If you’re not logging it, you can’t go back and recreate the logs after. And that’s one of the things we found out yesterday, right, is we weren’t logging it. We just never bothered to… We didn’t think it was something we were going to need is, why would we need to know what people were prompting and what people were getting back from the AI bot, right? But by recording that, we can now see if there’s an issue.

So you can do a lot of logging, but remember, when it comes to logging, all those logs have to be stored someplace. And so, if I’m logging every single action across a network that has a million hosts, which is, the last network I ran had a million endpoints across six continents, that is a lot of data. And the amount of money we spent on SIEM and using something like Splunk to be able to go through all that data, was immense.

And so, tuning your logging and how much or how little you’re going to log, is really important because while we want to say log everything, if you log everything, you’re going to be spending hundreds of thousands of dollars a month in server space to keep all that data.

And then the other question with logs is, how long do you keep it, right? Do you keep it for a day? Do you keep it for a week? Do you keep it for a year Longer is better, but it’s all more money, right? Because the more you have and the longer you keep it, the more money you’re going to spend because while hard drive space is cheap and server cloud space is cheap, it’s not free.

Kip Boyle:
Right. Not free.

Jason Dion:
And so that will add up to your bill pretty quickly. Anyway, so that’s my thoughts on it. That being said, I want to thank everybody for joining us for yet another episode of Your Cyber Path. We will see you next time on the next episode as we talk all about internships and things of that nature. And so we would love for you to leave us a review over on your favorite podcast streaming app, whether that’s Apple Podcast, Google Podcast, Spotify, Stitcher, whatever.

Leave us a review. Let us know what you think of the show. And if you ever have any questions for us, you can always reach us at yourcyberpath.com and clicking on the Ask Us question, put your question in there, and you’ll get a response back from me and Kip. That being said, we will see you next time on Your Cyber Path. Thanks.

Kip Boyle:
All right, thanks, Jason. See you everybody.