A conversation on cybersecurity in the built environment, hacking a BMS, and what you can do to stop it, with Fred Gordy from Intelligent Buildings.

Gridium: Hello and thanks for tuning in on this conversation of cybersecurity in the built environment and the anatomy of a BMS hack with Fred Gordy from Intelligent Buildings. This is Millen with Gridium.

Fred: Yes, thank you Millen. First let me,  thank you guys for allowing me the opportunity to speak on a subject that I’m very passionate about and I appreciate you guys.

Before we get started I wanna’ kinda’ tell you a little bit about also the company that I’m working with which is Intelligent Buildings and umm… now, at its core, Intelligent Buildings is a uh… a company that works for the customer and the reason the customers come to them is, if you can imagine, there are so many disparate systems out there:  parking  control systems; it’s not just about control systems, it’s about the entire smart building strategy.

Gridium: Mmhmm.

Fred: We can step in the gap, be the advocate for the customer, take a disparate system, pick and choose and help them, guide them through the process of putting together the best smart building package they can come up with. And as part of that, it would be remiss that cybersecurity was not part of its core foundational piece and everything that Intelligent Buildings does is built on that layer of cybersecurity.

I came from IT in 2000 and I moved over to the building control space,  to the smart building world in 2000 and I remember getting a call from this company  at the time and they explained what it was that they did and I still didn’t get it. (Laughs)

Gridium: Mmhmm.

Fred: So, I went to the interview and  they started talkin’ about web-based control for buildings and  that type of thing. And I did something at an interview you probably shouldn’t do: I looked at him and I said, “Are you sure you’ve got the right guy?”

Gridium: Mmm.

Fred: And  the  the gentlemen that was interviewing me, he said, “Yes, we do. And the reason is, is because we’re… we know that these systems are moving more and more to a global control kind of strategy and we need people that have IT background to kind of help us move into that world.” And  thus began the… the transition.  that company introduced me to umm… enterprise-level building control systems with people like AT&T and Macy’s. And umm… the… the security through obscurity thing kinda’… basically I drank the Kool-Aid, let’s just say because I went from a policy-control world to the Wild, Wild West. And I guess it was around about 2010 that I, you know, looking at the portfolio of who we were integrating with and what all could possibly be done. I had to  kinda’ shake off the Kool-Aid that I drank and start saying, you know, we really need to start taking a look at this stuff and trying to figure out a way to protect it because in and of itself, it’s not the design… it’s not designed to protect itself…

Gridium: Sure.

Fred: …and so I started my journey of cybersecurity for control systems and I found that, while there were entities like ICS, Scada and that kinda’ thing…

Gridium: Yeah, Fred. Thanks for telling us a little bit about your transition from the IT world to the OT world and… and let’s jump into what a BMS hack might look like. So, I think umm… I think what you’ve got showing here on this slide is this question of  is your building’s data searchable? And… and can you tell us a little bit about what kind of data is  searchable and where?

Fred: Sure.  a few years ago there was a gentleman by the name of John Matherly who  created Shodan and Shodan in and of itself is not a hacker tool by  its definition.

Gridium: Okay.

Fred: However, that’s what it’s become and what it does is umm… it’s running a series of queries 24/7, 365 and it’s looking for exposed devices and it’s looking specifically for OT devices. It’s not, you know… it’s just… it’s not lookin’ just for servers and that kinda’ thing. It’s lookin’ for things like control systems, cameras, so on and so forth. It catalogues those and it’s searchable to the point that you can not only put in the  the name of a, say of a particular manufacturer, but you can also find out where a geo loc is, which is the IP location of where it’s originating from. So, you… you get a… some background of what… what’s actually in the building.  a little bit about… you know, the manufacturer and there’s a lot of relative information there that is kind of tasty to the bad guy. But again, he didn’t design this for the purpose of bad. It was umm… you know, his… his attempt to, you know, kind of bring awareness to this and as a result there’s a  Censys is another indexer or OT search engine.  this one was created by a university. I… I’m actively… I actively use both of these. Now the third one is  a sketchy one. It’s called ZoomEye and as far as I understand it, its… its origin is in China and  but it’s, you know… it’s available for anybody to use.

So…once you have access to these… these search engines, like I said, you can put in a manufacturer name, you can put in a, even something like a chiller and it’ll bring back exposed chillers. Now, let me qualify what I mean by exposed: what this means is there is a public IP address that this… any one of these systems is actually registered a response back from. What that says is, it’s not behind the firewall, which is a huge no no, but…  to give you some stats and stuff, if you were to do what I call a “call all” for Censys and it would give you back somewhere in the neighborhood of 300 million connected devices in the world.

Gridium: Wow.

Fred: In the United States… yeah! In the United States, there’s over 6 million, so we’ve got a lot of work to do.

Gridium: Yeah.

Fred: (Laughs) so I mean, like  with a… with Shodan you just sign up for a free account. Now they… they… they do charge for little extra things. Censys is all free. ZoomEy is all free and anybody and everybody can use ‘em.

Gridium: In fact, I did sign up for Shodan myself and I was able to pull up a chiller at a private University in Southern California. It was quite remarkable.

Fred: It… it can be addicting. (Laughs) I mean, it really can… I mean it…

Gridium: Yeah.

Fred: …I… and… and let me ask you a question. Is… how long did it take you to do that?

Gridium: Seven minutes, maybe?

Fred: So, that gives you… that… in seven minutes time, you found a… a chiller at a university. Now, we don’t know what it was attached to and… I mean…

Gridium: Sure.

Fred: …we could dig and find out, but the reality is, that’s a chiller. A chiller is a large piece of equipment. Even if it’s nothing more than just to damage it physically, where it would’ve cost the university a lot of money to fix – and you did that in seven minutes. You connected within seven minutes.

Gridium: Yep.

Fred: Now, what you see here is just a basic layout  of a control network and the red circles indicate that somebody has put a public IP address into each one of these devices, meaning that Shodan, Censys, or ZoomEy could find it. Umm… now, over to the right there’s the generator. Umm… usually if you find a generator, it… it’s attached to something critical. A hospital, a data center… so on and so forth.

Gridium: Mmhmm.

Fred: So, just by the nature of what it is, if I’m a bad guy and I find generators, you know… that’s… that’s a… that’s a great find. That’s an easy find. Now, the interesting thing to me is if you look down the… the  the tree, so to speak, if you don’t know what a CRAC unit is, it’s a computer room air conditioning unit and these are usually found inside a data center. And their whole, sole purpose is to keep the server floor cool.

Gridium: Mmhmm.

Fred: Well, as a bad guy, if I can get to those and I can raise the set points on those dramatically enough, umm… the servers are not going to be able to keep up and what will happen is servers will begin to start trippin’ offline. That can interrupt ecommerce, you know? And same thing  you got, down through that little tree, you have a… a UTS, a PDU. A UPS, you know, most of us know what those are: the little ones sit on the desk. These are big guys that sit in data centers and a PDU is just a power distribution unit. You can find all of these things through these  search engines and like I say, just based on the equipment type itself it’ll tell you what it’s criticality level is because it’s… you’re not putting a UPS on something you don’t care about; you’re putting it on something you do care about.

Gridium: Yeah. And with this information exposed, you know… how does a… a hacker or an attacker take it  one level down?

Fred: What I’m showing is a schematic of a BACnet… of a BACnet attack and this is something that I’ve kinda’ really started harpin’ on lately. Is umm… BACnet by its nature, when it was originally developed, was to be an open protocol, meaning that anybody could talk to it. And that was good in our control world, because then we were getting away from propriety systems.
Well, there is no username or password required to get into a BACnet… BACnet network.

Gridium: Mmhmm.

Fred: Before, you know… say, two years ago, before anybody really started thinking “bad guy” way about getting a control network, you had to crack a password and the reality is the control system passwords are not that hard to crack. So… but it still, it slows you down. But once the umm… once you got out there is the big BBMD which is the BACnet Broadcast Management Device and its whole  purpose in life is to help controllers from one subnet from another subnet. And so, if I can find one of these BBMD through Shodan or Censys or whatever, I now can traverse down through the connect and BACnet network. So, a bad guy finds on Shodan or Censys and he looks up “BBMD”; I mean, it’s that simple. You could type BBMD.

Gridium: Mmhmm.

Fred: If it finds it, it’ll tell you in the description that it’s a router. And if it tells you it’s a router, that means that there’s attached things to it, so… what the bad guy would do here, all… if you look on the left side, you have basically human control: there’s comfort, on/off. On the right side might be representative of the data center. Well, if I can get to the left BBMD, which is shown in the red circle and then I use the tool that we’re fix… that I’m fixin’ to show you in just a second, you can then scan the network and find the CRACs, the UPSs and the gensets and then you can do real damage. And now, let’s flip to the next screen.

Okay, so… so now, once you’ve got that baby identified, there is free tools out there for anybody to download and this is one that I’ve found on SourceForge, and the scary thing about this one is, I’ve seen other free tools and they’ll let you scan the network and you can do a basic functionality; but with this particular tool, I can do everything. I mean it’s… it’s to the point that I can even turn controllers into breaks. And what I mean by that is, I can… I can pop a program in there that’ll just basically wreck what’s there and they may be able to recover it, but how long will it take?

Lookin’ at this screen here, what you see is, this is a… what I did was I found one BBMD and uh… this one is not the big list like I got, but one BBMD and when I scan the network I can… I’ve found over 200-somethin’ BACnet IP devices and then underneath them are BACnet MSTP or serially-connected devices.

Gridium: Let me see if I’ve got this…

Fred: So it’s…

Gridium: …Fred. Does this allow you to make changes?

Fred: Yes, it does. In fact, to the point, if you’re lookin’ over there on the   right side of that screen, it shows the editable properties. Even if the programmer or operator has set this thing to read only, with this tool I can go in and make it writable. And I can change set points, I can change  alarm parameters where, you know, kinda’… you and I talked one time about the Stuxnet,

Gridium: Mmhmm.

Fred: … Attack. Well the Stuxnet Attack masked itself so nobody would see what was going on. With this tool, I can do the same thing. I can turn your alarms off before I do anything and you won’t receive any alarms while I’m in there making changes…

Fred: …I can literally turn devices off. I can… you can even like a, using the central plant of the chiller, you can go in with this… this particular software and you can lock out the operator. In other words, I can mainly override it to where he can’t get into it without actually going and downloading the prog… or connecting directly to the device and having to jump through several hoops to take back control. But by the time he… if they could do that, I’ve done… you know, I’ve wrecked the place…

Gridium: Yeah.

Fred: …basically. So, and you can set schedules. re…  the trend logs that have been there. But again, the main point that I’m making is no… no username or pass… password was required to get to this level, because of these… because of BACnet open.

Gridium: Wow. So, there’s more online than we know. You’ve made that point quite well, Fred. That’s crystal clear, and when I’d first caught on to a little bit of this, I was quite surprised umm… Fred, and as I dug in, I realized there was a… much more to know here than first meets the eye. Umm… you’ve made a point in some of your material that I’ve read that there are people involved in these BMS attacks  you know, as well. And can you talk a little bit about the role that… that people have in… in the building controls and how attacks can take place?

Fred: In first off, cybersecurity, regardless if its IT or OT, umm… all the countermeasures in the world are never going to circumvent an operator’s intellect. Operator neglect is going to cert… is going to override any kind of security measures that you have in place. So, it’s really important that everybody consider themself a part of IT, but a part of the cybersecurity team, and that goes from, you know, the guys that are running the building to the… the security guards up front, anybody working in admin. And if I may interject, umm… using the term and I’m (Laughs) not tryin’ to pick on admin, but umm… there’s been a lot of talk about a particular breeching that everybody knows about and it’s called the… about Target’s breech.

Gridium: Mmhmm.

Fred: And it was originally reported that attack came through the HVAC system. That wasn’t exactly true because what happened was there’s an HVAC contr… contractor, not unlike the one I used to work for, but they were up in the northeast. Mechanical contractors, it’s not unusual for them to have a portal into the people that they serve in order for the admins to enter a bill. So, in other words, a service guy goes on site and at the Target store he does a repair, he comes back and he hands the work order into the people in… in admin and they open it… open the Target portal and went in and entered the  the billing information.

Gridium: Mmhmm.

Fred: Well, what happened here was that this particular admin got phished. And phished: P-H-I-S-I… I mean, excuse me. (Laughs) P-H-I-S-H means e-mails are sent out and hopefully you click something. It’s like there’s a hook inside the e-mail, and if you click it, it’s gonna’ do something.

Gridium: Yeah.

Fred: And in this particular case, it… it umm… launched Keyloggers onto the  system and the admins… and this is not a… not an unusual practice either, all of the admins had the same username and pass… password for Target. There was only one username and a password for several admin. So, it could’ve been any one of them, but once the bad guys found out, you know, or got in to their system, then they saw, “Well, wait a minute. We’re on the Target network. We see that there’s a Target network.” They got on the Target network and then they… the network they were on, the POS, the point of sale devices were not segmented from this DRP system and the bad guys were able to just run rampant across all those credit card machines inside the system.

Gridium: Yeah.

Fred: So, no amount (Laughs) of  firewalls or anything could catch that. It comes down to people. We all have to do our jobs and then, those days where we thought, “Oh, who cares?” or, you know, that kind of thing… they’re pretty much over. There’s… the example that I have here is something new. They had a control system sittin’ there and there’s some  ransomware called Server3. And you…

Gridium: And can you explain what ransomware is?

Fred: Sure, absolutely. Ransomware is a really nasty thing because what it does is, when it gets launched onto your network, it encrypts everything. The data is still there, but what they’ve done is they’ve basically padlocked all of your umm… your data, all your functionality, and you have to pay them to get the keys to unlock it.

Gridium: Mmhmm.

Fred: And so, the thing is… the thing about ransomware, and this is where I’m always kinda’ a little hesitant when somebody tells me is, “Well, we got it fixed.” You never know what these guys leave behind. Even if you pay them and they come and… and they will, they’ll unlock your data… what did they leave? Nobody knows unless you really dig through the system. But anyway, so that’s what ransomware is.

Gridium: Okay.

Fred: It basically holds you hostage until you pay them. And a unique thing about ransomware  and I… I mean, I hate to get into too many little rabbit trails here, but…

Gridium: No, that’s alright.

Fred: …the little…

Gridium: This is interesting.

Fred: …cool. Well, the little emblem up there, you see the gold emblem: that’s Bitcoin. And if you’re not familiar with Bitcoin, umm… I’m not going to sit here and say I’m a Bitcoin expert.

Gridium: Sure.

Fred: But basically, it’s a way for people to pay for services and that kind of thing and it’s actually been incorporated into the…  the bad world, so to speak, because it’s hard to trace who actually you’re paying. And the little servers that you see up there, the little computers where those lines are squiggling through there: when the ransomware is released and if you were to say, “Okay, I’m going to pay you this…”  in this particular case, I can’t remember how many Bitcoin it was, but it was… it wasn’t a… it wasn’t a huge but it wasn’t… it was a… it hurt. Umm… is if you pay… if you agree to pay the Bitcoin, or pay Bitcoin, you… you have to open an account and do all these steps and then, through what they call a Tor network, it basically is hopping from server to server to server, and while the bad guy might be sitting in Atlanta, Georgia, it looks like he’s in Paris, France. So, finding him is going to be almost next to impossible. So, in Cerber3 ransomware, they’re gonna’ lock your devices down, and then the payment method is so obfuscated, all the connectivity spaces…

Gridium: Mmhmm.

Fred: …there’s no way to ever really find out who it is. In this…

Gridium: And now, are we looking at an example of… of one of those cases that you know about?

Fred: Yes. Goin’ back to the… to the root of the problem here, and I hate to keep pickin’ on people, but I… that’s where it was… is, not only did somebody open an e-mail that they shouldn’t have opened. And there’s telltale signs and I can tell you about those: there’s telltale signs that’ll say, “Hey, there’s something not right with this e-mail.” That not only did they open it, but the machine that you see where the… with the guy with the lock to the side of him. He’s an authorized user, right?

Gridium: Mmhmm.

Fred: Well, these control system frontends, I’ve seen it time and time and time again, where they’re sitting on a building engineer’s desk, out in the public, anybody, and they’re using this thing to look at Facebook, to look at their… to check their mail, whatever you think. And this is the thing that’s controlling the building, but yet, people are using it for whatever they want to because… and they… there’s no restricted access. Well, so what happened was this person opened the e-mail on this control system, front end. The bad guys didn’t… they didn’t know what it was, all they cared about was getting their money. But to the e… but to the people that it happened to, it blinded them to their control system. It made everything inside that machine was now encrypted, but Server3 is exceptionally nasty because it doesn’t encrypt… encrypt folders, it can encrypt individual files with individual encryption keys. So, if you have a million files, that’s a million encryption keys and it would take billions of years to figure out (Laughs) what…

Gridium: (Laughs)

Fred: …the encryption…

Gridium: Wow.

Fred: …keys were…So there’s no way…

Gridium: Yeah.

Fred: Plus it… it did one other thing. It started dir… deleting shadow copies. And shadow copies are shop… copies that are, you know, recoverable copies, that you can pull… pull back into place. Now, this particular company,  they didn’t pay the ransomware, but it took them days to figure this out. And what they had to do is eventually wipe the machine clean and they found some backups that they had made months ago and they put it in place and everything’s back up and running, but all that data’s gone. It’s never comin’ back.

Gridium: Yeah.

Fred: Now, there’s one aspect didn’t happen to them, and I added this layer so people will be aware is, if you’ll notice the red lines that are going over to the little data cans in the server rack… well, what could happen or could’ve happened in this case, is if this machine had access rights to any servers on the corporate network, it could’ve propo… Server3 could’ve propagated itself not only on the control system, but through the rest of the network. And then now what you’ve got is that every machine or everybody’s… every  server that you have in your system is now encrypted. In that case, you might have to pay the ransom wherever it goes. You know, by the time… I mean, who knows how long you would be down.

Gridium: Yeah.

Fred: But that’s a… that’s a distinct possibility. And I… now all of this results… these results that I’ve spoke of come from people, so… we need to look at somethin’ real quick, if you can indulge me… the first off, control system is  like the goat in umm… Jurassic Park.

Gridium: (Laughs)

Fred: The goat was unsafe, right? I mean, it can’t defend itself. There’s just no way. And the control system, when I’ve heard people kinda’ yelling back to the manufacturer and yelling back to this and that. The control system is not designed to protect itself, just like the goat is not. So, what you’ve got to do is you either got to get the goat out of the pen, or you’ve got to get the T Rex out of the pen, but you’ve gotta’ separate them somehow.

Gridium: Yeah.

Fred: And you have to… and in order to do that, you’ve gotta’ get that control system inside the IT bubble. Once it’s inside the IT bubble, you’re gonna’ have to change the way you do things. And it’s typically…  it’s typical for a control systems vendor to have a backdoor into your system and it… it was done not for anything illegitimate means. It was done, just like with the mechanical contracts that I worked with, we had a 400 service guys on the road and so, any one of them could go to a building and fix it. Well, if umm… so, what that meant is that every system that this company serviced had the same username and password in there for the  for their service guys to go in and work on it. Well, that’s convenient and I get that, but that can’t exist anymore because all it takes is one of those guys to get mad, get fired, and hand that username and password off to somebody.

The other thing is,  because there’s been such a detachment from IT and OT, what has happened is over the years is that the facility guys have contracted with the integrator, “Hey, build me a network, and while you’re at it, build me remote access.” And so, the person remote… that owns all the remote access is not the building owner, but the integrator. And again, that’s bad IT practices. No IT comp… no IT group is ever going to allow that. But we’ve created that all for the sake of convenience.

Gridium: Yeah, Fred you know, that’s got me thinking, and you and I were discussing before launching this conversation, you know, the data is searchable and it’s quite scary to think about what’s so freely available online. Then there are programs which are also freely available online which you can use to… to take over the devices and then it’s not that hard to go from taking over the device to gaining access to the network, which is what we’re looking… what we’re looking at here. And so, one of the questions I had for you early on was, you know, how did we get here?

Fred: Well, one of the things  that we need to establish right off the bat is how do you go back to your customer and say, “Hey, by the way, we have set you up and made you pretty vulnerable.” It’s a hard… that’s a hard conversation to have. But the realization is when OT started out… I mean, you’re looking at these two triangles here: the left side is IT, the right side is OT. And these are principles that have been developed by ISC CERT, but they are foundationally to me the easiest way to explain things.

In the IT realm, the first and foremost thing that’s important is information, confidentiality. And that means that, you know, we’re gonna’ protect it first and we’re gonna’ worry about the other function second and third. The second function is the integrity of the data. The data has got to be right, but again, it’s second to the confidentiality. I’m gonna’ say no to you first, before I give you access. And then I’m gonna’ figure out do you really need access. Then once you do, I’m going to, as an IT manager, make sure that the data is correct and up-to-date and everything available are there. And then availability. That’s that piece where I finally say as an IT guy, “Okay, you have the rights to see this, but nobody else does.”

In OT, availability is the highest hallmark that we have to go for. And why is that? That is because control systems are… are machine-to-machine communication and decisions inside programs are being made in milliseconds. And so therefore, it’s… it’s paramount that nothing impede that… that communication between devices and between the devices that they patrol. So, availability is number one. Number two is integrity because then the integrity of the data and the integrity of the system, gives the operators real-time vision into their systems to see if they… they’ve got a problem that’s developing. And they can look at archival data, they look at real-time data and they say, “Well, okay. I’m seeing something that’s trending bad and so I’m gonna’ go and take care of her.” And confidentiality is… in… in this particular triangle, I’m showin’ it like ISC CERT does… sometimes when I do talk, I pull confidentiality over to the side as… as just broken away from the piece, because that… it is so far removed from the… the… the principles and the thought process behind puttin’ together control system strategy and network.

I kinda’ touched on a little bit of history. That’s the kind of the high level or the high-end view that I see as bird’s view. So, let’s look real quick back in at the history of control systems.

Gridium: Sure.

Fred: Control systems started out as pneumatic systems and then electronic  controls, DVC came into existence and that was just a series of ands, ors, and logic and that kinda’ thing that just happened to make things happen based on this condition. If this happens, then this happens, then do this.

Gridium: Mmhmm.

Fred: Well then, now there were the microprocessor-based control. What happened was, the integrator now had to learn IT. He had to learn how to run CAT5 cable, not just serial cable. He had to learn how to set up  Internet IP addresses, subnets, domains, all of that. But, he only needed to do as much as he needed to do to allow the devices to talk to each other and the humans to talk to the devices…

Gridium: Yup.

Fred: …and that’s where it ended. And again, and this is not picking on a building operator, but that’s not their forte either. What they are is they’re highly gifted individuals that know how to do things with a 300-tonne chiller that I’ll never be able to do because I don’t understand the deep mechanics of it.

Gridium: Sure.

Fred: But, so when they got these systems that were supposed to make their life’s easier, in some cases it (Laughs)… it made it a little more complicated. So, the integrator stepped in and said, “Let me help you, because we’re putting these in… we’re gonna’ add your users…” and the building engineers would say, “Hey, look. I just need one user for all of us to use.” “Okay.” Boom, boom, boom. “Do it.” The integrator, like I said earlier, had… may have a team of guys that come out there and you can imagine if you’re an integrator and you’ve got a thousand systems out there and you’ve got 50 employees that all they do is they service these thousands of systems. And if each one of ‘em has a unique username and a password, just the act… actual headache of being able to keep up all of that… all of that information to give, is overwhelming. And so, that’s how we got to the point where we had single users for both engineering and for the… the vendor. And then the next step was, is once the guys realized, “Hey, if I’m at a computer room on the 50th floor, I can see my control system. It would be really nice if on the weekend I could get to that control system if an alarm happened so I didn’t have to drive an hour to get to the s… to the site, because it makes sense…”

Gridium: Right.

Fred: “…because I could get the… I could do something before somethin’ blew up.”

Gridium: Yup.

Fred: So, goin’ back to the integrator… the integrator’s not an IT firm, that’s not their forte. They’re integration specialists and what they did is they… we’ve got Best Buy routers, and I’m not pickin’ on Best Buy, but I’m just sayin’… there’s Best Buy routers that are sitting out there will little home VPN or the next worse thing… or the  much more worse thing is you can get a public IP from your Internet Service Provider and you can stick that in the front end of your machine and there’s nothing between it and the world-wide web. All I’ve got to do is put in that public IP and get challenged for that single username and password that we all know, and that gets us in the door.

Gridium: Mmhmm.

Fred: So that’s how we got here.

Gridium: Fred, is this gap bridgeable?

Fred: Yes, it is, but not without some discomfort. And now, I say that… I don’t wanna’ make it sound like it’s a insurmountable task. There are some basic things that everybody could do. First off is NIST set up a framework, and this is publicly available information. I will say it’s a dry read and it’s a lot of stuff there.  I’ve read all of ‘em and it’ll put you to sleep pretty quick. (Laughs)

Gridium: That’s the National Institute of Science and Technology, is that right?

Fred: That’s correct. But umm… they’ve done a good job of creating  generic Cybersecurity Framework. They’ve also created things that are specific to control system. Now, the control systems they’re talkin’ about are power-grid or gas and that kind of thing. And it… it fits our world about 75%, 80%. But anyway, so the very first thing you got to do is umm… I would say is take a look at this.

But these five principles right here, is the foundation of the NIST Cybersecurity Framework: Identify. What that means is: know what’s on your network because I can guarantee ya’, I’ve seen this time and time again, I call it network drift… ‘cause once an integrator has turned it over to you, there’s a drift of 20-30% meaning people have plugged up additional things that shouldn’t be there. So, you need to take stock of your inventory, stock… whatever you want to call it, of your control system at work. Find out everything that’s attached.

Protect. That means is if you have any exposure whether it be through a public IP or you have a… a consumer-grade VPN or you have a printer that’s been plugged up that’s accessible by other people or cameras. Whatever the case may be, you need to take those and get them inside the IT bubble.

Detect. Now this is the… this is the difficult one for most people. I will say this first and before you go to the tech: if you do step 1 and 2, to me it’s a lot like umm… putting the sign out in front of your yard in your neighborhood where somebody’s driving through and they’re lookin’… they say, “Well, there’s an alarm system on that house. There’s an alarm system on that… oh! Here’s one that’s empty or doesn’t have an alarm system.” So, if you do that, the… the very first two, if you do that, guess what? Then Censys Showdan, ZoomEy is not going to see you. And if they don’t see you, the bad guy’s probably not going to see you, okay?

Now moving on to the detect…

Gridium: Got it, yeah.

Fred: This… this is a umm… well, the reason I say it’s hard is because it’s gonna’… it’s gonna’ ‘cause OT and IT to begin to have conversations, which we need to have them anyway. We need to start those now. It’s… okay. I have this… this system and we’ve done all these steps and we’ve got it protected. We’ve identified everything, we’ve protected it. We’ve put in change management, meaning I can’t plug anything into my network unless I let somebody know.  and in detection piece of that, if you have this piece in part… in… in place, if somebody were to pull something off your network and it wasn’t… it hadn’t been authorized or whatever, the detection would pick that up and you’d be able to maintain  your network architecture 100%, but then you would also be able to detect things like visible security intrusions, which we didn’t talk about. But, basically that just means there’s a generator in the parking lot and I unplugged from the generator and I plugged my laptop in. Boom! I’m now am in the network. So, there are… I mean, we have some… we work with some people that have detection software and that type of thing.

Gridium: Mmhmm.

Fred: There are more and more coming about, but it’s still… still got a little ways to go.
And then response, this is often the most… most, in the building control side has never really thought about, is  if an attack happened, let’s use Target for example. The company that I referred to…

Gridium: Mmhmm.

Fred: …they’re almost out of business now. It wasn’t their… I don’t wanna’ say it wasn’t their fault, but it really… more of the onus is on Target than it would’ve been on them. But their damage to their brand is insurmountable. And that’s the other thing I tell people, is what… whether the bad guys steal something from you or somebody sues you, at the core if your brand is no longer good, what does it matter ‘cause you don’t have customer… so you need to learn how to… how to respond internally and externally and you can’t put your head in the sand and pretend that it’ll go away. Umm… I’m sure you remember the  Blue Cross/Blue Shield or Anthem breach?

Gridium: That’s right.

Fred: Do you know, do you remember that one? It happened not long after Target, right? Who do they talk about the most now? Target. The reason they’re not talkin’ about Anthem as much is because within 24 hours the president of the company sent a e-mail out, letters out to every subscriber tellin’ them what they were gonna’ do. Nothin’ came out from the company that was the HVAC mechanical  service people from… for Target. And Target delayed their response. Home Depot had a breech. Which one do you hear about more still? Target. Home Depot responded. You’ve gotta’… you’ve gotta identify people inside your organization that are gonna’ come to the forefront and basically take charge of this situation and be the vehicle for public relations and that kinda’ thing. But then also, what do you do internally? Do you pull plugs off the wall? Do you shut things down? And then recover… I mean, once you’ve done all of these things, your recovery is making sure that you get all your data back in place. Making sure that your relationship with your customers are mended. And umm… then take a… a forensic look at what you’ve… what happened, how you can prevent it in the future and  move on from there. But I mean, to me, those five principles is what everybody needs to live by.

This is… this is a… this is not just a career for me, it’s a passion and there’s a lot that we’ve got to get done… and I know it can be scary  when you sit back and think of your budgetary concerns and that kind of thing. There is a lot of low hanging fruit that you can do that’s not gonna’ cost you a ton of money. Umm… but anyway, I just  I wanna’ appreciate and say thank you to you Millen, you guys for lettin’ me come on and talk about this subject.

Gridium: I appreciate you taking the time  to chat with us about BMS hacks and cybersecurity in the built environment. And so, thank you very much Fred.

Fred: Thank you.

About Millen Paschich

Millen has been thinking about the built environment since he was four, when he started with site walks on residential construction projects. He began his career at Cambridge Associates and has an MBA from UCLA. Talk to him about bicycling, buildings, businesses, and green chile burritos!

You may also be interested in...

EDF Webinar | Natural gas data, meet interval analytics

Bringing energy efficiency data analytics to new data sets, such as natural gas interval data, makes advanced building operations behavior more informed and more meaningful. This is simultaneous heating and cooling, new benchmarks, and leverage on new ENERGY STAR scores!

M&V of new motor technology

Basin Street Properties is dedicated to sustainable building operations, and its M&V analysis of SMC motor systems retrofit in seven RTUs shows an estimated 10% total energy savings from fan energy savings of 53%.

EBCE DR Pilot for PDP Buildings

Dan Lieberman, of East Bay Community Energy, discusses EBCE’s new Demand Response Pilot program for E-19 and E-20 PG&E customers participating in PDP. This limited-enrollment program will match PG&E’s PDP structure and incentives, while providing rate protection for participants.