Common Attack Pattern Enumeration and Classification

CAPEC User Summit Transcript - “Case Studies of Industry to Academics: CAPEC's Role in Threat Management at SJU”

Suzanna Schmeelk, St. John's University

Session 2 - Case Studies of Industry to Academics: CAPEC's Role in Threat Management at SJU | View all Summit transcripts

(00:00) Speaker: Rich Piazza (Summit Host)

We're now going to session two and this is something I'm looking forward to. It's using CAPEC in education and we have two University professors who are going to tell us how they use it in their classes or in the curriculum at their school. So first, I'm going to hand it over to Suzanna Schmeelk.

(00:01) Speaker: Suzanna Schmeelk

Thank you so much Rich. I really appreciate it. This is a wonderful gathering here today and thank you to our audience. Some of the things I'm going to reiterate are what Fil and Max already said today and we see it in industry. I worked in industry, many different places, so everything I've seen in industry I've brought back into the classroom.

It's always great to see it early in the classroom. I called the classroom sort of at sandbox and then once students go out into industry that's a very sharp learning curve, so the earlier everybody has these experience the better. In this talk we're going to look at case studies in four different arenas, two in the Master's Program, and two in the undergraduates of MITRE’s CAPEC role in threat management specifically here at Saint John’s University out here in Queens, New York City.

Please the next slide.

(00:42) Speaker: Suzanna Schmeelk

Here is a link to our Master’s degree program. We have a Masters in science in Cyber and Information Security. The full curriculum is here on the link. Everything here is organic -- so we keep improving them. We have improvement processes and things.

These are our current list. You can see the different tracks - cyber security, data science and analytics, and IT are some of the specific focus areas inside of our Master’s degree.

Next slide, please.

(01:20) Speaker: Suzanna Schmeelk

I'm going to look at both the Masters of Science in Cyber and Information Security, specifically our pen testing and ethical hacking course, which is CYB 715 as well as our principles of secure scripting and cryptography, which is CYB 625 and two Bachelor classes. We're going to call out two bachelors classes, secure software development, which is 1035, and then our network security courses 1012 and 1011.

Next slide please.

And again, we're going to start off with the pen testing which we'll hear we have very similar points as Fil, Max and Nav mentioned earlier.

Please, next slide.

(02:05) Speaker: Suzanna Schmeelk

We really focus on pen testing from what we've seen in industry and a lot of times, as we heard earlier, a pen test reports turn into basically risk assessments.

Risk assessments, there are actually different Federal, state, city--specifically federal laws that require risk assessments. The laws don't specifically, as we are hearing, layout the exact organization of risk assessments--for good and for bad. The good part is every organization can figure out what threats and what are the pain points for their organization, but the bad side is that you can have two organizations across the street, in the exact same sector, and the pen test reports and the way we describe risk can be entirely different. In this bad case, how do you even pick up the phone and have a conversation if risk is not standardized at some level.

I put in a screenshot here of the NIST special publication 800-30 guide to conducting risk assessments and again they don't really tell you how to take this visualization and turn it into a two-dimensional report. As we heard, a nice repository is up on GitHub and every organization probably has developed something internally for themselves.

And, this visualization again, you look for your threat source, you figure out what your thread event is, potentially the vulnerabilities and then you have your impact --you know what're you actually impacting if this thing goes down and how severe is the impact. And, then out comes the risk of each of these events, and then your overall risk, which is again another debate. If you have three highs risk events and a hundred lows where does that fit in terms of risk? Or, if you have a thousand lows risk findings, is that equivalent to a high finding? There is a lot of different debates that are internal to a lot of organizations. There's a couple of different risk models--we have the FAIR Institute, we have the NIST model, as well. One of the papers I just published a few years ago was, again, reiterating what Fil and Max were saying earlier.

Specifically, I think it depends too if you have internal pen testers versus external pen testers but one thing I was seeing in industry is that no two pen testers would write-up findings the same way …. similar to what everybody else today has been saying and so now if you, for legal reasons, have to say how many of this, are in the organization and how many of that, you have in the organization, now suddenly you have a NLP (Natural Language Processing) problem trying to figure out the way they've written up things can be the same problem, just written differently and now you're down into a whole wonderful research problem for NLP … if you choose to use NLP.

I gave an example here --- especially for healthcare side because you're under HIPAA which requires risk assessments…

Next slide, please.

(05:25) Speaker: Suzanna Schmeelk

…as we can see again, here's the NIST 800-30 guide for conducting risk assessments, they clearly have mentioned examples of threat. They provide representative examples of adverse adversarial and non-adversarial threat events at a level of detail that can be used for risk assessments. They cite CAPEC as a great source for actually going into depth into these different threat events.

Next slide, please.

As we heard earlier, one of the things I have the students do is build risk assessment reports as we pen test so that's one of the things they do is …. specifically right now we are following CompTIA models so that students have certifications when they finish different programs undergraduate and graduate, so we've structured it around Ethical Hacker exam as well as the CompTIA Pentest+. But as they work on these different areas of pen testing so that they build a report. It's up to them again. I'm going to have to direct them now out to the GitHub. But I want them to get familiar with how to write up these risks that they find through the pen testing and one of the things we can see is bringing in the CAPEC as well as the NIST controls again for these reasons of having a larger picture of all of the potential threats that are out there and understanding the bigger picture because as we said you can have all of the cameras on one door and leave the side door wide open.

So that conscious effort to make sure that we've connected to the larger system is essential as we get to be more and more online down the line.

Next slide, please.

(07:18) Speaker: Suzanna Schmeelk

I'd like to also talk about our secure scripting course. This is a different graduate level course specifically towards scripting. It does talk about cryptography. It's again in the Master’s Degree program.

Next slide, please.

…so one of the key deliverables at the very end of the course is a research paper applying secure scripting and it goes back to some of my earlier work I did with Professor Aho and Professor Yang back in 2015, when we were starting to look at static analysis techniques and trying to come up with sort of a meta solution to all these problems and so one of the things we did was map our findings to CAPEC so that we were looking at malware and we could see major gaps so a lot of the research at the time didn't put it into the larger framework.

I have the students do that again in the secure scripting course. I'll explain some of the different frameworks we look at, so again, it's up to them. It's their research; I mentor them … but some of the things they can look at is CVE, NVD, CAPEC, NIST 800-53 and the wonderful NIST bugs framework (BF), which was developed a few years ago, that really gets at the essence of our bugs.

Next slide, please.

(08:30) Speaker: Suzanna Schmeelk

For the undergraduate course secure software development…again we have students work on building their own software and every week we work on a new lab to secure it. A recent paper came out describing the whole process so feel free to go back and look at that. One of the things I'm asking students to do going forward is to connect all of these findings and vulnerabilities to these larger frameworks at the end in their final report

Next slide, please.

Another thing, I know it's not CAPEC, it's the CVE (it's MITRE) but we do have students become familiar with the CVE. I put it example appear for the recent log4J problems. In our course, we look at Java Python, PowerShell, in our course and get the feel of these different languages and how they interrelate a little bit.

Next slide, please.

(09:19) Speaker: Suzanna Schmeelk

And then our final courses are our network security courses which is CSS 1011 and 1012.

Please next slide.

They have a lot of labs all semester. One of the things I asked them to do on the final project is building the larger pictures for a sector that they want to go into.

And one of the things we are having them do going forward is connecting the labs and the findings from the labs and telling their story and interconnecting them to this larger picture that these wonderful ontologies provide for us.

Next slide, please.

Here's our references of all of the amazing work that we build on so thank you.

(10:03) Speaker: Rich Piazza

Thank you very much Suzanna.