Whether you perform “threat modeling”, conduct “design reviews” or engage in “risk assessments” for the purpose of identifying and hopefully correcting design flaws in an application’s design there is always a lingering question of completeness and accuracy. I’ll try not to derail the conversation and talk about what you do with the flaws (ie, how you rate them, prioritize them and a method for correction) you’ve found even though I really want to.
So, what about completeness and accuracy? Many organizations now go about performing some sort of activity prior to the construction of their widgets. Most probably think they’re doing a decent job. But how do we know? Is it the volume of flaws that are discovered? This is more like a warning light on an automobile than a measurement of the completeness of the efforts. I know, our buddies Saltzer and Schroeder have spoken about proving a negative requirement and I agree with them. I tend to think that this notion of completeness and accuracy are rolled up into a more well-known concept of code coverage. No, it’s not perfect, but it does a decent job here. Of course there is no “code” to cover in the sort of per-construction activities we perform. We deal with the raw materials that will eventually materialize into code. We don’t have the luxury of measuring the scope of our activities based on properties of the code (the number of lines, critical regions, etc). So how do we measure coverage? Do we rely upon methodological adherence? That seems dangerous. What about the quality of the design artifacts? Do they have use cases? The sort of inspection that we can perform seems to be proportional to the amount of time one has taken to articulate the design. But that only makes it more probable that we will be more complete and accurate. We can’t use that to measure our coverage either.
There are two aspects that contribute to quality code coverage ; identification of security code paths (I know there is no code yet) and depth of analysis. The first is the process whereby all untrusted points of input and output in the design are discovered and validated. In my world very few points are trusted, but there are levels of trust. The next is depth of analysis. It is not enough to go through a series of binary questions like “do you authenticate this communication channel” even if there is a well-defined policy governing authentication requirements. It isn’t just the coarse-grained patterns we’re speaking of here. It is the *design* of those patterns that must be evaluated. This is, I think, where many security efforts go awry. They boil down analysis and expertise into questions and answers. Questions do not achieve the depth of analysis criteria for code coverage. Without a doubt if a design doesn’t answer yes to these fundamental questions you’re at a hard stop, but for those that can answer yes you must go deeper.
So, are you confused? “Code coverage” in pre-construction security efforts must consider the methods to identify (and validate) inputs and outputs and the degree of analysis performed upon that data. How do we do this? I don’t know. What we can do is use these two categories (there may be more) as controls points for the consistency and/or reliability of our data. For example, if we’ve simply reviewed available documents to discover the project’s design elements there is probably a greater margin for errors and omissions. It follows then that our code coverage will not be as complete as it could have been. Yes, I know, it is only probabilistic. But that may be the best we can do. I’ll leave it up to everyone (all two of you) else to consider whether what I’m saying is valid. Good luck.
Security
I had a very interesting discussion with my carpool buddy about those atheism versus theism debates that are all the rage these days. He had some very astute observations despite his self-proclaimed lack of knowledge (he’s agnostic and I’m kidding). He noticed how the various camps typically claim that their side was the victor. See, debates aren’t exactly like the UFC. There isn’t a tap-out, a referee stoppage or a decision in the end. Instead, it is just a bunch of fans cheering for their fighter. What’s worse is that it is unlikely that one side would switch to the other as a result of such a brawl, but it is still entertaining and a great fuel source for conversation.
We discussed how atheistic arguments are sometimes made up of refutations of theistic arguments. Now, there is nothing wrong with this. If you can demonstrate that premises are incorrect or invalid you have successfully torpedoed the conclusion. What we observed is that in some cases this method (the refutation of theistic arguments) is successful. If they are successful (I think they are in some cases) then the argument for god is refuted. I agree with this. However, and I know this is obvious here, by refuting a positive proposition we have in no way confirmed it’s negative. In other words, refuting an argument for the existence of god does not get us to the truth claim that there is no god. I know, “the burden of proof is on you to prove god”. I agree. But if I cannot conjure up proof or my proofs are refuted, we simply slide into agnosticism. I can’t jump over the chasm into atheism without some logical help (I need some arguments). At the heart of it atheistic propositions, just like their theistic counterparts, are knowledge claims.
This of course led to all sorts of discussions regarding the problem of knowledge (a favorite of mine). Rarely, if ever, do I get the opportunity to talk about something that I think is fun and yet painful. So, I was sort of like the abominable snow man in this Looney Toons spoof. We talked about deduction and induction and the challenges of a priori knowledge. We talked about what meta-justification is. We even ventured off into the notions of “proof”. It seems that many today view scientific knowledge and proof in the same way and forget that even within science there are a priori assumptions at play; nevermind the fun that ensues when we talk about sense data and what that data represents. Needless to say debates are a great way to pass the time of a long commute!
Philosophy, Theology, Thoughts
Over the past couple of years I’ve documented my thoughts (in unpublished form) about the ultimate aim of education. Topics regarding virtue, utility, benefits to the state and socialization can be found all throughout my meandering thoughts. I stumbled across an excellent summary and thought I’d share.
For a true education aims at the formation of the human person in the pursuit of his ultimate end and of the good of the societies of which, as man, he is a member, and in whose obligations, as an adult, he will share.
- Pope Paul VI, Gravissimum Educationis (Declaration on Christian Education) October 28, 1965
In this short summary a proper balance between divine purposes and human existence is articulated. Education is more than being trained in a particular craft. It is also more than knowledge of things. It is a complete integration of techne, arete, episteme and other elements. Too much of one and not enough of the other leaves man underdeveloped and ill-equipped to participate in all facets of human existence in the 21st century. What do you think?
Homeschool, Philosophy
For those that weren’t aware I’m the Latin and Science teacher for our daughter. This was our first year of Latin and I used Prima Latina to introduce our daughter to Latin pronunciation, vocabulary and syntax. The set of books and cds allow us to use visual, aural and read/write mechanisms to acquire the language.
How did we do? If measurement is simply the volume of information retained by the student I think we did great. My daughter has memorized vocabulary, short prayers, verb conjugations, 1st declension nouns and other bits. I also know that we don’t (or shouldn’t measure) success in this one way. However, this text seems to teach to that end exclusively. I’m a fan of memorization where it works and in language acquisition it is a necessary feature when we are not learning directly. Yet, I still feel as though the text should have or could have made things a little more interesting. We all know that when children are interested in a topic they learn more effectively, perform better and acquire a greater body of information. When they are not, well, we know what happens there.
This is I think the failing of the text; It provides no real opportunities to cultivate interest, fun or anything else. It is simply a brute-force technique. I say this because my daughter, although she has learned all of the material (read: memorized) bemoans latin quite frequently. It is not because it takes tons of time, but because it is just tedious. Again, tedium is an unfortunate part of how we experience things, but when tedium can be avoided it should. In this case it wasn’t.
Prima Latina teaches ecclesiastical latin. I’m not a purist (or maybe I am), but this form of latin seems rather odd to teach unless you’ll be participating in some sort of liturgy. I would have liked to have seen the text teach latin in its classical forms. I had to rewrite the pronunciation rules provided in the text, avoid certain audio section and perform other minor surgeries on the material in order to align it more closely with its classical heritage. In summary, Prima Latina was useful for the arrangement of very introductory material, vocabulary lists, derivatives and tests. Those expecting stories, pictures, translation opportunities (short sentences) that make for a more well-rounded approach or those that would like to teach classical latin would do well to look elsewhere.
School