Epistemological Relativism

I’m constantly amused by the romanticism surrounding the transition and transformation of Colonial America into the collection of united states. We’ve all (those who have been educated in the U.S.) been exposed to the themes of oppression, misrepresentation and tyranny that are found throughout the writings of the period. In some ways this romanticism is well-founded. We witness a loose collection of colonies fight and scrape their way toward independence. Following on the heels of this new found independence they manage to construct and implement a system of government that was an amalgam of incredible ideas and ideals. The Enlightenment, the earlier Renaissance and the Scientific Revolution were the fertile seedbeds from which the founders harvested. All of these events, which seem trivialized by my meager coverage, are worthy of respect and study regardless of your political and religious views. Despite all of this, there is this irony just below the surface; the type of irony contained within a tragedian’s masterwork.

This tragedy was not, however, written by a poet attempting to craft a contemporary version of a Homeric epic replete with fatally flawed characters. Instead, this is the story of the truly tragic. Yet, in school, in our romanticism and admiration of the great and fantastic accomplishments all that is dark and gone awry is obscured or hidden away. There can be numerous reasons and explanations of why this is the way it is. How, though are we to learn from our mistakes and improve ourselves and our nation except through gazing long and hard at our past in all its greatness and imperfection?

Very early on we see the expansion and colonization (if I can use that word) of the west. As a newly united and sovereign nation there appears some implicit expectation of entitlement. We can see the transformation of the once oppressed into the oppressor. The Thrasymachian undercurrents can be seen when battle after battle is fought to annex more territory. Might makes right is what we can read between the lines. But, how can this be? Surely there are some foundational, unalienable rights that should not, no cannot, be violated. And yet by some weird twist of fate the new republic dons the mantle of tyranny.

What entitles a sovereign nation of any size to seize or purchase territory? What entitles a sovereign nation to marginalize an indigenous population in such a way as to sell the land that they live upon? This question raises a host of complex questions, that we loathe to address. Deep down we all know the answer. But to answer the question requires a great shift in thinking and action. Could this be why we don’t think about how our lovely land was formed?

To add additional irony, I am writing this from a chair in a state that was ceded to the United States after the Spanish-American War. Without a U.S. victory, I may not have been born here or anywhere for that matter. I enjoy the freedom to write and live as peacefully as possible. I enjoy what all the wars and innumerable deaths have provided. I am truly thankful, but it sounds odd or morbid to offer any sort of thanks for these events. The founders did great things in constructing a country such as this. It is unfortunate that it came at such an incredible price both before and after the founding of our nation.

Note: This isn’t some crazy anti-war polemic. I try to resist such polarization and classification, but if you must label me, consider me a supporter of patriotism, freedom and reform. Consider me optimistic that we can be truly human by improving ourselves through honest reflection.

For all of the “growing up” that the security industry has done in recent years it is amazing that proper distinctions are not being made when it comes to testing. No, not that testing, security testing. See, it demonstrates the point. When I say security testing all sorts of concepts are cognitively connected in that moment. Unfortunately, these connections are different for each of us. When some hear security testing they may think of using a new and improved fantastic commercial tool that produces beautiful output. Others think of an uber hacker sitting in a dimly lit room reading the network-byte-ordered data from some super secret application protocol. Yet others have never heard of the term security testing. Instead, they think you’re talking about pen-testing which, for the purposes of this discussion, is a fine synonym. The challenge is not with the definitions per se, but with the perceived effectiveness.

To be honest, it isn’t really the perceived effectiveness. Well it is, but let me elaborate. The challenge lies in the way in which we swap the effectiveness. Somehow we’ve managed to equate the ability of the uber hacker to find implementation bugs with that of a tool. We’ve given the tool the title of uber hacker. A title that, despite all of its merits, it cannot bear because it cannot deliver the goods. This does not mean that tools are not an expedient way to identify and respond to low-hanging fruits. They are great for that. But, sometimes it is thought that either low-hanging fruit represents the greatest risks or that this low-hanging fruit is really all the fruit there is on the tree. I don’t think this is the case and this is the problem. We cannot presume that what a tool can identify is all that there is and that what it can identify is all the low-hanging fruit there is. In the context of the tool it may be all there is, but in the context of an uber hacker or even a not-so-uber hacker it may not.

An example may help clarify what I am talking about. In a previous life we were testing an online application that was responsible for delivering sensitive documents to users with accounts. This was a manual, exploratory type of test because there were no real design documents that described the behavior. We immediately created an account and after some time visited the ‘I forgot my password screen’. The email we received looked odd. To make an exceedingly long story short, we discovered that the application was generating a pseudo-hash as the means of authenticating the user. This hash, if you can call it that, was really a simplistic Vignere cipher using the username as the key material. Of course you can imagine what happens next.

Running the variety of commercial tools out there could not find the flaw mentioned above, but we knew this. This was why we performed this ‘other’ kind of pen/security testing. Was this flaw easy to find? It was and tools were not equipped to find it. The fact is if we continue to label automated tools as uber-hackers-in-a-box, then we may be in for quite a surprise. This is why distinctions, especially with regards to pen/security testing are important. We have to be aware of how we are testing not simply that we are testing. And even with all that we have to remember that in the end test results without any identified vulnerabilities only tell us that we were unable to find any and not that they do not exist.

I was reading some of the SAML specification the other day (yes, I read those) and I found something particularly interesting in the Technical Overview.

In the HTTP message, the artifact carried (either as a query variable or as a control in a POST body). The artifact is a base-64 encoded string. It consists of a unique identity of the Identity Provider and a unique reference to the assertion (called the AssertionHandle). The artifact therefore enables the Service Provider to reference an assertion on the Identity Provider.

I am always the skeptic, especially when it comes to using words like unique. Is there really enough entropy in this AssertionHandle to guarantee its uniqueness? I decided to dig a bit deeper. Oh, just in case you can’t read my mind, this is important because a predictable AssertionHandle allows you impersonate an authenticated user on any system that uses the broken identity provider. Nice, isn’t it?

In section 3.6.4 contains the information I was looking for, I think.

With respect to this binding, an artifact is a short, opaque string. Different types can be defined and used without affecting the binding. The important characteristics are the ability of an artifact receiver to identify the issuer of the artifact, resistance to tampering and forgery, uniqueness, and compactness.

I mentioned AssertionHandle and the above citation uses artifact. An artifact is the collection of three pieces of information [Section 3.6.4.2]: TypeCode (determines the type of artifact), SourceID and MessageHandle. An AssertionHandle is a type of MessageHandle. The generalized requirements are decent, but please tell me there is a bit more direction.

The notation B64(TypeCode EndpointIndex RemainingArtifact) stands for the application of the base64 [RFC2045] transformation to the catenation of the TypeCode, EndpointIndex, and RemainingArtifact.

The following practices are RECOMMENDED for the creation of SAML artifacts:

• Each issuer is assigned an identifying URI, also known as the issuer’s entity (or provider) ID. See section 8.3.6 of [SAMLCore] for a discussion of this kind of identifier.

• The issuer constructs the SourceID component of the artifact by taking the SHA-1 hash of the identification URL. The hash value is NOT encoded into hexadecimal.

• The MessageHandle value is constructed from a cryptographically strong random or pseudorandom number sequence [RFC1750] generated by the issuer. The sequence consists of values of at least 16 bytes in size. These values should be padded as needed to a total length of 20 bytes .

The MessageHandle/AssertionHandle is important. The minimum length of this sequence is required to be 16 bytes, which contains roughly 128 bits of entropy. This is similar to the output produced by MD5. The artifact structure requires the MessageHandle to be 20 bytes in length so padding is used if the 16 byte minimum is used. Why the specification does not require 160 bits of entropy (20 bytes) instead is a mystery.

We get the requirement that the MessageHandle needs to be random. We even get the reference to RFC 1750. So far, so good. Then I remember the ‘RECOMMENDED’ at the beginning of the section. RFC 2119 says that when something is a recommendation that it can be ignored. Of course, it says it much nicer and with a few qualifications attached, but it can still be ignored. This, I’m sure, isn’t something to lose too much sleep over.

So, if the developer decides to implement the recommendations, utilize the advice in RFC 1750, uses a reliable PRNG source, reads the Security and Privacy Considerations, uses defensive programming techniques and does not produce any implementation bugs in the code that implements the artifact generation and validation then I think we’re okay.

I don’t mean to sound alarmist, because I’m sure there are some implementations that get it right (intentionally or unintentionally). I’d even say that if one used MD5, in all its broken glory, to generate the requisite random bits that it wouldn’t be the end of the world. This is especially so if artifact timing is used according to the recommendations in the Security and Privacy Considerations. What is worrisome is the distributed nature of SAMLs use. If a collection of systems use a Identity Provider that does not provide a unique MessageHandle then all systems are at risk. The increase of federation and identity services increases the impact of such an event. As a general principle this type of concentration should be given an extra measure of scrutiny. I will be snooping around the various implementations in the coming weeks and will hopefully document my findings.

There is an article over at Dark Reading that discusses the challenges of vulnerability management and the slow integration of disparate products to produce a more comprehensive platform for such initiatives. Many are familiar with the pain of managing the variety of automated and manual tasks to measure compliance. Especially fun is the prioritization of the torrent of findings produced by these tasks.

Enterprises also must consider risk from the business perspective rather than just go about wielding a patch checklist. “If a system has limited business value, it’s less important than a vulnerability in a server that holds our credit card numbers,” Maiwald says.

I agree a hundred percent with the first sentence and disagree 100 percent with the latter. That last statement is so filled with assumptions that it is hard to take it seriously. It is an exceedingly dangerous generalization. It takes a systems involvement in storing sensitive information as the primary metric in prioritization. Now, I know that it was only an example in the article, but it was a bad one. This is because there are many, many more factors that should be considered when prioritizing remediation. A quick shotgun list would include things like position in the network (it is not a binary internal/external game), the relationship (logical and physical) between a system and other identified critical systems, the characteristics of the identified vulnerability and the types and roles of users on the vulnerable system. I think it is time to handle prioritization in a way that isn’t a binary comparison between things like internal/external or sensitive/public, don’t you?

I spoke about vulnerability and threat ratings here a couple of weeks ago. What I didn’t really give enough attention to was the testing of this rating system. In short, once you’ve identified a vulnerability you have to somehow communicate the overall risk. This is typically done with assigning some value to impact and likelihood. You read more in my previous post. I think most of us forget a critical component in the development of these systems. We forget to test them.

Come on, we’re experts right? Do we really need to test? What is the value of the test? Well, if we are committed to delivering accurate and realistic information about identified vulnerabilities we must demonstrate that our system can provide that. A good way to provide this proof is to play test. I know, we all think we’ve created the silver-bullet so it feels wrong. Trust me though. If you do not test your new-fangled system the second it gets in the hands of a developer or infrastructure group (or even your own localized security group) and produces different results than what you came up with you will be in trouble.

One of the primary properties of such a rating system is reproducibility (the other, of course, is accuracy). A given vulnerability in the hands of a couple of knowledgeable people should roughly give similar results. If it does not, this should raise some sort of alarm bells. If you are in fact trying to create s system that is coherent and reproducible you you have to make sure it passes the litmus test. In rating systems it is imperative that you attempt to minimize subjectivity and increase reproducibility in the results. The problem of subjectivity is challenging because everyone has their own experiences that affect the process. Add to that the fact that some are limited by what they can conceptualize and results become amazingly erratic. As a good example, a system with a single dimension each for likelihood and probability is nearly impossible to deal with. This is because, and I’ve talked about this before, that there is too much information loaded into a single dimension. It is difficult to justify, for example, a particular value for likelihood. There are too many categories of information loaded into the concept. A simple test of your system will help determine if your rating elements are suffering from category overload. In the end if you approach your system with the goal of objectivity and reproducibility and add in testing at the end I think you’ll have a system that delivers, as much as possible, realistic and accurate information.

There is no question that software reliability is a requirement of most software development projects. Reliability, however, is often very narrowly defined. In the security world reliability is great label to describe how a system (application or otherwise) handles a variety of unexpected conditions that are not at all driven by normal use cases. There is a laundry list of useful techniques to measure a system’s reliability from the context of security. These are almost the complete opposite of how a normal QA cycle is run. Yes, there are performance testing aspects in the overall cycle, but these are still focused on the narrow definition of reliability

This narrow definition of reliability is focused on what I call positive availability. It takes the known and valid operations of a system as defined by the use cases and attempts to find situations where that system’s implementation deviates from what is expected. You’ll notice the change from reliability to availability. This is intentional because software reliability is so narrowly defined in this context that it appears to be nothing more than ensuring the availability of the system and its information through this sort of examination. I know I shouldn’t have to say this, but I’m not saying that this process is bad or not useful. It is useful. What it isn’t, though, is comprehensive.

This gets even more interesting because many times reliability is not viewed as a security concern. To some, security is the management of identities, assertion of these identities and the entitlements that these entities should receive. This is a grave mistake. Reliability is at the very core of security. Measuring how software protects confidentiality, maintains integrity and provides availability is the the primary task of security. Just as the process of “positive availability” is limited so too is the idea that security is only about managing and enforcing entitlements. And so, because of this, we have this chasm. The software process stops at positive availability and labels a project reliable. The security process stops at entitlements and labels the same project secure.

To provide the most utility the reliability that is measured by development projects should include negative availability. As for security, it needs to expand to meet the needs of not only who gets to what, but also how the application is structured and implemented with this same negative availability in view. I use the term negative availability to describe that vague process of examination where standard use cases are twisted and bent. Instead of trying to find if an application breaks through standard usage this process tries all forms of manipulation that are not defined in requirements and designs to find defects. The aim of this type of defect hunting is to find a failure in confidentiality, integrity and availability mechanism of software. Because standard use cases attempt to communicate how software should operate this sort of negative availability uses that source material to see if the software has protected itself when things do not go as expected. Typically software is not constructed with this in mind.

We quickly see that my negative availability moniker isn’t inclusive enough. Once a flaw is found in software through this sort of testing it isn’t always the loss of availability that is the result or even the goal. The fact remains, however, that this sort of testing and examination is crucial to understanding how reliable an application really is. It is dangerous to presume that software is reliable when it does not focus on this negative availability. It is as dangerous, if not more so, to presume that an application is secure because identities and entitlements are managed properly when no review of the software’s construction and operation is performed.

In the information security world nothing is more fun than using confusing terms. This isn’t done intentionally of course, but it happens. It happens often, I might add. In fact, one set of ideas that is constantly subject to debate is the notion of risk and threats. This isn’t because people enjoy arguing, but it is due to the context from which various people are approaching the subject. Who would have though that information security could be so post-modern? The variety of experiences lead many to define terms quite differently. I try to approach these individual elements not as linguistic islands, but as part of a system that must be coherent. If it does not pass this test, then I’ve gone wrong somewhere. So, I will now present my version. Keep in mind this is probably too specific for physical/environmental risks. It can still be mapped to these definitions, but there will be duplication. You’ll see what I’m talking about. I will try to avoid formalized dictionary definitions. Hopefully that will help.

Risk: This is an abstract concept or placeholder for some loss, damage or event that would adversely affect information.

Threat: This is where the wheels usually fall off so let’s see how we do. A threat is a concrete instance of a risk. For those OO programmers out there, think of this as an instance of the Risk class with the relevant instance variables that uniquely identify it. This is different than those with a military background where a threat is more like an entity, actor or agent that bears some responsibility for an event. I’ll talk more about this in a minute.

Vulnerability: This is a weakness that allows a threat to be realized. This weakness could be missing controls, implementation errors or a variety of conditions that make a threat move from possible, but not realizable to possible AND realizable however difficult it may in fact be.

Attack: This is the mechanism by which a vulnerability is exploited or used. There is usually some activity that must be performed to do something with a vulnerability. A vulnerability must be exploited by an attack. Without the attack component a vulnerability just remains a weakness of some sort. No event (see risk) has occurred until an attack is launched against vulnerability.

Threat Agent: This is the entity that is ultimately responsible in any measure for the realization of a threat. I would say particular threat, but threats are already particular, but I digress.

Now, these are working definitions, but let’s see how they all fit together:

A threat agent uses an attack to exploit a vulnerability in order to realize a threat.

This seems to work. But where does risk fit into all of this? Our above definition does not cohere with the rest of the system. I think we need to revise this a bit. Risk in this situation is particular and not abstract. It is concerned with communicating the impact and possibility of the above systematic statement mapped to a specific instance (I know it sounds funny to me too). Risk then must be anchored in some way to a particular. In order to communicate impact and likelihood you must be acquainted with the threat agents, attacks, vulnerabilities and threats. If you skimp on any of these you have not accurately measured risk. So, a risk assessment is concerned with understanding these particular threats and how they can be realized. I think this works.

In a bit I will break it out even more and show how the various components can be manipulated, measured and rated (and the challenges with each). Again, once we do this, we have to see if the system is still coherent when taken together to ensure we’re getting everything right.

UPDATE: In my haste I seem to have forgotten the most important element of all. I completely omitted assets. Assets are the objects that are threatened. Without them, all the other concepts seem to float around. An asset-centric approach is necessary to give realism to the risk analysis process. This keeps one from dreaming up threats and/or vulnerabilities that cannot be tied back to an asset.

Developing a coherent security strategy that focuses on real risks as opposed to the perceived risks in an environment is a notorious challenge. In most cases organizations rely on an ad-hoc approach that is informed by industry experts. Of course there is nothing wrong with this because in some cases these experts get it right. But trends in the industry, however reliable, should not be the primary justification for integrating these recommendations into an organization’s overall strategy. What I’ve been playing with for the past several months is a way to graphically represent the primary components of information systems.

This somewhat abstract representation provides, I think, a method for appropriately mapping the variety of security related concepts onto these primary components. It allows you to see the logical relationships between these primary components. This mapping and relationship allows you to “see” all the security-related practices, controls and technologies as a large story. It presents it as the forest rather than isolated collections of trees to use the “forest in the trees” analogy. I know, it sounds rather obvious doesn’t it? Sadly, there are not many working on this forest view. Sure there are some attempts being made and while they are still useful they have a hard time showing the relationships between the categories. Understanding this interaction is necessary in order to identify and understand real risk.

After many, many sketches, some thinking, research and more sketching I’ve come up with four large abstract classes in this taxonomy. They are not so large as to lose meaning and not small that they become too specific. The real challenge to this is graphically representing these classes while still maintaining their relationship to one another. It is easy to draw a pretty set of circles with data in the innermost circle and the relevant classes of security controls, but such a picture does not accurately communicate the common relational qualities of the individual classes and this is what we are trying to achieve in order to see the risks and develop a practical strategy.

Application - All services, components and applications that are not a required part of an operating system

Platform - The combination of the instruction set architecture and operating system

Persistence - The location and mechanism used to maintain data at rest

Data - The elements that are Created, Read, Updated and Deleted (Yes, this is also an abstraction)

Yes, that really is it. Using logical (for network-based) and physical access methods gives the classes the necessary linkage to one another. Adding actors, entities or whatever term you’d like to use to identify a thing responsible for accessing systems allows you to map the end-to-end relationship of generic (or specific if you’d like) information flow. Before I completely define what these classes are and explain why something like ‘network’ is conspicuously absent I’ll let you think about it for a bit. Oh, I almost forgot. There is a another taxonomy of people, process and technology elements that you can use to “inspect” these other classes to determine what level of security maturity they possess. I’ll give you some ideas on what those are in a later post.

Many times in my religious experience I’ve had a desire to throw the baby out with the bath water because of the sheer stupidity of the present. I’m sure I’m not alone there either. For example, when reading about early church history and practices I would imperiously declare that the Greco-Roman Gentile converts-turned-leaders had gotten it all wrong. It was because of this confusion that Christendom was in such a sorry state today or so I thought. Naturally, the only real alternative was to throw everything out and start from the beginning. It took quite some time to realize that this beginning-ness had problems of its own. Where was the beginning? This jettisoning of tradition (whatever was left to begin with), the community of faith (past and present) and the general attitude of mistrust, however, made it nearly impossible to recover any sort of religious bearings. Barth is amazing because he manages to understand this dilemma and chart a course that avoids the problems that come from this type reaction and yet remain fluid enough to introduce needed corrections to the community. Barth will not allow everything to be discarded. He may give away too much in assuming that the community of faith did not go critically awry in the not-so-distant past, but he does not create an ivory tower out of this community of the past that is hitherto immutable.

Certainly, the assumption behind all this will be that the community itself may have been on the right track in the recent or remote past, or at any rate on a not altogether crooked path. Consequently, fundamental trust instead of mistrust will be the initial attitude of theology toward the tradition which determines the present-day Church. And any questions and proposals which theology has to direct to the tradition will definitely not be forced on the community like a decree; any such findings will be presented for consideration only as well-weighed suggestions. Nevertheless, no ecclesiastical authority should be allowed by theology to hinder it from honestly pursuing its critical task, and the same applies to any frightened voices from the midst of the rest of the congregation. The task of theology is to discuss freely the reservations as well as the proposals for improvement which occur to it in reflection on the inherited witness of the community.

– Karl Barth, Evangelical Theology (Eerdmans, 1963) 43.

–Update: Did anyone notice that I used the word ‘hitherto’?

As a parent who is always proud of my children’s successes, their best efforts despite failure and the ability to laugh at a moments notice, I have to say that I was particularly impressed this afternoon. I was reading my new copy The Dangerous Book for Boys (more on that in another post) when I turned to a page with a picture of Gaius Julius Caesar. I had been showing my children page after page of all the cool things in this book. I asked my seven-year-old daughter who she thought the person in the picture might be. She nonchalantly replied that it was Julius Caesar. Now, I don’t know about you and maybe I don’t get out enough, but I sure didn’t know who Julius Caesar was when I was in 2nd grade. Another proud moment for a parent.