Nabble - CWE Research List

Re: Some asking about CWE.

2008-09-05T13:46:23Z

On Mon, 25 Aug 2008, Tadashi Yamagishi wrote:

> Question1:About Hierarchy diagram.
> I made CWE-635(Weaknesses Used by NVD) a hierarchy diagram
> referring to cwe_classification_tree.pdf.
> The hierarchy diagram is appended.
> cwe_classification_tree.pdf shows the following.
> CWE-20 is a child of CWE-19.
> CWE-22 is a child of CWE-21.
> CWE-134 is a child of CWE-133.
> However, CWE-1000(Natural Hierarchy) shows another parents.
> I am confused. Are two or more parents permitted in CWE ?

Yes. This is for two reasons:

(1) there are multiple ways of looking at the same weakness, so we want
to support these different ways. So, different views can express
different relationships.

(2) Some weaknesses can be fairly complex, so they can be classified in
different ways, even within the same view. Ideally, it would be
good to have a view in which every weakness can have only one
parent. This is very difficult to achieve in practice; we think
that the concept of chains and composites helps to explain why
this classification is so difficult. We are making significant
progress within the "natural hierarchy" (view 1000), but we
will not have this finished by the release of CWE 1.0.

> Question2:About the classification of Dos( Denial of Service ).
> DoS is not classified in CWE.

DoS is not classified because it's a "consequence" of some weakness - just
like "loss of integrity" is a consequence. In CWE 1.0, we will have
multiple ways of trying to determine which weaknesses can lead to a DoS:

- a new OWASP Top Ten 2004 view has category A9, Denial of Service.

- as a result of schema changes in 1.0, our Consequence element has
been improved so that you will be able to search CWE for
Consequence_Scope = Availability.

> How do you classify it when the cause of the DoS is not understood
> in the vulnerability report?

This is a very difficult question, and I'm not sure how to handle it. In
the case of databases of public vulnerabilities (like NVD), the databases
often don't have solid information about the underlying weaknesses that
led to the vulnerability. Sometimes, the only vulnerability information
is something like "Product X can crash from a malformed packet" - you
might see this in a software vendor advisory, for example.

Since the "DoS" phrase alone doesn't talk about any specific weakness, CWE
is not currently capable of modeling it.

This is an example of a challenge: how should you map to CWE when you're
dealing with issues that aren't exactly weaknesses? We hope to be able to
develop methods of handling these kinds of issues. In fact, one of the
upcoming white papers will cover some of the common difficulties that
people will face when mapping to CWE. In addition, sometime after the
release of CWE 1.0, we will try to improve the current NVD classifications
so that they are more suitable for handling incomplete vulnerability
information. Hopefully we will be able to address this issue somehow.

- Steve

CWE 1.0 to be released Tuesday, September 9, 2008

2008-09-05T13:29:09Z

All,

We will be releasing CWE 1.0 on Tuesday, September 9.

While we had a goal for everything to be finished by the end of August, we
want CWE 1.0 to be as stable as possible, especially with respect to the
schema. Ironing out the issues with the schema has taken more time than
we wanted, plus we have added a number of new elements.

During our interactions with various community members over the summer,
we've realized that it would be best for us to write a number of white
papers, as well as creating some new views. These were somewhat
unexpected additions that came in July and August, so this introduced more
work than we had originally expected.

We didn't want to release CWE 1.0 too early, only to make some more
changes soon after we released it because it was incomplete. So we've
slipped in our schedule, but the quality will be higher.

This is our biggest release to date, and we believe that it will be worth
the wait. Thank you for your patience.

- Steve

Some asking about CWE.

2008-08-25T02:52:26Z

Dear CWE group,

I am Tadashi Yamagishi
in Information-technology Promotion Agency, Japan (IPA).
We(IPA) have Vulnerability Countermeasure
Information Database (JVN iPedia) for Japanese IT user.
http://jvndb.jvn.jp/index_en.html

JVN iPedia adapted CVSS(Common Vulnerability Scoring System) last year.
The next step, I think that JVN iPedia need CWE.
I am studying CWE draft 9 now.
I have three questions about CWE.

Question1:About Hierarchy diagram.
I made CWE-635(Weaknesses Used by NVD) a hierarchy diagram
referring to cwe_classification_tree.pdf.
The hierarchy diagram is appended.
cwe_classification_tree.pdf shows the following.
CWE-20 is a child of CWE-19.
CWE-22 is a child of CWE-21.
CWE-134 is a child of CWE-133.
However, CWE-1000(Natural Hierarchy) shows another parents.
I am confused. Are two or more parents permitted in CWE ?
I think that cwe_classification_tree.pdf and CWE-1000
are comprehensible when it is the same.

Question2:About the classification of Dos( Denial of Service ).
DoS is not classified in CWE.
How do you classify it when the cause of the DoS is not understood
in the vulnerability report?

Question3:About XSS vulnerabilities.
There are lots of XSS vulnerabilities
by the UTF-7 encoded string problems in Japan.
for example:
CVE - CVE-2008-1468
http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2008-1468
JVNDB-2008-000018 - JVN iPedia
http://jvndb.jvn.jp/contents/en/2008/JVNDB-2008-000018.html

CVE - CVE-2008-2168
http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2008-2168

CVE - CVE-2008-0005
http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2008-0005

I want to classify the detail of XSS.
Can I choose a CWE-ID more detail than CWE-79
about XSS(UTF-7 encoded string problems)?

I look forward to your reply.

Sincerely yours,
Tadashi Yamagishi
IT Security Center (ISEC)
Information-technology Promotion Agency, Japan (IPA)
E-mail: t-yamagi@...

CWE_635.PNG (17K) Download Attachment

FW: Natural Hierarchy email

2008-07-30T07:32:42Z

Hi All,

In earlier CWE drafts, the CWE hierarchy was mostly based on weakness
abstractions - that is parent / child relationships were focused on the
abstraction of the core weakness being addressed by each. But
weaknesses in some areas of the hierarchy were grouped together based
on a common attribute such as language, resource or consequence, the
relevance of which can change based on the context of the weakness
occurrence or the perspective of the viewer. As a result, weaknesses
were sometimes children of categories that had little to do with the
underlying problem, but provided a convenient way in which to view
issues from certain perspectives. While these perspective and context
based views into CWE are very useful for some, a view based solely on
weaknesses and their abstractions is also needed.

In an effort to provide this additional perspective into CWE, the CWE
team began adding additional relationships to some entries in the
release of draft 9. These relationships are based on the fundamental
weakness behind each entry and how it relates to other weaknesses in
terms of abstraction and small variations on similar issues. We are
attempting to identify relationships that are as independent of
specific language, technology, programming idiom, and resource as
possible.

The slight shift in focus between drafts 7 and 8 led to several new
schema constructs being added for draft 9, one of which being the Class
/ Base / Variant attribute added to the weaknesses structure - a
derivation of the draft 8 Grouping attribute - in order to facilitate
the abstraction-based relationships. The CWE team has expanded on this
for the release of version 1.0, and can be demonstrated in part by the
top level of the natural hierarchy, view-1000, and a small subset of
their children shown below.

* Range Errors
o Unbounded Transfer ('Classic Buffer Overflow')
o Incorrect Offset into a File
* Equivalence [Draft Concept for 1.0]
o Insufficient Comparison
o Failure to Resolve Case Sensitivity
o Encoding Error
* Incorrect Calculation
o Off-by-one Error
o Divide by Zero
* Failure to Fulfill API Contract (aka 'API Abuse')
o Failure to Follow Specification
o Failure to Provide Specified Functionality
* Coding Rule Violation
o Violation of Secure Design Principles
o Use of Inherently Dangerous Function
o Use of Potentially Dangerous Function
* Security Features (Protection Mechanism Failure) [Merging is new for
1.0]
o Incomplete Blacklist
o Weak Encryption
o Insufficient Authentication
* Use of Insufficiently Random Values
o Insufficient Entropy
o Small Space of Random Values
* Interaction Error
o Compiler Removal of Code to Clear Buffers
o Interpretation Conflict
o Reliance on Data Memory Layout
o Behavioral Change in New Version or Environment
* Insufficient Control of Resource Through its Lifetime
o Improper Resource Shutdown or Release
o Incorrect or Incomplete Initialization
o External Influence of Sphere Definition
o Incorrect Resource Transfer Between Spheres
o Operation on Resource in Wrong Phase of Lifetime
* Insufficient Control Flow Management
o Insufficient Synchronization
o Always-Incorrect Control Flow Implementation
o Incorrect Behavior Order
o Deployment of Wrong Handler
* Failure to Handle Exceptional Conditions
o Missing Error Handling Mechanism
o Unexpected Status Code or Return Value
* Failure to Maintain Cohesion in Structured Messages or Data [New
Concept for 1.0]
o Failure to Sanitize Special Elements
o Deletion of Data Structure Sentinel
o Improper Null Termination

It is important to emphasize that CWE will still be navigable by common
attributes, such as language, consequence and platform after the
release of CWE version 1.0 through various "views". Prior relationships
were preserved in the development of the natural hierarchy and can
still be viewed from the individual CWE Definition pages or through an
alternate view where the relationships carry more meaning. For example,
CWE will be supporting a Programming Concepts view as well as a 7
Pernicious Kingdoms view starting with version 1.0.

The 7 Pernicious Kingdom view will be based on the original 7PK paper
and is focused primarily on usefulness to developers and grouping
weaknesses by categories. The Programming Concepts view will be an
expansion of sorts from the 7PK view which will incorporate the
additional content that has been added to CWE. These views will
accommodate navigation similar to previous drafts of CWE using
categories such as pointer issues, mobile code issues, error handling,
data handling, cleansing, comparison and canonicalization, time and
state, temporary file issues, and channel and path errors. Different
views allow for the organization of the content in the manner that
makes the most sense for the intended audience of each view.

The natural hierarchy as described above is the result of a
reorganization of the draft 8 hierarchy. We focused more on
abstractions of the core issues behind each weakness to try to unify
the perspective throughout the natural hierarchy. This entails moving
away from relationships based on weakness context, environment, or
technologies. In draft 8 of CWE, for example, CWE-5 Misconfiguration:
Data Transmission without Encryption is a child of CWE-4 J2EE
Environment Issues. While this is a useful relationship, especially for
a J2EE developer, CWE-4 is a technology specific grouping of CWE-5 and
there may or may not be any relationship between CWE-5 and its siblings
other than a common technology. In version 1.0 of CWE, an additional
"ChildOf" relationship was added to CWE-5 relating it to CWE-319
Plaintext Transmission of Sensitive Data because CWE-5 is a technology
specific variant of the more abstract weakness, CWE-319. It is
important to note that CWE-5 is still a "ChildOf" CWE-4. This
relationship is maintained in a different "View", another schema
construct added for draft 9.

To further illustrate these changes, CWE-444 HTTP Request Smuggling was
a child of CWE-442 Web Problems in draft 8. Once again, this
relationship might be a useful way for a web developer to come across
relevant problems, but it doesn't tell us very much about the more
abstract issue behind CWE-444 nor does it give us any indication as to
how CWE-444 is related to its siblings aside from web technologies. For
draft 9, a relationship was added to CWE-444 making it a "ChildOf"
CWE-436 Interpretation Conflict, which is more aligned with the
fundamental problem behind CWE-444.

A hierarchy focused on abstraction can allow for more intuitive and
consistent navigation of the CWE tree from top to bottom. This approach
may be more useful to researchers, educators and coverage mapping for
code analysis tool vendors. Organizing the weaknesses hierarchically by
abstraction helps the CWE team identify gaps and inconsistencies in CWE
and it also helps tool vendors identify gaps in their mappings. For
example, duplicates CWE-132 Miscalculated Null Termination and CWE-170
Improper Null Termination used to be in disjoint segments of the
hierarchy. When reorganized by core weakness, they fell in the same
location; thus identifying the content of each entry as duplicates was
much more obvious. Furthermore, applying "Chains" and "Composites" to
an abstraction-based hierarchy can be useful for identifying higher
level trends in weakness occurrences, potential mitigation strategies
and their impact, and is another useful mechanism for finding gaps in
CWE coverage. In addition, these concepts have helped us to understand
why classification has been such a difficult challenge in the past.

Another benefit of collocating similar core weaknesses is the ease of
applying a consistent vocabulary across CWE. Weaknesses that had no
relationships before are brought together and allow the CWE team to see
inconspicuous relationships and trends. For example, CWE-696 Incorrect
Behavior Order was created as a weakness class for several entries
where the fundamental problem was performing operations in the
incorrect sequence. The first level of children of CWE-696 under the
natural hierarchy now read:

* CWE-179 Incorrect Behavior Order: Early Validation
* CWE-408 Incorrect Behavior Order: Early Amplification
* CWE-551 Incorrect Behavior Order: Authorization Before Parsing
and Canonicalization

As a result, when we peer into other views of CWE and come across these
weaknesses, we can immediately identify what the core issue is.

The natural hierarchy view is simply one view meant to unify the
weaknesses within CWE in a way that can be understood by a broad
audience based on underlying factors of each weakness rather than the
context in which the weakness occurs or how the weakness can be
mitigated. By ignoring such variables as environment, consequence,
mitigation, attacks, and relationship impact, we have been able to find
the factors that make each weakness unique and canonical. Likewise, we
have been able to identify gaps and create a more mature CWE model,
have identified areas for further research, and laid the groundwork for
a more solid understanding of the complexity of weaknesses and their
impact. With such a foundation we can then start adding in the
previously discounted variables to create a more complex and thorough
understanding of issues caused by the existence of these weaknesses.

Any thoughts, suggestions or concerns are welcome, either to this
thread or cwe@....

-Conor Harris

RE: Industry Analyst Coverage of CWE

2008-07-09T09:40:26Z

You typically start this process by filling out the briefing requests
on each analyst site:

http://www.forrester.com/Briefing/Process
http://www.burtongroup.com/Contact/VendorBriefing.aspx
http://www.gartner.com/it/about/vbriefings_faq.jsp

-----Original Message-----
From: Robert A. Martin [mailto:ramartin@...]
Sent: Wednesday, July 09, 2008 12:24 PM
To: McGovern, James F (HTSC, IT); cwe-research-list@...
Subject: Re: Industry Analyst Coverage of CWE

Hi James,

That would be something we would be interested in doing but probably
through direct discussion with specific interested individuals at each
organization.

Anyone with suggestions on who to work with at these or other such
groups or on ideas about how to educate them about CWE please feel free
to contact me directly or through this list.

We know some of the appropriate people but would welcome suggestions and
ideas.

Bob Martin
CWE Project Leader

At 11:57 AM -0400 7/9/08, McGovern, James F (HTSC, IT) wrote:
> As soon as the next version of CWE is released, is there an
>equivalent plan to communicate this to Gartner, Forrester and The
Burton Group?

>
>
>***********************************************************************
>** This communication, including attachments, is for the exclusive use
>of addressee and may contain proprietary, confidential and/or
>privileged information. If you are not the intended recipient, any
>use, copying, disclosure, dissemination or distribution is strictly
>prohibited. If you are not the intended recipient, please notify the
>sender immediately by return e-mail, delete this communication and
>destroy all copies.
>***********************************************************************
>**

*************************************************************************
This communication, including attachments, is
for the exclusive use of addressee and may contain proprietary,
confidential and/or privileged information. If you are not the intended
recipient, any use, copying, disclosure, dissemination or distribution is
strictly prohibited. If you are not the intended recipient, please notify
the sender immediately by return e-mail, delete this communication and
destroy all copies.
*************************************************************************

Re: Industry Analyst Coverage of CWE

2008-07-09T09:24:24Z

Hi James,

That would be something we would be interested in doing but probably
through direct discussion with specific interested individuals at
each organization.

Anyone with suggestions on who to work with at these or other such
groups or on ideas about how to educate them about CWE please feel
free to contact me directly or through this list.

We know some of the appropriate people but would welcome suggestions and ideas.

Bob Martin
CWE Project Leader

At 11:57 AM -0400 7/9/08, McGovern, James F (HTSC, IT) wrote:

> As soon as the next version of CWE is released, is there an equivalent
>plan to communicate this to Gartner, Forrester and The Burton Group?
>
>
>*************************************************************************
>This communication, including attachments, is
>for the exclusive use of addressee and may contain proprietary,
>confidential and/or privileged information. If you are not the intended
>recipient, any use, copying, disclosure, dissemination or distribution is
>strictly prohibited. If you are not the intended recipient, please notify
>the sender immediately by return e-mail, delete this communication and
>destroy all copies.
>*************************************************************************

Industry Analyst Coverage of CWE

2008-07-09T08:57:59Z

As soon as the next version of CWE is released, is there an equivalent
plan to communicate this to Gartner, Forrester and The Burton Group?

*************************************************************************
This communication, including attachments, is
for the exclusive use of addressee and may contain proprietary,
confidential and/or privileged information. If you are not the intended
recipient, any use, copying, disclosure, dissemination or distribution is
strictly prohibited. If you are not the intended recipient, please notify
the sender immediately by return e-mail, delete this communication and
destroy all copies.
*************************************************************************

Re: Suggestion of Repositioning CWE #244

2008-07-07T17:11:42Z

On Thu, 3 Jul 2008, ljknews wrote:

> At 3:08 PM -0400 7/3/08, koo wrote:
>
> > We suggest that CWE #244, Failure to Clear Heap Memory Before Release,
>
> It seems to me that it would be sufficient for the operating
> system to clear the memory before reallocation to a process.

One thing that I try to do when thinking about classification within CWE
is to separate the solution from the weakness itself. Most weaknesses
could have multiple solutions.

> Is there a separate item for clearing stack memory ? That
> would seem vulnerable in the same ways.

We don't have a CWE that's about stack memory.

But, this raises an abstraction question that we've been dealing with, and
which I touched on in the fall of 2007.

The general question is: do we create an individual CWE for stack memory?
How about the other resource types that were mentioned by Robert Seacord?

Both "failure to clear stack memory" and "failure to clear heap memory"
are related to resources. Their common parent is "memory," which we
currently think of as a fairly basic resource that's reasonable to cover
in CWE. So maybe there's a conceptual parent, "failure to clear memory."
Then you could go up another level, to the general concept of "resource",
i.e. "failure to clear resource" (which is basically a rephrasing of 404
and/or 459).

This "resource-specific abstraction" happens in other places in CWE, for
example CWE-122 (Heap-based buffer overflow) and CWE-121 (Stack-based
buffer overflow), as well as the various descendants of CWE-552 Files or
Directories Accessible to External Parties; each descendant covers a
different type of file, such as a backup or log file.

The general issue is, how specific must we get in order to create CWEs?
This was discussed in the fall. A combinatorial explosion could result if
we go too deep; we lose expressiveness if we're not specific enough.
This problem is now less severe since we have abstraction levels (Class,
Base, Variant) - we'll usually label resource-specific abstractions as
variants, so these could be removed from various views that don't want to
go that deep. It might also be useful to label the "dimension" along
which variants can occur, such as "resource-specific."

If we have this type of data available, then we don't need to reach the
same depth across all of CWE. We could add new nodes on an "as-needed"
basis if there is sufficient demand for it, and those nodes would exist in
some views, but not others.

In this particular case, the question is - is there a need to create
separate CWE entries for the failure to "clear" different types of memory,
and/or the different types of resources in general?

- Steve

Re: Suggestion of Repositioning CWE #244

2008-07-07T16:51:58Z

On Thu, 3 Jul 2008, koo wrote:

> We suggest that CWE #244, Failure to Clear Heap Memory Before Release, also
> be a child of CWE #226, Sensitive Information Uncleared Before Release.
> #244 is just the specific case of (mis)using realloc() for sensitive
> information.

This makes sense. Pending further commentary from the researcher list, we
will do this.

> They are both children of CWE #633, Weaknesses that Affect Memory, but
> don't have any other connection in the CWE that we can see.

This is a good example of the kind of "cleanup" and mapping task that we
are hoping to partially address with improved mapping guidance and
organizational views such as the natural hierarchy. We hope that the
natural hierarchy would provide a mechanism for more easily finding these
relationships.

> For that matter we suggest #244 NOT be a child of #404, Improper Resource
> Shutdown or Release. #404 is about freeing what you allocate, unlocking
> what you lock, etc. - it can lead to memory leak or resource leak.

We intend for 404 to be a pretty high-level entry that covers any
situation in which the developer does not properly "get rid of" resources
such as memory, locks, files, processes, and connections. Note how this
is separable from the *consequences* of those actions, such as information
leaks or resource consumption. It might be good for us to modify 404 to
better emphasize how we view it.

> #244 talks about sensitive information being exposed because it is not
> erased from a resource before being released.

Here, #244 suffers a little bit from a perspective problem.
Specifically, it is currently written to emphasize the information leak,
instead of the root cause of using memory allocation routines that might
release memory back to the OS in an uncontrolled fashion.

So, the descriptive text for 244 would need to be modified a bit to "fix"
the perspective to focus more on the weakness.

Now, about the relationships between 226, 244, and 404.

As you observed, 226 and 244 don't share any parents besides CWE-633,
Weaknesses that Affect Memory. 226 is currently classified under CWE-200,
Information Leak. But "information leak" is usually a consequence (i.e.,
resultant) - so it's typically the final link of a chain. While we
currently say that 226 is a ChildOf information leak, it would be better
represented as a CanPrecede relationship.

We then face the question of where 226 fits under the natural hierarchy,
since neither of its two parents are "natural parents:" 630 is actually a
view, and 200 is a chaining relationship, not a parent/child relationship.

We still think that 244 belongs somewhere under 404, probably as a
grandchild through its ChildOf relationship with 226 (i.e. 244 as a
ChildOf 226, and 226 as a ChildOf 404). 404 itself is a child of 664,
Insufficient Control of a Resource Through its Lifetime, which we view as
a likely "pillar" (top-level node) of the natural hierarchy.

There is another perspective issue to consider. Some might argue that
244, which is specifically about using realloc() to resize buffers, should
fall under CWE-227, API Abuse. This is how it was categorized in Seven
Pernicious Kingdoms, for example. CWE 1.0 will support a Seven Pernicious
Kingroms view (CWE-700, specifically) that will preserve this
relationship. However, we are also trying to figure out how to clearly
define CWE-227 in a way that doesn't effectively include everything that's
a weakness - after all, writing code that allows writing outside of a
memory buffer could distincly be counted as "API abuse" as well. We think
that CWE-227 has a place in the natural hierarchy, but that means that 244
would have multiple natural parents. Currently, we are allowing this to
happen - so "hierarchy" is currently a misnomer.

I hope this addresses your questions and helps to illuminate some of the
issues we face leading up to CWE 1.0.

- Steve

Re: Suggestion of Repositioning CWE #244

2008-07-07T07:15:10Z

Pascal & everyone,

Here is the recommendation we wrote for this rule for the C programming language:

MEM03-A. Clear sensitive information stored in reusable resources returned for reuse

Besides the neat use if alliteration, we list the following examples of reusable resources:

dynamically allocated memory
statically allocated memory
automatically allocated (stack) memory
memory caches
disk
disk caches

thanks,
rCs

ljknews wrote:

At 3:08 PM -0400 7/3/08, koo wrote:

We suggest that CWE #244, Failure to Clear Heap Memory Before Release,

It seems to me that it would be sufficient for the operating
system to clear the memory before reallocation to a process.
Why be concerned about the state when no process can access
it ?

Can you, or should you, as the paranoid secure programmer of an
application, trust the OS to do wipe heap memory before it passes the
memory on to another process or even uses it itself?

Is there a separate item for clearing stack memory ?  That
would seem vulnerable in the same way


There probably should be one, c.f. GCC Mudflap Pointer Debugging, the
-wipe-stack option at http://gcc.gnu.org/wiki/Mudflap_Pointer_Debugging

Koo's suggestion makes sense to me (moving 244).

Cheers,
Pascal

-- 
Robert C. Seacord
Senior Vulnerability Analyst
CERT/CC 

Work: 412-268-7608
FAX: 412-268-6989

Re: Suggestion of Repositioning CWE #244

2008-07-07T05:59:09Z

At 8:31 AM -0400 7/7/08, Pascal Meunier wrote:

> ljknews wrote:
>> At 3:08 PM -0400 7/3/08, koo wrote:
>>
>>> We suggest that CWE #244, Failure to Clear Heap Memory Before Release,
>>
>> It seems to me that it would be sufficient for the operating
>> system to clear the memory before reallocation to a process.
>> Why be concerned about the state when no process can access
>> it ?
>>
> Can you, or should you, as the paranoid secure programmer of an
> application, trust the OS to do wipe heap memory before it passes the
> memory on to another process or even uses it itself?

On the operating system I use, absolutely.
--
Larry Kilgallen

Re: Suggestion of Repositioning CWE #244

2008-07-07T05:28:59Z

ljknews wrote:
> At 3:08 PM -0400 7/3/08, koo wrote:
>
>> We suggest that CWE #244, Failure to Clear Heap Memory Before Release,
>
> It seems to me that it would be sufficient for the operating
> system to clear the memory before reallocation to a process.
> Why be concerned about the state when no process can access
> it ?
>
Can you, or should you, as the paranoid secure programmer of an
application, trust the OS to do wipe heap memory before it passes the
memory on to another process or even uses it itself?

> Is there a separate item for clearing stack memory ? That
> would seem vulnerable in the same way

There probably should be one, c.f. GCC Mudflap Pointer Debugging, the
-wipe-stack option at http://gcc.gnu.org/wiki/Mudflap_Pointer_Debugging

Koo's suggestion makes sense to me (moving 244).

Cheers,
Pascal

Re: Suggestion of Repositioning CWE #244

2008-07-03T13:37:46Z

At 3:08 PM -0400 7/3/08, koo wrote:

> We suggest that CWE #244, Failure to Clear Heap Memory Before Release,

It seems to me that it would be sufficient for the operating
system to clear the memory before reallocation to a process.
Why be concerned about the state when no process can access
it ?

Is there a separate item for clearing stack memory ? That
would seem vulnerable in the same ways.
--
Larry Kilgallen

Suggestion of Repositioning CWE #244

2008-07-03T12:08:50Z

We suggest that CWE #244, Failure to Clear Heap Memory Before Release, also be a child of CWE #226, Sensitive Information Uncleared Before Release. #244 is just the specific case of (mis)using realloc() for sensitive information.

They are both children of CWE #633, Weaknesses that Affect Memory, but don't have any other connection in the CWE that we can see.

For that matter we suggest #244 NOT be a child of #404, Improper Resource Shutdown or Release. #404 is about freeing what you allocate, unlocking what you lock, etc. - it can lead to memory leak or resource leak. #244 talks about sensitive information being exposed because it is not erased from a resource before being released.

For your convenience, here are the URLs and descriptions

http://cwe.mitre.org/data/definitions/244.html

Using realloc() to resize buffers that store sensitive

information can leave the sensitive information exposed to

attack, because it is not removed from memory.

http://cwe.mitre.org/data/definitions/226.html

The software does not fully clear previously used information in

a data structure, file, or other resource, before making that

resource available to a party in another control sphere.

http://cwe.mitre.org/data/definitions/404.html

The program fails to release - or incorrectly releases - a

system resource before it is made available for re-use.

Michael Koo

on Behalf of SAMATE Team

Identifying Perspective Issues in CWE

2008-07-02T13:22:08Z

All,

As we've been developing the natural hierarchy, we've started paying
more attention to the problem in which CWE supports multiple different
perspectives. This is posing challenges for developing the natural
hierarchy, but it is also reflected in how tools map to CWE.

The purpose of this message is to raise awareness about these
challenges, and to solicit feedback from CWE researchers on ways of
handling these.

Here's a live example to work with. A fairly well-known research team
called Security Reason recently released an advisory that maps to a
CWE number:

CVE-2008-2665
http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2008-2665

PHP 5.2.6 posix_access() (posix ext) safe_mode bypass
URL:http://securityreason.com/achievement_securityalert/54

In short, PHP's safe mode feature is a protection mechanism. It is
can limit which OS resources can be accessed by a PHP application,
such as files or potentially dangerous functions. The PHP interpreter
uses an incorrect order of behaviors related to canonicalization,
causing its safe mode check to be insufficient. This could allow an
attacker to conduct path traversal attacks against applications.

Here, Security Reason mapped to CWE-264, which is a Category for
"Permissions, Privileges, and Access Controls." They presumably did
this mapping because of the emphasis on bypassing safe mode - or,
because it's in the NVD Slice, which Bob Martin and I have been
advocating as a reasonable mapping system. At any rate, the "safe
mode" functionality is the LOCATION of the issue - a protection
mechanism intended to provide access control. It is not really
talking about WHAT the issue actually is.

Many people probably would have mapped this same issue to CWE-22 for
path traversal, which is the ultimate consequence. (For those who are
interested in chains, notice how path traversal is the end link in
this case.)

Alternately, you could map to the more generic "protection mechanism
failure" (CWE-693), a new entry, which might be regarded as a
"property of the consequence," or alternately, an intermediate link in
a chain.

Then there's another weakness that people often call the "root cause."
It appears that some inputs are checked against safe mode, but then
those inputs are later canonicalized in a way that generates a result
that bypasses safe mode. This is CWE-180, "Incorrect Behavior Order:
Validate Before Canonicalize." This entry's perspective is largely
based on "code properties" - i.e., it basically involves properties of
code that are effectively independent of the types of security
problems they introduce (Consider CWE-227, API Abuse, as another
example of an entry focused on code properties. The implications of
the API abuse will vary widely depending on the functionality and
assumptions of the API).

So, we have at least 4 potential CWEs that this issue could be mapped
to, because each CWE is written from different perspectives, or
applies to different parts of the chain. This should have obvious
implications for understanding tool capabilities. If one tool maps to
CWE-x, and another tool maps to CWE-y, then it will appear like two
unrelated problems are being covered, when they might just be
different pieces of the same problem.

The question becomes, which mapping should tool vendors or researchers
use, and in which context? Ideally, we would like mapping to be
repeatable, i.e. independent of who's doing the mapping.

Should CWE get rid of CWE-264 entirely, since it's a category?
Absolutely not! It's a very useful way to group things in a way that
reflects how humans think, and essential for some views. And in the
CVE/NVD world where you don't have much information, and you only want
a couple dozen CWE's to map to, it serves its purpose well. But,
neither does CWE-264 belong in the natural hierarchy, because it's not
about a specific type of weakness. So, we're moving it out of the
natural hierarchy (along with other categories), into a new view that
we're informally referring to as a "programming concepts" view.

What about defining guidelines that would map to CWE-22 (path
traversal)? This is probably how most people would map these days.
One could argue that CWE-22 should only be applied in cases where
there's a failure to even TRY to prevent '..' and related sequences.
In this specific case though, in my personal opinion, the most
appropriate mapping is CWE-180, the root cause.

So - even if we establish mapping guidelines that say "stay away from
categories," a mapper would still be looking at a couple different
weaknesses. If we suggest: "stay away from categories AND
consequences," then the mapper would (hopefully) arrive at CWE-180 as
well. This assumes that the person doing the mapping is following
these guidelines, however - which in practice doesn't always happen.

Suggesting that a mapper should "find the root cause" might be more
useful, but that would require a certain mindset that not all mappers
are familiar with, especially if they conduct application
vulnerability research. In addition, tools aren't necessarily focused
on root causes. If we go in this direction, then we might want to
actively discourage mapping to weaknesses that are solely (or
primarily) about consequences, such as NULL pointer dereferences.

Then there's another possible guideline - "map to everything that
matches." This has certain strengths and limitations. It would
effectively support audiences with multiple perspectives, so it would
be useful for people to understand the general contents of a single
tool or repository. But it probably wouldn't work well for a deep
analysis of multiple tools.

The issues I've discussed are illustrative; I don't expect us to come
up with any quick and easy answers.

But, now that we're more specifically aware that we have these perspective
challenges, we are working on identifying and labeling the different
perspectives that are currently being used in CWE. We are currently
examining various tool repositories to provide real-world examples for us
to work with. This effort won't be complete for the release of CWE 1.0,
but I'd like to be able to describe the problem in an understandable way.

Any suggestions or concerns are more than welcome, whether to this list or
to cwe@....

- Steve

Upcoming plans for CWE 1.0

2008-07-01T09:45:42Z

All,

MITRE plans to release CWE 1.0 sometime in August. Here is a summary
of our main goals for that release.

1) Finish existing systemic changes. We want CWE 1.0 to represent a
stable point in CWE's development, which means finalizing the
systemic changes that we've been making over the past year or two.
For this, we are de-prioritizing general "content maintenance" -
i.e., localized modification of individual entries - except as
those modifications might relate to the systemic changes. After
CWE 1.0 is released, we plan to move more into a content
development and refinement mode, in which there will be greater
emphasis on accuracy and completeness of individual entries.

2) Stable schema. We have been making significant schema changes over
the past year, primarily to support our development of views, as
well as the needs of new stakeholders. Our primary goal for CWE
1.0 is to have the schema be stable. We are conducting an internal
review and have outlined the major limitations that still need to
be addressed.

3) Viable views. We have been developing the view concept and
implementation for almost a year now, and we think we finally have
a handle on how we want to support them. So CWE 1.0 will have
multiple views that support different use-cases and stakeholders,
and the schema infrastructure will be in place to support adding
more views without requiring schema modifications.

4) Refinement of the Natural Hierarchy. We have come to realize that
we need to do a better job of communicating what we're trying to
accomplish with the Natural Hierarchy (CWE-1000). In short, we are
attempting to build a classification scheme based on inherent
features of weaknesses of large portions of CWE weaknesses, and
their inter-relationships. My personal hope is that it will take
Seven Pernicious Kingdoms and CLASP one step further. All past
versions of CWE have had multiple ways of grouping weaknesses
together that would lead to difficulty or inconsistency in
performing mappings. It would also be difficult to infer where
knowledge gaps existed. The MITRE team has found that the ongoing
development of the natural hierarchy has helped us significantly in
understanding much of what we have in CWE, and why. Academic
researchers might be especially interested in its development.

Ironically, the natural hierarchy might not seem so "natural" to
regular developers; so, we need to actively support the developer
view. This is one major challenge that we face.

In the coming weeks, we will be releasing a more detailed white
paper to the community on MITRE's goals for the natural hierarchy.
Traces of it exist in CWE Draft 9, but it is far from complete (and
we've since made significant inroads in our 1.0 development). To
get an idea of where we are headed, see: CWE-664 ("Insufficient
Control of a Resource Through its Lifetime"), CWE-682 ("Incorrect
Calculation"), and CWE-691 ("Insufficient Control Flow
Management"). If you are particularly interested in this effort,
then contact us offline and we will give you our current status.

5) More active community engagement. Leading up to CWE 1.0, we will
be actively engaging community members on important issues for CWE.
This discussion list will be one of the main places in which we
solicit feedback. So, this summer is the time to voice any
concerns you have.

6) Resolution of outstanding issues. In the fall of 2007, we brought
up various issues related to CWE content maintenance, including
which types of issues we should include, and what level of
abstraction we should use. We will be actively resolving many of
those issues. See the Working Documents section at
http://cwe.mitre.org/community/workingdocs.html for a refresher.

7) Identifying CWE gaps with respect to current tools, including
guidance for mapping. Several tool vendors have sent us updated
lists of their checks, some of which had CWE mappings. We are
evaluating these mappings to ensure that CWE 1.0 will be able to
support the existing perspectives under which these tools operate.
This might include creating high-level entries that act as
placeholders for future content creation of lower-level entries.
We will not have the time to create new entries for every gap that
we encounter, at least by the release of 1.0, but we will have a
solid understanding of what remains to be done.

Re: Examples of Name and Relationship Changes in CWE Draft 9

2008-04-23T07:26:56Z

Steven M. Christey wrote:
> we changed the names of over 200 entries in Draft 9 alone
...
> 401 Memory Leak
>
> RENAMED: Failure to Release Memory Before Removing Last Reference
> (aka 'Memory Leak')

I like the _idea_ of more-specific-names, but this one isn't quite right
on two counts:

1. Many memory leaks are due to circular structures, e.g., A references
B, and B references A, yet NOTHING refers to either. This _IS_ a memory
leak, but not by this name.

2. Memory leaks only happen if the run-time doesn't support the
necessary kind of garbage collection. Some systems build in
reference-counting collectors, which means you don't need to worry about
releasing memory UNLESS there's a circularity.

I think what you mean is something like
"Failure to Release Memory After It Becomes Unreferenceable"

--- David A. Wheeler

Examples of Name and Relationship Changes in CWE Draft 9

2008-04-21T14:15:21Z

If you examine the difference report at
http://cwe.mitre.org/data/reports/diff_draft_8_9.html , you will see that
we changed the names of over 200 entries in Draft 9 alone, and we added
275 relationships while removing 75. We also changed nearly 200
descriptions.

The main goals were:

- make the CWE name and description more clear about the weakness
being covered, and try to keep the perspective on the weakness
itself, instead of the attack or consequence - but preserve such
terminology if it's commonplace.

- when the CWE is identifying a weakness, try to classify it under
the Natural Hierarchy view (CWE-1000), i.e. it should have a parent
that is a Weakness (Variant, Base, or Class). If a new node is
necessary, create it (or flag the issue for review after Draft 9's
release).

We tried to change the names so that a CWE consumer would not have to
depend so much on looking up the item's description and context notes,
just to figure out what the item was talking about. We tried to
remove perspective problems where feasible, such as when a name was
too focused on the associated attack.

The litmus test for a name change was simple: if a CWE analyst didn't
know what the issue was about upon reading the name, then most CWE
users probably wouldn't know either. As a result, we removed a lot of
non-specific terms such as "insecure," "improper," and "erroneous," or
tried to develop some consistency when we needed to use more general
terms, such as "sanitization" as an over-arching term that could cover
failure to filter, decode, quote, validate, etc.

We didn't identify all the names that needed fixing, but 37% of CWE
entries were modified, so this was a solid start.

We definitely didn't identify the natural parents for every entry,
although this effort did produce many of the new entries that were
added to Draft 9. We expect this to be an ongoing process. See the
CWE-1000 definition for additional explanation of the natural
hierarchy.

Below are a few examples of the name changes, along with relationships
that we added, to give people a sense of what we did and why.

--------------------------------------------------------

582: Mobile Code: Unsafe Array Declaration

- what's unsafe about it - is this permissions? buffer overflow?
something else?

New name: Array Declared Public, Final, and Static

--------------------------------------------------------

568: Erroneous Finalize Method

- what's the error? does the software define the finalize method
incorrectly? is this permissions? Does the method do too much?
too little? operates on the wrong object? has a memory leak?
sends private data?

New name: finalize() Method Without super.finalize()

Old parent: 399: Resource Management Errors

New parents:

573 - Failure to Follow Specification
404 - Improper Resource Shutdown or Release

--------------------------------------------------------

73: Path Manipulation

- name is attack-focused

- how is it being manipulated - symbolic link? long pathname? path
traversal? appending "%20" to retrive source code?

- first code example is path traversal (CWE-22)

- second code example may or may not be path traversal

- RENAMED: "External Control of File Name or Path"

- ADDED ChildOf 99 Insufficient Control of Resource Identifiers (aka
'Resource Injection')

--------------------------------------------------------

4: J2EE Environment Issues

This is a general category node whose name is self-explanatory. In
draft 8, however, its children rarely had any natural parents.

child: 5 J2EE Misconfiguration: Insecure Transport

RENAMED: J2EE Misconfiguration: Data Transmission Without Encryption

ADDED: ChildOf 311 Failure to Encrypt Sensitive Data

child: 555 J2EE Misconfiguration: Password in Configuration File

RENAMED: J2EE Misconfiguration: Plaintext Password in
Configuration File

ADDED: ChildOf 522 Insufficiently Protected Credentials

DESCRIPTION: modified

child: 6 J2EE Misconfiguration: Insufficient Session-ID Length

ADDED: ChildOf 334 Small Space of Random Values

child: 7 J2EE Misconfiguration: Missing Error Handling

Unchanged

child: 8 J2EE Misconfiguration: Entity Bean Declared Remote

ADDED: ChildOf 668 Exposure of Resource to Wrong Sphere

child: 9 J2EE Misconfiguration: Weak Access Permissions

RENAMED: J2EE Misconfiguration: Weak Access Permissions for EJB Methods

ADDED: ChildOf 275 Permission Issues

--------------------------------------------------------

597 Erroneous String Compare

- what's the error - only a portion of the string is compared? It
compares a string in a case-insensitive manner? It doesn't handle
when one string is shorter than the other?

RENAMED: Use of Wrong Operator in String Comparison

--------------------------------------------------------

591 Memory Locking

- is this about not locking memory? Locking it incorrectly? is
this a category of all different types of weaknesses that can
occur during memory locking?

RENAMED: Sensitive Data Storage in Improperly Locked Memory

--------------------------------------------------------

590 Improperly Freeing Heap Memory

- does this mean double free? running free() on an object that was
allocated using new() ?

RENAMED: Free of Invalid Pointer Not on the Heap

--------------------------------------------------------

560 Often Misused: umask()

- is this about setting an insecure umask? Not specifying a umask
and using one that you've inherited from the caller of your
program?

RENAMED: Use of umask() with chmod-style Argument

FORMER PARENT: 559 "Often Misused: Arguments and Parameters"

New parent: 687 Function Call With Incorrectly Specified Argument Value

--------------------------------------------------------

474 Inconsistent Implementations

- is this about things like how web browsers can behave differently?

RENAMED: Use of Function with Inconsistent Implementations

--------------------------------------------------------

401 Memory Leak

RENAMED: Failure to Release Memory Before Removing Last Reference
(aka 'Memory Leak')

Natural parents: none in draft 8

ADDED: ChildOf 404 Improper Resource Shutdown or Release

CWE Draft 9 Major Schema Changes, and Outstanding Issues

2008-04-16T14:16:41Z

All,

We've significantly modified the schema for Draft 9. The primary
driver was to improve support for multiple views, and to better
distinguish between the different types of elements that we are
covering in CWE. Thanks to Sean Barnum for figuring out the bulk of
this. The MITRE team took his inputs and made some small tweaks here
and there.

As CWE is at a crossroads with respect to the schema, we welcome any
feedback or alternatives to our current approaches. Specifically,
while we have chosen XML so far, we are open to leveraging other
techniques to storing and working with the data, if those techniques
are more effective. For example, if it makes sense to store CWE in a
database and use an application server to help present and link
everything together, we are open to pursuing that. We also plan to
investigate RDF, XGGML, and other languages that might be more
directly supportive of graph-based relationships.

Please note that even if we stay with XML and related technologies, we
expect that the schema will still need to change a little bit.
However, we believe that one requirement for "CWE 1.0" is to have
stable schema. In Draft 9, we are definitely a lot closer than we
were. Bob and I will post our requirements for "CWE 1.0" once they've
been finalized.

For Draft 9, some of the highest level schema changes are covered
here:

http://cwe.mitre.org/data/reports/diff_xsd_10_3.0.html

The rest of this document assumes that you read the preceding
document.

Any and all feedback would be appreciated, especially if there are
still outstanding issues in the schema that prevent you from using CWE
as extensively as you would want to.

Schema Evaluation Criteria
--------------------------

Here are some of the criteria that I think we should be applying while
finalizing the schema:

- Expressiveness: we should be able to express everything that we
want to. In Draft 9, some examples of this are the creation of
explicit views, and the requirement for relationships to specify
the views they are part of. But, we still don't have a way of
saying things like "this issue theoretically affects any language
that performs direct memory management, but it's especially common
in C." That's important, because if C is not explicitly mentioned
in an element, then that element won't be part of the C language
view.

- Extraction: it should be as easy as possible for CWE users to
extract the data that they want, using commonly available XML
parsers and related tools. In Draft 9, the relevant data for
named chains are not necessarily easy to extract.

- Maintenance

- minimize maintenance costs: the MITRE team, and outside
contributors, should be able to quickly represent the necessary
information.

- minimize preventable errors in data entry: we want to minimize
errors in the CWE representation that cannot be caught by an XML
validator, but nonetheless require consistency.

- minimize XML "bloat": this is hopefully self-explanatory. The
relationships in Draft 9 might exhibit some bloat, although at
the same time, there's a major benefit to their increased
expressiveness.

- Flexibility: ideally, the schema would remain stable, while
allowing us to build in additional capabilities. For Draft 9, we
believe that we've added flexibility for defining new kinds of
relationships and views. The introduction of compound elements
will hopefully allow us to support other kinds of concepts besides
chains and composites that might arise in the future; for example,
some CWE nodes are really talking about multiple distinct issues
and could be called "loose composites."

In light of these criteria, I wanted to explain some of the rationale
for the schema changes, and what we have left ahead of us for CWE 1.0.

Views
-----

We added a number of views to CWE Draft 9. For the most part, this
involved converting weakness/"groupings" from Draft 8, into the new
Views type for draft 9.

See http://cwe.mitre.org/data/index.html for a list of views.

Slices are basically lists of elements, without any relationships
between them. Membership in a slice can be explicit or implicit. In
explicit slices, all the relevant entries have some ChildOf
relationship where the View node is the parent; see CWE-630
(Weaknesses Examined by SAMATE) and CWE-635 (Weaknesses Used by NVD)
for examples.

In implicit slices, the slice has some filtering criteria that define
membership, and there aren't any relationships within the XML that are
explicitly defined. For example, CWE-658 is a slice that covers
weaknesses found in the C language. This implicit slice has a Filter
that specifies that member entries have "C" under the
Applicable_Platforms field.

The Comprehensive CWE Dictionary view, CWE-2000, is actually an
implicit slice that selects everything from CWE by using a filter that
always returns true.

Views can also be graphs, such as CWE-1000 (Natural Hierarchy).
Currently, graphs are expected to have explicit ChildOf relationships
within the member elements. Before Draft 9, everything was
effectively under the Natural Hierarchy. In Draft 9, however, some of
those elements have been removed from the Natural Hierarchy
altogether, like deprecated nodes and the resource-based view.

We suspect that some individual views might be best described as a
combination of slices *and* graphs, with a combination of implicit or
explicit membership. A view might be best expressed via some set of
explicit relationships (maybe between some implicit slices), then
defaulting to the relationships of a different view at some point.

The most concrete example of this is CWE-631 (Resource-specific
Weaknesses), at:

http://cwe.mitre.org/data/graphs/631.html

The higher-level nodes have explicit relationships defined within View
631. Its children - such as the Category node CWE-632 (Weaknesses
that Affect Files or Directories) - have explicitly specified children
such as CWE-22 (Path Traversal). That is, Path Traversal has an
explicit "ChildOf CWE-632" relationship. However, instead of the
explicit relationships, CWE-632 could potentially be defined as an
implicit slice of "all elements that have an Affected_Resource field
of File/Directory." That would reduce maintenance costs and improve
accuracy, but it is not possible in Draft 9, because CWE-632 is a
Category type - it's *in* a view, but not a view itself.

In addition, the resource-based view, CWE-631, could be more
comprehensive by "view hopping." In Draft 9, CWE-631 stops at CWE-22
(Path Traversal), but there are several children under CWE-22 that
would also match - except those children are only listed under the
natural hierarchy (view CWE-1000). It would probably be quite tedious
and error-prone just to copy all the natural hierarchy relationships
over to this new view. This might be best handled by allowing views
to link to each other, but this is not possible in Draft 9. In
addition, the "hops" might wind up including elements that were not
intended.

Finally, we have encountered some difficulties in generating a
"Comprehensive Graph" that merges all views together - the natural
hierarchy, the resource-based graph, the language-specific slices,
etc. So, there isn't a single graph on the CWE web site that covers
the entire CWE. We do have a PDF file that contains most nodes; it
focuses on the natural hierarchy (CWE-1000), and all other nodes are
effectively "orphans." We don't necessarily have to solve this
problem for a comprehensive view - after all, it's not clear who would
have a need for such a thing - but I thought it was worthwhile to
mention.

Relationships
-------------

The expression of relationships has changed significantly for Draft 9.
Much of this is covered by the schema diff report listed at the top of
this document, but there are some fields that I wanted to highlight.

Relationship_Type:

The Draft 8 version of "Relationship_Type" has been renamed to
"Relationship_Nature". The Draft 9 version of this field is
intended to identify the type of the entry that is being linked to.
Since we now have multiple types of entries in CWE, this field might
be useful in simplifying some extraction and presentation logic for
XSLT's. We have not needed this field in generating the web site
for Draft 9, although it might be convenient for others. However,
this field is currently being manually maintained, and this value
was often incorrect, because we changed the types of a number of
elements in Draft 9, which immediately invalidated this field in
dozens of relationships. We are able to perform a consistency check
to ensure that these values are correct before release, but it's
still a little bit of labor.

As a result, we will be looking at this field more closely, trying
to balance utility to the community with maintenance costs to the
CWE team.

Relationship_View_IDs:

We anticipate that, in the future, we will have multiple views that
share a lot of the same structure. As one example - CWE's Natural
Hierarchy (CWE-1000) is beginning to diverge more from the Seven
Pernicious Kingdoms (SPK) way of organizing the world, so it might
be reasonable to create a view into CWE that's useful for people who
are knowledgeable about SPK. The Natural Hierarchy and an SPK view
would probably have a lot of different elements near the top of the
tree, but they would share a lot at a lower level.

With closely overlapping views, this would produce a large number of
duplicate relationships that might contribute significantly to XML
bloat. The MITRE team decided that allowing multiple
Relationship_View_IDs would be a useful shorthand that might be
easier to maintain.

Current Challenges
------------------

Here are some of the current challenges that we still face, and plan
to resolve by CWE 1.0.

1) The Draft 9 schema does not have the expressiveness to define the
more complex views, and there are some associated maintenance
costs, as outlined in the previous sections.

2) Chains and composites, views, and categories all have some
overlapping uses that we'd like to clarify and, to the degree
possible, unify.

For example, both chains and composites involve a small selection
of entries from CWE, and dictate relationships between them. In
this sense, they can be regarded as views - perhaps micro-views.
Yet, we expect that they will have a distinct and important role
throughout CWE.

As another example, the resource-based view (CWE-632) has children
that are categories. These categories might be best described by
defining what their membership should be, but in Draft 9, this type
of automatic population is only possible through filters in View
elements. So, we had to manually create ChildOf relationships.

3) Relationship Directionality

Some views, like CWE-635 (Weaknesses Used by NVD), are defined more
by external criteria than anything that is implicit within
individual nodes, so these are explicit slices. In terms of
maintenance costs and ease of extraction, it might be best for
CWE-635 to explicitly state what its "members" are. Instead, each
member has a ChildOf relationship, with View_ID=635, that is a
ChildOf 635. Thus, maintenance of the NVD slice is done not by
operating on the slice itself, but by operating on its individual
members. This proved to be moderately expensive for us to do when
we changed the membership of the SAMATE view in Draft 8 - it took
an hour or so to edit some nodes to remove the SAMATE relationship,
and then edit other nodes to add the SAMATE relationship; if we
could just edit the SAMATE list directly, it would have been a
5-minute task. However, as I understand it, one of the mantras of
knowledge management is that data is kept as close to individual
nodes as possible; but relationships "belong" to multiple nodes,
even though in Draft 9 they are only explicit in one node.

It would be possible for us to automate some of those maintenance
tasks, but that would involve additional development.

Also, we have multiple relationships that are mutual, but only
expressed in one direction. For example, "X ChildOf Y" might be
specified in the XML, which implies "Y ParentOf X" - but we have no
ParentOf relationships that are explicitly stated. The same thing
applies for relationships that support chains and composites. As a
result, extraction logic can be complicated, because an entry
doesn't explicitly know what its children are. As a result of this
complexity, the extraction logic can be hard to maintain, and
sometimes computationally expensive. We have encountered this
problem in various ways while generating web site pages.

One possibility would be to create separate XML files and
representations for the relationships (and maybe for views),
possibly with separate schema. This might preserve expressiveness
and simplify maintenance, but it might make it more difficult for
some people to extract.

4) Named Chains

There are a couple issues with named chains. See
http://cwe.mitre.org/data/reports/chains_and_composites.html for
background.

All the data that's required to determine the links of a named
chain are within the XML, so there is sufficient expressiveness.
However, extraction is a little more difficult. For a named chain
X, the code has to search throughout all of CWE for entries with
all the CanPrecede relationships with a Chain_ID of X, then order
them appropriately. If you want to know what elements are in a
named chain, you HAVE to do this navigation throughout CWE - a
named chain does not explicitly state what its links are. Just
like composites explicitly state which items they require, it might
be reasonable to have named chains explicitly know what their
starting links are.

In addition, named chains can be difficult to classify, especially
under the natural hierarchy. Because named chains are new, we
decided not to create a separate view to handle them. We do have a
view that lists chain elements (CWE-679), but that view is actually
an implicit slice for extracting components of all the CanPrecede
relationships, whether they're related to a Named Chain or not.

The extraction and presentation logic for presenting chains in
general was too complicated for us to handle cleanly by the release
of Draft 9, so they are generated by external programs, instead of
through XSLT.

Release of Draft 9

2008-04-14T14:16:02Z

All,

Just in case some of the researchers are not subscribed to the CWE
announce list, following is the announcement of Draft 9. We have
accomplished a lot in this draft, and I hope you will agree.

In the next day or so, I'll post some more extensive discussion of some of
the schema changes and some outstanding issues, and I'll discuss some of
the content changes as well.

If you have any questions or concerns, please notify the CWE team, or post
to this list. We do want to have some active discussion on this list, and
the transition between Draft 9 and "CWE 1.0" will provide ample
opportunity.

- Steve

=============

The ninth draft of CWE has been posted on the CWE List page. It
includes several important changes. A report is available that
lists specific differences between Draft 8 and Draft 9.

Specific changes for Draft 9 include: significant schema changes
to better distinguish and link between Weaknesses, Categories,
Views, Chains, and Composites; 39 new entries, many of which
improve the organization of CWE; changes to the names or
descriptions of over 200 entries, in order to more accurately
reflect each entry; modification of relationships related to
classification under a "natural hierarchy" view; addition of a
Status field to reflect the maturity of each entry; an updated
report on prioritization of fields; the introduction of named
chains; and many other changes affecting over 450 entries.

RE: Patr Traversal and some challenges for CWE

2007-11-20T06:32:04Z

Wagoner, Larry wrote:

>Therefore, the "real" leaf nodes in CWE should be "allows unvalidated
input"
>"language that does not do array bounds checking" "return addresses can
be modified"

You can get even more precise than this. "Allows unvalidated input"
could be "allows unvalidated input by input size" or many other flavors
of input validation that all protect against different weaknesses in the
code. The "return address can be modified" is really just one of many
ways a buffer overflow can be exploited at a higher severity than just
overwriting data. "function pointers can be modified" or "heap
structures can be modified" are other ways. I like this level of
precision because it helps language and platform designers understand
precisely what the mechanisms of insecure software are.

In a subsequent post Dave McKinney made an argument against lower levels
of classification because it would go lower level than needed by the
people who need to understand and manage fixing vulnerable code. This
seems to me a split akin to atoms and sub-atomic particles. One has
chemical properties and can exist as a molecule or be combined with
other atoms to make bigger molecules. Chemists work at this level.
Physicists try to understand the way the universe works at a lower level
and work with things that really don't exist in the "real world". We
just need to figure out where the split is. Something like chemists
(sofware developers)use CWE and physcists (language and platform
designers) use X.

-Chris

-----Original Message-----
From: owner-cwe-research-list@...
[mailto:owner-cwe-research-list@...] On Behalf Of Wagoner,
Larry D.
Sent: Thursday, November 15, 2007 9:49 AM
To: cwe-research-list@...
Cc: Matt Bishop; Jarzombek, Joe
Subject: RE: Patr Traversal and some challenges for CWE

I think your chains have to be made at a consistent lower level. For
instance, a buffer overflow is not a fundamental weakness, but a
combination of several weaknesses that together allow a system to be
comprimised. This is an idea that has been around for quite a while
(see Matt Bishop's paper "Vulnerability Analysis"
(http://nob.cs.ucdavis.edu/bishop/papers/1999-raid/similar ) and has
been talked about lately in the Intro. to Vulnerability Theory paper.
That is, a buffer overflow, integer coercion, path traversal, etc. are
instantiations of a combination of weaknesses or primitive flaws.
Frequently if one of the underlying flaws are removed, then the weakness
(as currently in CWE) is no longer a vulnerability. For example, if
some of the components of the buffer overflow weakness are eliminated
such as if one does not have unvalidated input, a language that doesn't
do bounds checking, and a system that allows return addresses to be
modified, then a buffer overflow cannot occur. Therefore, the "real"
leaf nodes in CWE should be "allows unvalidated input" "language that
does not do array bounds checking" "return addresses can be modified"
(no doubt better names could be created). Buffer overflow would be a
parent to these three nodes. And tying this to the view strategy that
CWE wants to move to, then under a Java or Pascal view, the leaf node
"language that does not do array bounds checking" would not appear in
that view, but "allows unvalidated input" would. Path traversal would
also be a parent of "allows unvalidated input." Integer coercion would
be very interesting to reduce to its fundamental components -- such as
"allows data casting," "allows unsigned data," etc.

Those fundamental entries and the links to them would solve several
problems:
1. Would allow tools to make claims about the real weaknesses.
Languages or even hardware platforms could also make claims ("Wow, look
at how much of the CWE tree disappears when you use Acme") 2. Would make
generation of views easier -- any node that is a parent to "allows
unsigned data" could be eliminated a Java view 3. Would reduce (likely
not eliminate, unfortunately) the problem of one tool calling it one
weakness and another tool calling it another weakness and both being
argubly right -- this would put things at a more fundamental level where
things are, at the least, a little more definitive 4. Would address the
real problem at the lowest level possible 5. Would likely expose other
weaknesses

Larry

<quote author="Steven M. Christey-2">
On Wed, 14 Nov 2007, Chris Telfer wrote:

> There are usually a huge number of different ways for a programmer to
> produce the same type of weakness. We already acknowledge this in
some
> part through the notion of "complexities" that color a flaw (ala
> SAMATE). I believe that different ways to express the same weakness
> (e.g. '/' vs '\\' as a path separator) are simply "complexities" for a

> weakness that don't apply generally across all weakness types.

Lately, Bob Martin and I have begun exploring certain concepts of
"chains"
of weaknesses, as well as "composites" of properties whose combination
forms a weakness.

An increasingly-reported "chain" would be an integer overflow that
causes insufficient allocation of memory, which then leads to a buffer
overflow.
A second chain involging integer overflow would lead to an infinite
loop, and could be independent of memory allocation altogether (we see
examples of this in CVE).

The "weakness that is better known by the attack term 'Symbolic Link
Following'" can be thought of as a composite in which one needs
predictability (of filenames), a race condition, path equivalence (where
two paths ultimately point to the same file object), and a shared
directory to exist all at once, in order to be present. Removing any
one of these elements would prevent the issue or, at worst,
significantly reduce its scope.

Many of the CWE path traversal variants are probably reflections of
different chains and/or composites. This gets closer to understanding
the variety of ways for programmers to cause the same problem.

Note: Bob and I are only in the early stages of developing these
concepts.

- Steve

</quote>

Re: Patr Traversal and some challenges for CWE

2007-11-15T09:28:52Z

On Thu, Nov 15, 2007 at 09:48:55AM -0500, Wagoner, Larry D. wrote:

>
> I think your chains have to be made at a consistent lower level. For
> instance, a buffer overflow is not a fundamental weakness, but a
> combination of several weaknesses that together allow a system to be
> comprimised. This is an idea that has been around for quite a while
> (see Matt Bishop's paper "Vulnerability Analysis"
> (http://nob.cs.ucdavis.edu/bishop/papers/1999-raid/similar ) and has
> been talked about lately in the Intro. to Vulnerability Theory paper.
> That is, a buffer overflow, integer coercion, path traversal, etc. are
> instantiations of a combination of weaknesses or primitive flaws.
> Frequently if one of the underlying flaws are removed, then the weakness
> (as currently in CWE) is no longer a vulnerability. For example, if
> some of the components of the buffer overflow weakness are eliminated
> such as if one does not have unvalidated input, a language that doesn't
> do bounds checking, and a system that allows return addresses to be
> modified, then a buffer overflow cannot occur. Therefore, the "real"
> leaf nodes in CWE should be "allows unvalidated input" "language that
> does not do array bounds checking" "return addresses can be modified"
> (no doubt better names could be created). Buffer overflow would be a
> parent to these three nodes. And tying this to the view strategy that
> CWE wants to move to, then under a Java or Pascal view, the leaf node
> "language that does not do array bounds checking" would not appear in
> that view, but "allows unvalidated input" would. Path traversal would
> also be a parent of "allows unvalidated input." Integer coercion would
> be very interesting to reduce to its fundamental components -- such as
> "allows data casting," "allows unsigned data," etc.

While I think there is a real need for a more academically rigorous approach
to creating a vulnerability taxonomy, I think the current level of
abstraction used by the CWE is adequate for the audience and its
intended users. For example, describing something as a cross-site
scripting vulnerability or a sub-set of XSS is probably the right
level for tool vendors, security researchers, software vendors,
non-technical managers, the media, etc.

We can understand and agree that a vulnerability can be broken down
further into primitives that must exist in tandem for
the vulnerability to occur. However, my fear is that this may
needlessly complicate the description and classification of simple vulnerabilities.

I am also concerned of the possibility of arriving at a system where many
of the classifications are not actually security weaknesses in and of
themselves, which may make many of the entries irrelevant for
practically classifying vulnerabilities. Like for example, you have a
vulnerability that occurs because of one legit security weakness
combined with nine other environmental or programming
language-specific entries which are not weaknesses in and of
themselves. I am also concerned about overlap with other SCAP related
standards if the CWE is addressing a lot of configuration or
environmental factors that cause a vulnerability to exist in one
installation where it wouldn't exist in another.

My understanding of what Steve was suggesting in terms of combining
vulnerabilities is that there will be some cases where the system will
need to be robust enough to describe vulnerabilities that exist only
where certain weaknesses are combined in tandem. There are also some
conditions that are best described in terms of interactions, like an
integer handling issue that lets the attacker influence a memory copy operation
further down the execution chain in a way that the developer did not
intend. It may be useful to describe this in terms of multiple
weaknesses working in tandem.

--
Dave McKinney
Symantec

keyID: E461AE4E
key fingerprint = F1FC 9073 09FA F0C7 500D D7EB E985 FAF3 E461 AE4E