Nabble - Apache XML - Security - Dev

RE: Undeclared namespace prefix

2008-10-05T21:16:54Z

> Granted that it's insane (and I appreciate your opinion about this) is it
> *documented* insanity, or is it defined by implementation? From my limited
> reading, it seems to be a very grey area. The c14n documentation doesn't
> mention the DOM at all, but there may be some documentation on the DOM
side
> that I'm unaware of.

It's not documented any *other* way, that's kind of the only way to say it.
If you follow the text, you find that when handing a c14n engine a DOM
element (and an implied node set of everything below it), nothing in the
spec says to create a namespace declaration if one isn't already present.
The lone exception is the InclusivePrefixes list.

So if it's not declared, it won't get declared, and the XML that results
won't be well-formed (and therefore probably isn't what you meant to sign).

Serializers are not the same as a c14n engine, and it isn't typical to use
one before you sign a DOM, so using serializers to "fix up" namespaces is
really not part of a robust signing application.

-- Scott

RE: Undeclared namespace prefix

2008-10-05T16:15:32Z

Scott Cantor wrote:

> Aside from the embarrassment though, it remains the case that the
> serializer does not need the assistance of setAttributeNS; it does this
> job itself.

There's no such thing as "the serializer". There are hundreds of things called "serializers". DOM3 includes an LS interface that includes "a serializer", but that's just one example.

Every serializer has its own properties and behaviors, usually close to undocumented, so relying on that isn't robust.

Furthermore, signing takes place in many cases over a DOM, not a serialized document. It is not tenable to create the DOM, serialize it, parse it back in, and then sign it. So you have to ensure namespace correctness in the DOM itself using your own code (or a toolkit designed to address that).

The point is, you can't punt this in the manner you described. You may find that insane, but I find that most people conclude that about XML pretty quickly.

-- Scott

Granted that it's insane (and I appreciate your opinion about this) is it *documented* insanity, or is it defined by implementation? From my limited reading, it seems to be a very grey area. The c14n documentation doesn't mention the DOM at all, but there may be some documentation on the DOM side that I'm unaware of.

RE: Undeclared namespace prefix

2008-10-05T14:42:07Z

> Aside from the embarrassment though, it remains the case that the
> serializer does not need the assistance of setAttributeNS; it does this
> job itself.

There's no such thing as "the serializer". There are hundreds of things called "serializers". DOM3 includes an LS interface that includes "a serializer", but that's just one example.

Every serializer has its own properties and behaviors, usually close to undocumented, so relying on that isn't robust.

Furthermore, signing takes place in many cases over a DOM, not a serialized document. It is not tenable to create the DOM, serialize it, parse it back in, and then sign it. So you have to ensure namespace correctness in the DOM itself using your own code (or a toolkit designed to address that).

The point is, you can't punt this in the manner you described. You may find that insane, but I find that most people conclude that about XML pretty quickly.

-- Scott

Re: Undeclared namespace prefix

2008-10-05T08:00:49Z

Peter B. West schrieb:
<SNIP ---- SNAP>

>
> Werner,
>
> It helped a great deal. I'm embarrassed at my misreading of the API.
>
> I forgot about the xmlns *namespace* itself, possibly because
> createElementNS has no need for its explicit expression.
>
> Aside from the embarrassment though, it remains the case that the
> serializer does not need the assistance of setAttributeNS; it does this
> job itself. Is that unreasonable? What other interpretation of
> createElementNS is possible?

I have no idea what a serializer does - however, the c14n serializer
does not include the xmlns:ds="...." attribute if it is not present
at the element. If you the try to parse the resulting XML doc the
parser complains regarding "namespace prefix not bound" (or something
simliar).

See the discussins I hat with Scott and Raul because of a modification
in xmlsec-1.4.2 where the KeyInfo object (element) was modified to omit
the namespace declaration for the ds: prefix. Which resulted in problems.

Regards,
Werner

>
> Peter
>
>

Re: Undeclared namespace prefix

2008-10-05T03:42:16Z

Werner Dittmann wrote:

> Peter B. West schrieb:
> <SNIP ----- SNAP>
>>> you're using namespaces.
>>>
>>
>> setAttributeNS doesn't work for me following createElementNS. I get
>> org.w3c.dom.DOMException: NAMESPACE_ERR: An attempt is made to create or
>> change an object in a way which is incorrect with regard to namespaces.
>>
>> This makes sense, but means that the only way to get the xmlns attribute
>> expressed is to normalizeDocument(). My feeling is that this reinforces
>> my point about the expected behaviour of the serializer.
>>
>> Does it work for you? Maybe I'm doing something wrong.
>>
>
> Here a code snippet how this set attribute stuff works (at least for me
> :-) ),
> also some explanations why the namespace stuff works that way in DOM.
>
> As an example the following element:
> - a Reference element that is part of some OASIS WSS namespace (not the
> same as
> Reference in Signature)
>
> - the element also contains an attribute in another WSS namespace
>
> Example:
> <wsse:Reference
> xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd"
>
> wsu:Id="someIdentifier"
> xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">
>
> ...
> </wsse:Reference>
>
>
> Create the element first:
>
> Element element = doc.createElementNS(WSConstants.WSSE_NS,
> "wsse:Reference");
>
> this creates a namespace qualified element whose namespace is
> "http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd"
>
> (this is the value of the string constant "WSConstants.WSSE_NS").
>
> Please note that wsse:Reference is the element's qualified name
> (Qname), which
> may consist of "prefix:local name" or "local name" alone (refer to W3C
> namespace
> specification).
>
> Why using the namespace URI when creating the element? Refer to the
> following
> sentences given in W3C's namespace spec:
>
> <quote>
> The Prefix provides the namespace prefix part of the qualified name,
> and MUST be
> associated with a namespace URI reference in a namespace declaration.
>
> Note that the prefix functions only as a placeholder for a namespace name.
> Applications SHOULD use the namespace name, not the prefix, in
> constructing
> names whose scope extends beyond the containing document.
> </quote>
>
> This says: the prefix is a convenience (IMHO meant for "humans", not
> necessarily for
> computers :-) ). The real namespace is attached to the element.
>
> Next step is to bind the prefix with a namespace using a namespace
> declaration:
>
> element.setAttributeNS(WSConstants.XMLNS_NS, "xmlns:wsse",
> WSConstants.WSSE_NS);
>
> The namespace declaration attribute:
>
> - "WSConstants.XMLNS_NS" is the string
> "http://www.w3.org/XML/1998/namespace" which
> is fixed and MUST be given this way. This creates an namespace
> qualified attribute
> whose namespace is "http://www.w3.org/XML/1998/namespace".
>
> - "xmlns:wsse" is the name of the attribute. Only the substring "xmlns:"
> defines this
> attribute as a namespace declaration
>
> - the namespace URI of the element, for example
>
> "http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd"
>
> in case of Web Service Security extensions.
>
> Now the namespace qualified attribute of the above element:
>
> element.setAttributeNS(WSConstants.WSU_NS, wsu:Id", id);
> element.setAttributeNS(WSConstants.XMLNS_NS, "xmlns:wsu",
> WSConstants.WSU_NS);
>
> Hope this helped a bit :-) .
>
> Regards,
> Werner

Werner,

It helped a great deal. I'm embarrassed at my misreading of the API.

I forgot about the xmlns *namespace* itself, possibly because
createElementNS has no need for its explicit expression.

Aside from the embarrassment though, it remains the case that the
serializer does not need the assistance of setAttributeNS; it does this
job itself. Is that unreasonable? What other interpretation of
createElementNS is possible?

Peter

>
>>>> <my:name xmlns:my="a:b:c" xmlns:my="a:b:b"/>
>>> I'm pretty sure that's because of your bug, using a DOM1 call to
>>> create that
>>> initial attribute.
>>>
>>>> I'm not sure when this arrangement of the namespace and attribute
>>>> occurs,
>>>> but the ordering may be affected by the order of creation -
>>> createElementNS
>>>> first, followed by the attribute setup. In any case, I don't know
>>>> what the
>>>> canonicalizer would do with this. Neither do I know which xmlns:my
>>>> would
>>> win
>>>> when the stream is read next time.
>>> Neither will because that XML isn't well-formed. It won't parse.
>>>
>>> Try using setAttributeNS and I think you'll have more luck.
>>>
>>> -- Scott

--
Peter B. West <http://cv.pbw.id.au/>
Folio <http://defoe.sourceforge.net/folio/>

Re: Undeclared namespace prefix

2008-10-05T00:41:40Z

Peter B. West schrieb:
<SNIP ----- SNAP>

>> you're using namespaces.
>>
>
> setAttributeNS doesn't work for me following createElementNS. I get
> org.w3c.dom.DOMException: NAMESPACE_ERR: An attempt is made to create or
> change an object in a way which is incorrect with regard to namespaces.
>
> This makes sense, but means that the only way to get the xmlns attribute
> expressed is to normalizeDocument(). My feeling is that this reinforces
> my point about the expected behaviour of the serializer.
>
> Does it work for you? Maybe I'm doing something wrong.
>

Here a code snippet how this set attribute stuff works (at least for me :-) ),
also some explanations why the namespace stuff works that way in DOM.

As an example the following element:
- a Reference element that is part of some OASIS WSS namespace (not the same as
Reference in Signature)

- the element also contains an attribute in another WSS namespace

Example:
<wsse:Reference xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd"
wsu:Id="someIdentifier" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">
...
</wsse:Reference>

Create the element first:

Element element = doc.createElementNS(WSConstants.WSSE_NS, "wsse:Reference");

this creates a namespace qualified element whose namespace is
"http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd"
(this is the value of the string constant "WSConstants.WSSE_NS").

Please note that wsse:Reference is the element's qualified name (Qname), which
may consist of "prefix:local name" or "local name" alone (refer to W3C namespace
specification).

Why using the namespace URI when creating the element? Refer to the following
sentences given in W3C's namespace spec:

<quote>
The Prefix provides the namespace prefix part of the qualified name, and MUST be
associated with a namespace URI reference in a namespace declaration.

Note that the prefix functions only as a placeholder for a namespace name.
Applications SHOULD use the namespace name, not the prefix, in constructing
names whose scope extends beyond the containing document.
</quote>

This says: the prefix is a convenience (IMHO meant for "humans", not necessarily for
computers :-) ). The real namespace is attached to the element.

Next step is to bind the prefix with a namespace using a namespace declaration:

element.setAttributeNS(WSConstants.XMLNS_NS, "xmlns:wsse", WSConstants.WSSE_NS);

The namespace declaration attribute:

- "WSConstants.XMLNS_NS" is the string "http://www.w3.org/XML/1998/namespace" which
is fixed and MUST be given this way. This creates an namespace qualified attribute
whose namespace is "http://www.w3.org/XML/1998/namespace".

- "xmlns:wsse" is the name of the attribute. Only the substring "xmlns:" defines this
attribute as a namespace declaration

- the namespace URI of the element, for example
"http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd"
in case of Web Service Security extensions.

Now the namespace qualified attribute of the above element:

element.setAttributeNS(WSConstants.WSU_NS, wsu:Id", id);
element.setAttributeNS(WSConstants.XMLNS_NS, "xmlns:wsu", WSConstants.WSU_NS);

Hope this helped a bit :-) .

Regards,
Werner

>>> <my:name xmlns:my="a:b:c" xmlns:my="a:b:b"/>
>> I'm pretty sure that's because of your bug, using a DOM1 call to create that
>> initial attribute.
>>
>>> I'm not sure when this arrangement of the namespace and attribute occurs,
>>> but the ordering may be affected by the order of creation -
>> createElementNS
>>> first, followed by the attribute setup. In any case, I don't know what the
>>> canonicalizer would do with this. Neither do I know which xmlns:my would
>> win
>>> when the stream is read next time.
>> Neither will because that XML isn't well-formed. It won't parse.
>>
>> Try using setAttributeNS and I think you'll have more luck.
>>
>> -- Scott
>

Re: Undeclared namespace prefix

2008-10-04T18:27:11Z

Scott Cantor wrote:
>> I am uncertain about the underlying assumptions here. Please correct me
>> where I am wrong in what follows.
>
> I have to warn you, I'm not a c14n expert, and I know approximately zero
> about XPath. What I'm knowledgable about are the practical aspects of using
> a DOM and signing.
>

And I know nothing about c14n, DOM or signing, and a little about XPath.

>> When I construct a DOM with default configuration using createElementNS,
> the
>> elements have an associated prefix, namespace and local name. I assume
> such
>> a DOM is a valid representation of an XML document, because when I
> serialize
>> such a DOM, I get a valid octet stream.
>
> No, you're incorrect there. First of all, "valid" isn't a term of art there
> unless you apply a DTD or schema. The term is "well-formed", and it is the
> case that unless you add a namespace declaration attribute to the element
> (or it's in scope above), the XML will not be well formed if you simply
> output the DOM directly.
>

Point taken. What I meant was just that the the DOM serializes with the
correct namespaces.

> Now, if you run it through a fix-up in which the serializer "just fixes
> stuff" like adding missing declarations, then yes, it might come out
> well-formed.
>

The serializer associated with JDK 1.6.0 does precisely this. I don't
know whether such behaviour is mandated or discretionary, but it is
entirely reasonable. Without is, what is the point of having a method
createElementNS? It's presence, along with getPrefix, getNamespaceURI
and getLocalName, simply muddies the waters.

And if the serializer can interpret the DOM in such a (to me,
reasonable) way, why can't the canonicalizer?

I can't see that anyone would deliberately rely on the fact that, having
created a namespaced element with createElementNS (including the prefix
in the qualified name), that subsequent canonicalization would _not_
include the namespace attribute. I can, however, see that there might be
code out there that is de facto depending on this behaviour.

> Also, I believe that formally speaking, a DOM itself has no well-formedness
> property. I could be wrong about that. If it did, then I'm saying that in
> your example, the DOM isn't.
>
>> However, such a DOM does not, before
>> I normalizeDocument, have attribute nodes containing namespace
>> specifications. It has something, though, because it serializes correctly.
>
> It serializes because any DOM can be serialized. "Correctness" depends on
> context.
>
>> It seems to me that this situation more closely represents the XPath data
>> model than does the same DOM after normalization, in that namespace nodes
>> cannot be found by searching the attribute axis. The can, however, be
> found
>> by means specific to namespaces: getprefix, getNamespaceURI, getLocalName.
>> After normalizeDocument, these relationships still pertain.
>
> That's true, but the fact is that setting a prefix and namespace URI on an
> element does NOT imply that the prefix is bound to that namespace. That
> might be nice, but that's not how it works in a DOM.
>
>> If, however, I attempt to set the namespace/attribute node specifically,
>> using, say,
>> root.setAttribute("xmlns:my", "a:b:d");
>> things get messy.
>
> Welcoome to XML. And FWIW, you'd use setAttributeNS. NEVER use DOM1 calls if
> you're using namespaces.
>

setAttributeNS doesn't work for me following createElementNS. I get
org.w3c.dom.DOMException: NAMESPACE_ERR: An attempt is made to create or
change an object in a way which is incorrect with regard to namespaces.

This makes sense, but means that the only way to get the xmlns attribute
expressed is to normalizeDocument(). My feeling is that this reinforces
my point about the expected behaviour of the serializer.

Does it work for you? Maybe I'm doing something wrong.

>> <my:name xmlns:my="a:b:c" xmlns:my="a:b:b"/>
>
> I'm pretty sure that's because of your bug, using a DOM1 call to create that
> initial attribute.
>
>> I'm not sure when this arrangement of the namespace and attribute occurs,
>> but the ordering may be affected by the order of creation -
> createElementNS
>> first, followed by the attribute setup. In any case, I don't know what the
>> canonicalizer would do with this. Neither do I know which xmlns:my would
> win
>> when the stream is read next time.
>
> Neither will because that XML isn't well-formed. It won't parse.
>
> Try using setAttributeNS and I think you'll have more luck.
>
> -- Scott

--
Peter B. West <http://cv.pbw.id.au/>
Folio <http://defoe.sourceforge.net/folio/>

Re: Undeclared namespace prefix

2008-10-04T00:40:33Z

Raul Benito schrieb:
> I was talkinb about the use of them alone and then need to be c14n by
> itself. Anyway I see the point, and I think is one of the sane ones to be
> use outside of the signature. But please in order to not repeat it can you
> send me the junit test case. It will be make the change faster, and it also
> will allowed us not do the same mistake again.
>

Sure - I need to extract a "pure" xmlsec unit test out out the overall WSS4J
unit test.

Just another thought in this context:
The XML Signature specification (and the many other XML specifications in
general) do not restrict usage of all the XML elements they define. Usually
there is no definition of "this is an internal element" or "this is an external
element" (xmlsec implements elements as objects).

For example KeyInfo is used in XML Signature as well as in XML Encryption
specifications. Other elements specified in XML Signature may be re-used
elsewhere (see to the large set of OASIS Web Service specifications :-) ).

In my understanding an implementation of XML Security specification (such as
xmlsec) shall expect that _every_ element could be used in some other context,
even stand-alone. There is no reason why an application shall not be able to
re-use for example a "Reference" element, or a "X509Data" element as a
stand-alone element if the application's XML structure requires this. And of
course an application shall be able to use xmlsec in this case - because it
exists, is tested, and implements these elements.

There is also no such definition as "a sane use outside as signature" - any
application decides on its own what is "sane" or "insane" with respect to the
XML structures it uses.

As a summary: there is *no reason* (and in large parts it's counter-productive)
to single out elements that are defined in the specifications and make them
usable in one specific context only.

Regards,
Werner

>
> On Fri, Oct 3, 2008 at 5:15 PM, Werner Dittmann <Werner.Dittmann@...
>> wrote:
>
>> Raul Benito schrieb:
>>
>>> Hello,
>>> I think I made the change so I will try to defend it, first of all the use
>>> of KeyInfo out of a Signature it is not a use case I was looking to.
>>>
>> Raul,
>>
>> KeyInfo as such (as an XML element) is not used inside Signature only. If
>> you
>> have a look into the OASIS WSS specification you will see that KeyInfo is
>> used everywhere (nearly everywhere) a key is used, thus also to store
>> information
>> and references to encryption keys and so on. And these are exactly the test
>> cases
>> that break when we use KeyInfo to implement OASIS WSS.
>>
>> Regards,
>> Werner
>>
>>
>> So
>>
>>> perhaps we break it as we don't look at it. And sadly the old api is full
>>> of
>>> internal objects that can be use external. And I see KeyInfo like that.
>>> So in order to fix, can you write a test case that fails and submit a bug,
>>> I
>>> will update the code in SVN head.
>>> Thanks,
>>> Raul
>>>
>>>
>

Re: Undeclared namespace prefix

2008-10-03T09:19:20Z

I was talkinb about the use of them alone and then need to be c14n by itself. Anyway I see the point, and I think is one of the sane ones to be use outside of the signature. But please in order to not repeat it can you send me the junit test case. It will be make the change faster, and it also will allowed us not do the same mistake again.

On Fri, Oct 3, 2008 at 5:15 PM, Werner Dittmann <Werner.Dittmann@...> wrote:

Raul Benito schrieb:

Hello,
I think I made the change so I will try to defend it, first of all the use
of KeyInfo out of a Signature it is not a use case I was looking to.

Raul,

KeyInfo as such (as an XML element) is not used inside Signature only. If you
have a look into the OASIS WSS specification you will see that KeyInfo is
used everywhere (nearly everywhere) a key is used, thus also to store information
and references to encryption keys and so on. And these are exactly the test cases
that break when we use KeyInfo to implement OASIS WSS.

Regards,
Werner

So

perhaps we break it as we don't look at it. And sadly the old api is full of
internal objects that can be use external. And I see KeyInfo like that.
So in order to fix, can you write a test case that fails and submit a bug, I
will update the code in SVN head.
Thanks,
Raul

Re: Undeclared namespace prefix

2008-10-03T09:15:00Z

Raul Benito schrieb:
> Hello,
> I think I made the change so I will try to defend it, first of all the use
> of KeyInfo out of a Signature it is not a use case I was looking to.

Raul,

KeyInfo as such (as an XML element) is not used inside Signature only. If you
have a look into the OASIS WSS specification you will see that KeyInfo is
used everywhere (nearly everywhere) a key is used, thus also to store information
and references to encryption keys and so on. And these are exactly the test cases
that break when we use KeyInfo to implement OASIS WSS.

Regards,
Werner

So
> perhaps we break it as we don't look at it. And sadly the old api is full of
> internal objects that can be use external. And I see KeyInfo like that.
> So in order to fix, can you write a test case that fails and submit a bug, I
> will update the code in SVN head.
> Thanks,
> Raul
>

Re: Undeclared namespace prefix

2008-10-03T08:55:24Z

Hello,
I think I made the change so I will try to defend it, first of all the use of KeyInfo out of a Signature it is not a use case I was looking to. So perhaps we break it as we don't look at it. And sadly the old api is full of internal objects that can be use external. And I see KeyInfo like that.
So in order to fix, can you write a test case that fails and submit a bug, I will update the code in SVN head.
Thanks,
Raul

On Fri, Oct 3, 2008 at 4:31 PM, Werner Dittmann <Werner.Dittmann@...> wrote:

Scott Cantor schrieb:

Well, the bug is holistic. What I mean, and this is what I meant in my
original comment, is that the bug is a consequence of the inconsistent XML
handling across the range of libraries. It could be viewed as a bug in lots
of places.

In this specific case, I think it's probably easy to argue that the xmlsec
library has no reason not to declare that namespace, I suppose, but
ultimately
what xmlsec needs to do is define the contract...what is it prepared to
promise about the DOM it gives you?

Well looking at the documentation of the KeyInfo class it says that it will
look for "ds:KeyInfo" when constructed from an element. This is missing when
"creating from scratch" but one can assume the KeyInfo is somewhat symmetric
in its usage, thus also takes care of the ds: and namespace handling. IMHO,
if an element creation class has do deal with specific and well defined namespaces
that it should be the task of this class to take care of this. KeyInfo class
creates an element with a namespace prefix, thus it should also add the necessary
namespace attribute to it.

In WSS4J we do it similarly: we return a full Doc tree where elements have
their associated namespaces attributes thus the application does not need to
know about the specific/efined namespaces.

At least it seems that this modification in xmlsec-1.4.2 has some impacts on other
applications as well because this modification broke an "assumed" contract.

Regards,
Werner

You have to work from that contract, or (as in the case of OpenSAML) build
your own XML processing model that takes the responsibility on.

But it's also too easy to say "the caller has to fix up namespaces", because
unfortunately those namespaces are intrinsic to the signature process. You
can't just add them later and expect it to work.

-- Scott

Re: Undeclared namespace prefix

2008-10-03T08:31:57Z

Scott Cantor schrieb:

>
> Well, the bug is holistic. What I mean, and this is what I meant in my
> original comment, is that the bug is a consequence of the inconsistent XML
> handling across the range of libraries. It could be viewed as a bug in lots
> of places.
>
> In this specific case, I think it's probably easy to argue that the xmlsec
> library has no reason not to declare that namespace, I suppose, but
> ultimately
> what xmlsec needs to do is define the contract...what is it prepared to
> promise about the DOM it gives you?
>

Well looking at the documentation of the KeyInfo class it says that it will
look for "ds:KeyInfo" when constructed from an element. This is missing when
"creating from scratch" but one can assume the KeyInfo is somewhat symmetric
in its usage, thus also takes care of the ds: and namespace handling. IMHO,
if an element creation class has do deal with specific and well defined namespaces
that it should be the task of this class to take care of this. KeyInfo class
creates an element with a namespace prefix, thus it should also add the necessary
namespace attribute to it.

In WSS4J we do it similarly: we return a full Doc tree where elements have
their associated namespaces attributes thus the application does not need to
know about the specific/efined namespaces.

At least it seems that this modification in xmlsec-1.4.2 has some impacts on other
applications as well because this modification broke an "assumed" contract.

Regards,
Werner

> You have to work from that contract, or (as in the case of OpenSAML) build
> your own XML processing model that takes the responsibility on.
>
> But it's also too easy to say "the caller has to fix up namespaces", because
> unfortunately those namespaces are intrinsic to the signature process. You
> can't just add them later and expect it to work.
>
> -- Scott
>
>
>

RE: Undeclared namespace prefix

2008-10-03T08:08:23Z

> thanks for your hints :-) . Of course - the xmlns:ds attribute is the
> important part. After checking the attributes of KeyInfo it turned out
> that no xmlns:ds attribute was found.

I figured.

> This functions differs between 1.4.1 and 1.4.2 in some important lines
when
> it comes to add the xmlns:ds attribute to the KeyInfo element (see code
> snippets below). Why was this modification done? I couldn't
finddocumentation
> about this on the xmlsec Web pages. Does KeyInfo expect the user of
KeyInfo
> to set the xmlns:ds attribute? Or is it just a simple, plain old bug? :-)

Well, the bug is holistic. What I mean, and this is what I meant in my
original comment, is that the bug is a consequence of the inconsistent XML
handling across the range of libraries. It could be viewed as a bug in lots
of places.

In this specific case, I think it's probably easy to argue that the xmlsec
library has no reason not to declare that namespace, I suppose, but
ultimately
what xmlsec needs to do is define the contract...what is it prepared to
promise about the DOM it gives you?

You have to work from that contract, or (as in the case of OpenSAML) build
your own XML processing model that takes the responsibility on.

But it's also too easy to say "the caller has to fix up namespaces", because
unfortunately those namespaces are intrinsic to the signature process. You
can't just add them later and expect it to work.

-- Scott

Re: Undeclared namespace prefix

2008-10-03T03:22:04Z

Scott,

>
> Sorry, that's not the point. Is there a namespace *declaration* there? The
> fact that the DOM knows that it has that namespace is irrelevant to the c14n
> process, as far as I'm aware. If the DOM doesn't have the xmlns:ds
> attribute, you will not get it back out.
>

thanks for your hints :-) . Of course - the xmlns:ds attribute is the
important part. After checking the attributes of KeyInfo it turned out
that no xmlns:ds attribute was found.

The source code that creates the document uses the
org.apache.xml.security.keys.KeyInfo java class to create the keyinfo element
and to link it into the security header. Digging a bit deeper (via
"org.apache.xml.security.utils.SignatureElementProxy") I found the function
"createElementInSignatureSpace()" in "org.apache.xml.security.utils.XMLUtils".

This functions differs between 1.4.1 and 1.4.2 in some important lines when
it comes to add the xmlns:ds attribute to the KeyInfo element (see code
snippets below). Why was this modification done? I couldn't find documentation
about this on the xmlsec Web pages. Does KeyInfo expect the user of KeyInfo
to set the xmlns:ds attribute? Or is it just a simple, plain old bug? :-)

Regards,
Werner

Code snippets:

**** 1.4.1 :
public static Element createElementInSignatureSpace(Document doc,
String elementName) {

if (doc == null) {
throw new RuntimeException("Document is null");
}

if ((dsPrefix == null) || (dsPrefix.length() == 0)) {
Element element = doc.createElementNS(Constants.SignatureSpecNS,
elementName);

element.setAttributeNS(Constants.NamespaceSpecNS, "xmlns",
Constants.SignatureSpecNS);

return element;
}
String namePrefix=(String) namePrefixes.get(elementName);
if (namePrefix==null) {
StringBuffer tag=new StringBuffer(dsPrefix);
tag.append(':');
tag.append(elementName);
namePrefix=tag.toString();
namePrefixes.put(elementName,namePrefix);
}
Element element = doc.createElementNS(Constants.SignatureSpecNS, namePrefix);
element.setAttributeNS(Constants.NamespaceSpecNS, xmlnsDsPrefix,
Constants.SignatureSpecNS);

return element;

}

**** 1.4.2 :
public static Element createElementInSignatureSpace(Document doc,
String elementName) {

if (doc == null) {
throw new RuntimeException("Document is null");
}

if ((dsPrefix == null) || (dsPrefix.length() == 0)) {
return doc.createElementNS(Constants.SignatureSpecNS, elementName);
}
String namePrefix=(String) namePrefixes.get(elementName);
if (namePrefix==null) {
StringBuffer tag=new StringBuffer(dsPrefix);
tag.append(':');
tag.append(elementName);
namePrefix=tag.toString();
namePrefixes.put(elementName,namePrefix);
}
return doc.createElementNS(Constants.SignatureSpecNS, namePrefix);
}

RE: Undeclared namespace prefix

2008-10-02T09:44:05Z

> the findElement(...) function finds a node, then the code prints its local
> name and the namespace URI:
>
> Before c14n
> KeyInfo, http://www.w3.org/2000/09/xmldsig#
>
> After running c14n the xmlns: namespace binding is gone.

Sorry, that's not the point. Is there a namespace *declaration* there? The
fact that the DOM knows that it has that namespace is irrelevant to the c14n
process, as far as I'm aware. If the DOM doesn't have the xmlns:ds
attribute, you will not get it back out.

> findElemen(...) is just a lookup function to find an element according to
> the parameters and it does not modify the doc tree in any way.

Right, and it also doesn't pay any attention to namespace declarations.

-- Scott

Re: Undeclared namespace prefix

2008-10-02T09:34:24Z

After a small update of the test like this:

....
Canonicalizer c14n =
Canonicalizer.getInstance(Canonicalizer.ALGO_ID_C14N_WITH_COMMENTS);
System.out.println("Before c14n");
Node n = WSSecurityUtil.findElement(doc.getDocumentElement(), "KeyInfo", "http://www.w3.org/2000/09/xmldsig#");
if (n == null) {
System.out.println("KeyInfo not fond");
}
else {
System.out.println(n.getLocalName() + ", " + n.getNamespaceURI());
}
System.out.println(XMLUtils.PrettyDocumentToString(doc));
byte[] canonicalMessage = c14n.canonicalizeSubtree(doc);
System.out.println("After c14n: " + new String(canonicalMessage));
....

the findElement(...) function finds a node, then the code prints its local
name and the namespace URI:

Before c14n
KeyInfo, http://www.w3.org/2000/09/xmldsig#

After running c14n the xmlns: namespace binding is gone.
findElemen(...) is just a lookup function to find an element according to
the parameters and it does not modify the doc tree in any way.

Except for the new printout the output of this test of xmlsec-1.4.2 is the
same as attached to my last e-mail

Regards,
Werner

Scott Cantor schrieb:

>> Is is still necessary to file a new bug for this, or has Werner's work
>> covered this?
>
> I think it's still being investigated. If there's no bug filed yet, I would
> suggest doing so, and send Werner the bug number so he can attach the
> results of his tests.
>
> -- Scott
>
>
>

RE: Undeclared namespace prefix

2008-10-02T08:15:47Z

> I am uncertain about the underlying assumptions here. Please correct me
> where I am wrong in what follows.

I have to warn you, I'm not a c14n expert, and I know approximately zero
about XPath. What I'm knowledgable about are the practical aspects of using
a DOM and signing.

> When I construct a DOM with default configuration using createElementNS,
the
> elements have an associated prefix, namespace and local name. I assume
such
> a DOM is a valid representation of an XML document, because when I
serialize
> such a DOM, I get a valid octet stream.

No, you're incorrect there. First of all, "valid" isn't a term of art there
unless you apply a DTD or schema. The term is "well-formed", and it is the
case that unless you add a namespace declaration attribute to the element
(or it's in scope above), the XML will not be well formed if you simply
output the DOM directly.

Now, if you run it through a fix-up in which the serializer "just fixes
stuff" like adding missing declarations, then yes, it might come out
well-formed.

Also, I believe that formally speaking, a DOM itself has no well-formedness
property. I could be wrong about that. If it did, then I'm saying that in
your example, the DOM isn't.

> However, such a DOM does not, before
> I normalizeDocument, have attribute nodes containing namespace
> specifications. It has something, though, because it serializes correctly.

It serializes because any DOM can be serialized. "Correctness" depends on
context.

> It seems to me that this situation more closely represents the XPath data
> model than does the same DOM after normalization, in that namespace nodes
> cannot be found by searching the attribute axis. The can, however, be
found
> by means specific to namespaces: getprefix, getNamespaceURI, getLocalName.
> After normalizeDocument, these relationships still pertain.

That's true, but the fact is that setting a prefix and namespace URI on an
element does NOT imply that the prefix is bound to that namespace. That
might be nice, but that's not how it works in a DOM.

> If, however, I attempt to set the namespace/attribute node specifically,
> using, say,
> root.setAttribute("xmlns:my", "a:b:d");
> things get messy.

Welcoome to XML. And FWIW, you'd use setAttributeNS. NEVER use DOM1 calls if
you're using namespaces.

> <my:name xmlns:my="a:b:c" xmlns:my="a:b:b"/>

I'm pretty sure that's because of your bug, using a DOM1 call to create that
initial attribute.

> I'm not sure when this arrangement of the namespace and attribute occurs,
> but the ordering may be affected by the order of creation -
createElementNS
> first, followed by the attribute setup. In any case, I don't know what the
> canonicalizer would do with this. Neither do I know which xmlns:my would
win
> when the stream is read next time.

Neither will because that XML isn't well-formed. It won't parse.

Try using setAttributeNS and I think you'll have more luck.

-- Scott

RE: Undeclared namespace prefix

2008-10-01T22:42:25Z

Scott Cantor wrote:

> Because we use it in WSS4J we always rely on the backward compatibility.
> Changing the WSS4J code just to circumvent a problem would cause a major
> effort to many applications that actually use WSS4J. Before changing
> WSS4J we shall check if a fix in xmlsec is more appropriate.

If the issue is with how the XML is being serialized, that's your
responsibility, not xmlsec's. Namespace declarations can't be assumed. You
need to manage the DOM rigorously and with great attention to detail.

The fact that a lot of code in these Apache projects doesn't is exactly why
we did our own.

-- Scott

Scott,

I am uncertain about the underlying assumptions here. Please correct me where I am wrong in what follows.

The problem I have seen involves canonicalization. The c14n processing model is defined in terms of the XPath data model. As I understand it, the XPath data model separates the attribute and namespace axes. You can't find the namespaces on the attribute axis.

Implementations of c14n may be based on XPath, or not. They may accept a node set as input, or not. When they do accept a node set, whether that node set is, in fact, a node set returned by XPath or a DOM nodeset, should it not be treated in a manner which is compatible with the XPath data model?

When I construct a DOM with default configuration using createElementNS, the elements have an associated prefix, namespace and local name. I assume such a DOM is a valid representation of an XML document, because when I serialize such a DOM, I get a valid octet stream. However, such a DOM does not, before I normalizeDocument, have attribute nodes containing namespace specifications. It has something, though, because it serializes correctly.

It seems to me that this situation more closely represents the XPath data model than does the same DOM after normalization, in that namespace nodes cannot be found by searching the attribute axis. The can, however, be found by means specific to namespaces: getprefix, getNamespaceURI, getLocalName. After normalizeDocument, these relationships still pertain.

If, however, I attempt to set the namespace/attribute node specifically, using, say,
root.setAttribute("xmlns:my", "a:b:d");
things get messy. For example,

Document doc = newDocument();
Element root = (Element) doc.createElementNS("a:b:c", "my:name");
root.setAttribute("xmlns:my", "a:b:b");
doc.appendChild(root);
System.err.println("root ns: "+root.getNamespaceURI());
System.err.println("prefix: "+root.getPrefix());
System.err.println("localname: "+root.getLocalName());
NamedNodeMap map = root.getAttributes();
for ( int i = 0; i < map.getLength(); i++ ) {
Attr attr = (Attr) map.item(i);
System.err.println("attr name: "+attr.getName());
System.err.println("value: "+attr.getValue());
}
serializeTestData(doc, System.err);
System.err.println("");
doc.normalizeDocument();
System.err.println("root ns: "+root.getNamespaceURI());
System.err.println("prefix: "+root.getPrefix());
System.err.println("localname: "+root.getLocalName());
map = root.getAttributes();
for ( int i = 0; i < map.getLength(); i++ ) {
Attr attr = (Attr) map.item(i);
System.err.println("attr name: "+attr.getName());
System.err.println("value: "+attr.getValue());
}
serializeTestData(doc, System.err);
System.err.println("");

gives me:

root ns: a:b:c
prefix: my
localname: name
attr name: xmlns:my
value: a:b:b
<my:name xmlns:my="a:b:c" xmlns:my="a:b:b"/>
root ns: a:b:c
prefix: my
localname: name
attr name: xmlns:my
value: a:b:c
attr name: xmlns:my
value: a:b:b
<my:name xmlns:my="a:b:c" xmlns:my="a:b:b"/>

I'm not sure when this arrangement of the namespace and attribute occurs, but the ordering may be affected by the order of creation - createElementNS first, followed by the attribute setup. In any case, I don't know what the canonicalizer would do with this. Neither do I know which xmlns:my would win when the stream is read next time.

The situation is artificial, but the getPrefix, getNamespaceURI and getLocalName values are constant.

Peter

Re: Can't trace org.jcp.xml.dsig.internal.dom.DOMXMLSignature in NetBeans

2008-10-01T18:50:46Z

Thanks Sean.

Yes, I tried that. I was interested to see if any problems that would surface for a user of the JDK, that were fixed in 1.4.2.

Peter

sean.mullan wrote:

Peter B. West wrote:
> That was quick. The source for org.jcp is not part of src.zip in the JDK
> distribution. If I try to use the xmlsec-1.4.2.jar sources, things go wrong,
> naturally. Where can I get the sources that were included in the JDK? I'm
> used 1.6.0_07 and 1.6.0_10RC.

See: http://download.java.net/jdk6/

It might be easier to use Apache XMLSec 1.4.2 jars with JDK 6, as then
you can debug the sources as they exist at Apache and use this alias for
reporting any issues.

You can do this with the endorsed override mechanism of the JDK. What
you need to do is download the following jars:

1) Java XMLSec 1.4.2
http://xml.apache.org/security/dist/java-library/
2) Commons Logging:
http://commons.apache.org/downloads/download_logging.cgi

You need the commons logging library because the Apache implementation
uses that instead of the JDK logging mechanism.

Put these two jars in a lib directory, and then specify that lib
directory as the endorsed directory when running your application, for
example:

java -Djava.endorsed.dirs=lib ...

--Sean

>
>
> Peter B. West wrote:
>> I've been trying to track down problems by tracing in NetBeans. I am using
>> the JSR-105 API for the signature components, so I assume that I am
>> getting these classes from rt.jar, as there is no services SPI in the
>> xmlsec-1.4.2.jar.
>>
>> When I try to trace the calls using NetBeans, the trace skips straight
>> over the xmlsec classes, but happily traces into the other classes in the
>> JDK. The Call Stack shows the DOMXMLSignature.sign method call in Hidden
>> Source Calls, but knows the line number.
>>
>> See http://www.nabble.com/file/p19752612/DOMSignContext.png
>> DOMSignContext.png
>>
>> Do you guys have any idea why this might be the case.
>>
>> Peter
>>
>

Re: Can't trace org.jcp.xml.dsig.internal.dom.DOMXMLSignature in NetBeans

2008-10-01T11:04:43Z

Peter B. West wrote:
> That was quick. The source for org.jcp is not part of src.zip in the JDK
> distribution. If I try to use the xmlsec-1.4.2.jar sources, things go wrong,
> naturally. Where can I get the sources that were included in the JDK? I'm
> used 1.6.0_07 and 1.6.0_10RC.

See: http://download.java.net/jdk6/

It might be easier to use Apache XMLSec 1.4.2 jars with JDK 6, as then
you can debug the sources as they exist at Apache and use this alias for
reporting any issues.

You can do this with the endorsed override mechanism of the JDK. What
you need to do is download the following jars:

1) Java XMLSec 1.4.2
http://xml.apache.org/security/dist/java-library/
2) Commons Logging:
http://commons.apache.org/downloads/download_logging.cgi

You need the commons logging library because the Apache implementation
uses that instead of the JDK logging mechanism.

Put these two jars in a lib directory, and then specify that lib
directory as the endorsed directory when running your application, for
example:

java -Djava.endorsed.dirs=lib ...

--Sean

>
>
> Peter B. West wrote:
>> I've been trying to track down problems by tracing in NetBeans. I am using
>> the JSR-105 API for the signature components, so I assume that I am
>> getting these classes from rt.jar, as there is no services SPI in the
>> xmlsec-1.4.2.jar.
>>
>> When I try to trace the calls using NetBeans, the trace skips straight
>> over the xmlsec classes, but happily traces into the other classes in the
>> JDK. The Call Stack shows the DOMXMLSignature.sign method call in Hidden
>> Source Calls, but knows the line number.
>>
>> See http://www.nabble.com/file/p19752612/DOMSignContext.png
>> DOMSignContext.png
>>
>> Do you guys have any idea why this might be the case.
>>
>> Peter
>>
>

Re: Questions about xml-security-1.4.2 and Java 1.6

2008-09-30T19:15:29Z

Sean, you mention that JDK6 include 1.3.1 + bug fixes. Where can I get hold of the source as included in 1.6?

Peter

sean.mullan wrote:

dan costelloe wrote:
> Greetings,
>
> I've recently been tinkering with xml-security-1.4.2 and the experience so far
> has left me with some questions. Perhaps someone on this list may be able to
> shed some light:
>
> 1) (Sun) Java 1.6 seems to already contain an xml-security implementation. Which
> version of the implementation is this?

Yes, v 1.3.1 (plus a few important bug fixes).

> 2a) Is it possible to use the xml-security-1.4.2 classes in a java application
> so that the 1.4.2 classes take a higher preference than those in rt.jar? If so,
> can this be done without using the -Xbootclasspath argument to the JVM?

Yes, place xmlsec.jar in the endorsed standards directory [1]. You will
also need to put the Apache commons logging jar [2] there as well, since
the Apache code uses a different logging mechanism than the JDK.

> 2b) Can it be done without the removal of the xml-security classes from rt.jar?

Yes, see above.

--Sean

1: http://java.sun.com/javase/6/docs/technotes/guides/standards/index.html
2: http://commons.apache.org/logging/

Re: Can't trace org.jcp.xml.dsig.internal.dom.DOMXMLSignature in NetBeans

2008-09-30T17:40:03Z

That was quick. The source for org.jcp is not part of src.zip in the JDK distribution. If I try to use the xmlsec-1.4.2.jar sources, things go wrong, naturally. Where can I get the sources that were included in the JDK? I'm used 1.6.0_07 and 1.6.0_10RC.

Peter B. West wrote:

I've been trying to track down problems by tracing in NetBeans. I am using the JSR-105 API for the signature components, so I assume that I am getting these classes from rt.jar, as there is no services SPI in the xmlsec-1.4.2.jar.

When I try to trace the calls using NetBeans, the trace skips straight over the xmlsec classes, but happily traces into the other classes in the JDK. The Call Stack shows the DOMXMLSignature.sign method call in Hidden Source Calls, but knows the line number.

See DOMSignContext.png

Do you guys have any idea why this might be the case.

Peter

RE: Undeclared namespace prefix

2008-09-30T17:17:18Z

> Is is still necessary to file a new bug for this, or has Werner's work
> covered this?

I think it's still being investigated. If there's no bug filed yet, I would
suggest doing so, and send Werner the bug number so he can attach the
results of his tests.

-- Scott

Can't trace org.jcp.xml.dsig.internal.dom.DOMXMLSignature in NetBeans

2008-09-30T17:10:11Z

I've been trying to track down problems by tracing in NetBeans. I am using the JSR-105 API for the signature components, so I assume that I am getting these classes from rt.jar, as there is no services SPI in the xmlsec-1.4.2.jar.

When I try to trace the calls using NetBeans, the trace skips straight over the xmlsec classes, but happily traces into the other classes in the JDK. The Call Stack shows the DOMXMLSignature.sign method call in Hidden Source Calls, but knows the line number.

See DOMSignContext.png

Do you guys have any idea why this might be the case.

Peter

Re: Undeclared namespace prefix

2008-09-30T16:53:08Z

Sean,

Is is still necessary to file a new bug for this, or has Werner's work covered this?

Peter

sean.mullan wrote:

I don't know what the cause of this regression could be.

The best thing to do is for Arnaud or Peter to file a new bug at
http://issues.apache.org/bugzilla under the Security project and if possible,
attach a standalone (i.e. not dependent on WSS4J) test case that reproduces the
problem.

Thanks,
Sean

Werner Dittmann wrote:
> All,
>
> why did it work with 1.4.0 and 1.4.1 then? Were these two version
> buggy in that case? Or what was changed in 1.4.2 to get this
> incompatible behaviour?
>
> Because we use it in WSS4J we always rely on the backward compatibility.
> Changing the WSS4J code just to circumvent a problem would cause a major
> effort to many applications that actually use WSS4J. Before changing
> WSS4J we shall check if a fix in xmlsec is more appropriate.
>
> Regards,
> Werner
>
> Arnaud schrieb:
>> Peter B. West <lists <at> pbw.id.au> writes:
>>
>>> Arnaud,
>>>
>>> I'm using 1.4.2, and the problem I had may not be the same as yours, but
>>> see my messages "Unbound prefix after decryption".
>>>
>>> In my case, the problem was that the DOM being passed for encryption id
>>> not have the xmlns attribute set before it was processed. A
>>> normalizeDocument() in the right place fixed the problem.
>>>
>>> Peter
>>>
>>> Arnaud wrote:
>>>> Hi all,
>>>>
>>>> Since using Chinese in soap messages which had their header encrypt
>>>> was not working (and after visiting the XML security page) I
>>>> upgraded the libraries
>> to
>>>> 1.4.1 as recommended.
>>>> And now I have the following exception on the serverside:
>>>> com.ctc.wstx.exc.WstxParsingException: Undeclared namespace prefix "ds"
>>>> at [row,col {unknown-source}]: [6,12]
>>>>
>>>> The "[row,col {unknown-source}]: [6,12]" refers to the <ds:KeyInfo>
>>>> of the
>> soap
>>>> header. If I compare the message sent by the client before the upgrade:
>>>>
>>>> <ds:KeyInfo xmlns:ds="http://www.w3.org/2000/09/xmldsig#">
>>>> ...
>>>> </ds:KeyInfo>
>>>>
>>>> and after:
>>>> <ds:KeyInfo> ...
>>>> </ds:KeyInfo>
>>>>
>>>> I can see the namespace disappeared but I have no idea why and how
>>>> to fix that...
>>>> If anyone has an idea please let me know because I don't know what
>>>> to do
>> about
>>>> it and I've alread spend many hours on this...
>>>>
>>>> Thanks in advance for any help.
>>>>
>>>> Arnaud.
>>
>> Hi Peter,
>>
>> Thanks a lot for your quick reply!
>> In my case, I'm not using the xml security libs directly but WSS4J
>> does that for me. Anyway, after reading your post I decided to
>> download the WSS4J source... And I added the normalizeDocument() just
>> after WSS4J calls the xml security to handle encryption and
>> signature... And you are right this fix the problem!!! I will
>> probably send a mail to the WSS4J team to let them know about this.
>>
>> Again thanks a lot!!!
>>
>> Arnaud.
>>
>>
>>
>

Re: Undeclared namespace prefix

2008-09-30T09:29:20Z

Scott Cantor schrieb:
>> Thus the printout "before c14n" is the doc tree just before c14n. IMHO the
>> Transformer does not add/modify during transformation of the doc tree to a
>> string.
>
> Unless we can prove that, my bet might be that the DOM before that step
> doesn't contain the declaration, and the DOM after it does. Thus the
> transformer might be adding it to "fix-up" the DOM.
>

I don't know what the transfomer does (never looked at the code). Of course I'll
insert some code to check if the xmlns:ds is contained in KeyInfo - I'll keep
you informed :-)

Regards,
Werner

> Can you inject some DOM code to just check the KeyInfo element and see if it
> has xmlns:ds declared? If it does, I think there has to be bug in the c14n
> code, although it sure seems like a blatant one...
>
> -- Scott
>
>
>
>

RE: Undeclared namespace prefix

2008-09-29T12:39:37Z

> Thus the printout "before c14n" is the doc tree just before c14n. IMHO the
> Transformer does not add/modify during transformation of the doc tree to a
> string.

Unless we can prove that, my bet might be that the DOM before that step
doesn't contain the declaration, and the DOM after it does. Thus the
transformer might be adding it to "fix-up" the DOM.

Can you inject some DOM code to just check the KeyInfo element and see if it
has xmlns:ds declared? If it does, I think there has to be bug in the c14n
code, although it sure seems like a blatant one...

-- Scott

Re: Undeclared namespace prefix

2008-09-29T09:44:48Z

All,

I've checked the input and output of c14n of xmlsec-1.4.2 and it seems
the c14n does not emit the ds: name space binding of the KeyInfo node.
Attached is the text file of the test run.

The same test but using xmlsec-1.4.1 shows the name space binding before
and after c14n.

The following the code snippet was used to produce the test output file:

...
Canonicalizer c14n =
Canonicalizer.getInstance(Canonicalizer.ALGO_ID_C14N_WITH_COMMENTS);
System.out.println("Before c14n");
System.out.println(XMLUtils.PrettyDocumentToString(doc));
byte[] canonicalMessage = c14n.canonicalizeSubtree(doc);
System.out.println("After c14n: " + new String(canonicalMessage));
...

the XMLUtils.PrettyDocumentToString(doc) (XMLUtils is not the xmlsec XMLUtils
but an own one :-) ) performs as follows:

public class XMLUtils {
public static String PrettyDocumentToString(Document doc) {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ElementToStream(doc.getDocumentElement(), baos);
return new String(baos.toByteArray());
}

public static void ElementToStream(Element element, OutputStream out) {
try {
DOMSource source = new DOMSource(element);
StreamResult result = new StreamResult(out);
TransformerFactory transFactory = TransformerFactory.newInstance();
Transformer transformer = transFactory.newTransformer();
transformer.transform(source, result);
} catch (Exception e) {
e.printStackTrace();
}
}

....

Thus the printout "before c14n" is the doc tree just before c14n. IMHO the Transformer
does not add/modify during transformation of the doc tree to a string.

Regards,
Werner

Scott Cantor schrieb:

>> Questions here: does the XML doc that goes into C14N misses any
>> xmlns: declarations at some important positions? If so - where should
>> we include these?
>
> With either incl or excl, the ds namespace prefix should be emitted in both
> spots, since it's visibly used in that element, and not used anywhere up
> above it. So the first/only place it should appear is in the KeyInfo
> element.
>
> If you're saying that the XML listed first is directly passed (in DOM form)
> into the c14n step, and the output is missing the ds namespace, then it's a
> c14n bug.
>
> Otherwise the bug is in the process being used to turn the original XML into
> a DOM that you give to the c14n code.
>
> -- Scott
>
>
>

Before c14n

<?xml version="1.0" encoding="UTF-8"?><soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xenc="http://www.w3.org/2001/04/xmlenc#" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<soapenv:Header>
<wsse:Security xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" soapenv:mustUnderstand="1"><wsse:BinarySecurityToken xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3" wsu:Id="urn:uuid:53A8D861C43C9251E812227056952511" xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd">MIIDNDCCAp2gAwIBAgIBEDANBgkqhkiG9w0BAQQFADBmMQswCQYDVQQGEwJERTEPMA0GA1UECBMGQmF5ZXJuMQ8wDQYDVQQHEwZNdW5pY2gxDTALBgNVBAoTBEhvbWUxFTATBgNVBAsTDEFwYWNoZSBXU1M0SjEPMA0GA1UEAxMGV2VybmVyMB4XDTA4MDQwNDE5MzIxOFoXDTEwMDQwNDE5MzIxOFowYTELMAkGA1UEBhMCREUxDzANBgNVBAgTBkJheWVybjEPMA0GA1UEBxMGTXVuaWNoMQ8wDQYDVQQKEwZBcGFjaGUxDjAMBgNVBAsTBVdTUzRKMQ8wDQYDVQQDEwZXZXJuZXIwgZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJAoGBAINlL3/k0H/zvknpBtLo8jzXwx/IJU/CGSv6MsqJZ2fyZ6kpLlXCuSBUZ/tfkdxpuzhYq/Sc7A8csIk9gDf9RUbrhK0qKw0VP6DoCIJjS5IeN+NeJkx8YjmzLPmZqLYbNPXr/hy8CRrR6CqLTTSkBwoEJ+cDkfZrdH2/bND0FEIZAgMBAAGjgfYwgfMwCQYDVR0TBAIwADAsBglghkgBhvhCAQ0EHxYdT3BlblNTTCBHZW5lcmF0ZWQgQ2VydGlmaWNhdGUwHQYDVR0OBBYEFFSZXv0I5bG7XPEwjylwG3lmZGdiMIGYBgNVHSMEgZAwgY2AFL/FsHHolGIMacU1TZW/88Bd2EL6oWqkaDBmMQswCQYDVQQGEwJERTEPMA0GA1UECBMGQmF5ZXJuMQ8wDQYDVQQHEwZNdW5pY2gxDTALBgNVBAoTBEhvbWUxFTATBgNVBAsTDEFwYWNoZSBXU1M0SjEPMA0GA1UEAxMGV2VybmVyggkAuBIOAWJ19mwwDQYJKoZIhvcNAQEEBQADgYEAUiUh/wORVcQYXxIh13h3w2Btg6Kj2g6V6YO0Utc/gEYWwT310C2OuroKAwwoHapMIIWiJRclIAiA8Hnb0Sv/puuHYD4G4NWFdiVjRord90eZJe40NMGruRmlqIRIGGKCv+wv3E6Ux1cWW862f5H9Eyrcocke2P+3GNAGy83vghA=</wsse:BinarySecurityToken><xenc:EncryptedKey Id="EncKeyId-urn:uuid:53A8D861C43C9251E812227056952682" xmlns:xenc="http://www.w3.org/2001/04/xmlenc#">
<xenc:EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#rsa-1_5"/>
<ds:KeyInfo xmlns:ds="http://www.w3.org/2000/09/xmldsig#">
<wsse:SecurityTokenReference xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd"><wsse:Reference URI="#urn:uuid:53A8D861C43C9251E812227056952511" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3" xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd"/></wsse:SecurityTokenReference>
</ds:KeyInfo>
<xenc:CipherData><xenc:CipherValue>QNzRQyTf8By78r5vjLJRMgwdAWqcVArd7ycxrVADdWItCf+buh12Es05rAwO1GETg4LgPrENdkvsQ5uhwITQYveBW7f6ikDHENruKNhYzYQGofihxzQ+o6xIgQrla2SyqBZ26fdImMdZiCzyFb4YEk7w0GmASi2af+HlROgXRlA=</xenc:CipherValue></xenc:CipherData>
<xenc:ReferenceList><xenc:DataReference URI="#EncDataId-1246293429"/></xenc:ReferenceList></xenc:EncryptedKey></wsse:Security></soapenv:Header><soapenv:Body><xenc:EncryptedData Id="EncDataId-1246293429" Type="http://www.w3.org/2001/04/xmlenc#Content" xmlns:xenc="http://www.w3.org/2001/04/xmlenc#"><xenc:EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#tripledes-cbc" xmlns:xenc="http://www.w3.org/2001/04/xmlenc#"/><ds:KeyInfo xmlns:ds="http://www.w3.org/2000/09/xmldsig#">
<wsse:SecurityTokenReference xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd"><wsse:Reference URI="#EncKeyId-urn:uuid:53A8D861C43C9251E812227056952682" xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd"/></wsse:SecurityTokenReference>
</ds:KeyInfo><xenc:CipherData xmlns:xenc="http://www.w3.org/2001/04/xmlenc#"><xenc:CipherValue xmlns:xenc="http://www.w3.org/2001/04/xmlenc#">JSjGbM2Jb7+oPGNEwdf9PhqrOIWJnGwyB/HHoZiy3cjvZM2D7U0G5Kgb9Bwz+LRjMFQghSnpHNnS
uwH9EeU8XnJLuI2BUQSb3X3x/FuaHWcjZncOtwg9HllTpD26MTEzuHJFpSnC/gEHcLueXzg324bN
zEwj9XqBvdBgmbRblDfNREcY+9XBrp5OxCWfkaGWoNUDefLBoTRsaTi5heclSxcudFx6lk/riHzZ
SRGQamR1kNbVj9BwJjKmNuEQHpq4ZOgkjB4cPFIHNBW9vwX+yQOgMpp3k5A79GkDDa9qCdaqvrka
nqFkZUrl4rhslRUDFKCnhve0vmDTtDta3o3uuOMxVolWpjn3dt04q5IcaXoHsKesNVDHsA==</xenc:CipherValue></xenc:CipherData></xenc:EncryptedData></soapenv:Body></soapenv:Envelope>

After c14n:

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xenc="http://www.w3.org/2001/04/xmlenc#" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<soapenv:Header>
<wsse:Security xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" soapenv:mustUnderstand="1"><wsse:BinarySecurityToken xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3" wsu:Id="urn:uuid:53A8D861C43C9251E812227056952511">MIIDNDCCAp2gAwIBAgIBEDANBgkqhkiG9w0BAQQFADBmMQswCQYDVQQGEwJERTEPMA0GA1UECBMGQmF5ZXJuMQ8wDQYDVQQHEwZNdW5pY2gxDTALBgNVBAoTBEhvbWUxFTATBgNVBAsTDEFwYWNoZSBXU1M0SjEPMA0GA1UEAxMGV2VybmVyMB4XDTA4MDQwNDE5MzIxOFoXDTEwMDQwNDE5MzIxOFowYTELMAkGA1UEBhMCREUxDzANBgNVBAgTBkJheWVybjEPMA0GA1UEBxMGTXVuaWNoMQ8wDQYDVQQKEwZBcGFjaGUxDjAMBgNVBAsTBVdTUzRKMQ8wDQYDVQQDEwZXZXJuZXIwgZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJAoGBAINlL3/k0H/zvknpBtLo8jzXwx/IJU/CGSv6MsqJZ2fyZ6kpLlXCuSBUZ/tfkdxpuzhYq/Sc7A8csIk9gDf9RUbrhK0qKw0VP6DoCIJjS5IeN+NeJkx8YjmzLPmZqLYbNPXr/hy8CRrR6CqLTTSkBwoEJ+cDkfZrdH2/bND0FEIZAgMBAAGjgfYwgfMwCQYDVR0TBAIwADAsBglghkgBhvhCAQ0EHxYdT3BlblNTTCBHZW5lcmF0ZWQgQ2VydGlmaWNhdGUwHQYDVR0OBBYEFFSZXv0I5bG7XPEwjylwG3lmZGdiMIGYBgNVHSMEgZAwgY2AFL/FsHHolGIMacU1TZW/88Bd2EL6oWqkaDBmMQswCQYDVQQGEwJERTEPMA0GA1UECBMGQmF5ZXJuMQ8wDQYDVQQHEwZNdW5pY2gxDTALBgNVBAoTBEhvbWUxFTATBgNVBAsTDEFwYWNoZSBXU1M0SjEPMA0GA1UEAxMGV2VybmVyggkAuBIOAWJ19mwwDQYJKoZIhvcNAQEEBQADgYEAUiUh/wORVcQYXxIh13h3w2Btg6Kj2g6V6YO0Utc/gEYWwT310C2OuroKAwwoHapMIIWiJRclIAiA8Hnb0Sv/puuHYD4G4NWFdiVjRord90eZJe40NMGruRmlqIRIGGKCv+wv3E6Ux1cWW862f5H9Eyrcocke2P+3GNAGy83vghA=</wsse:BinarySecurityToken><xenc:EncryptedKey Id="EncKeyId-urn:uuid:53A8D861C43C9251E812227056952682">
<xenc:EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#rsa-1_5"></xenc:EncryptionMethod>
<ds:KeyInfo>
<wsse:SecurityTokenReference><wsse:Reference URI="#urn:uuid:53A8D861C43C9251E812227056952511" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3"></wsse:Reference></wsse:SecurityTokenReference>
</ds:KeyInfo>
<xenc:CipherData><xenc:CipherValue>QNzRQyTf8By78r5vjLJRMgwdAWqcVArd7ycxrVADdWItCf+buh12Es05rAwO1GETg4LgPrENdkvsQ5uhwITQYveBW7f6ikDHENruKNhYzYQGofihxzQ+o6xIgQrla2SyqBZ26fdImMdZiCzyFb4YEk7w0GmASi2af+HlROgXRlA=</xenc:CipherValue></xenc:CipherData>
<xenc:ReferenceList><xenc:DataReference URI="#EncDataId-1246293429"></xenc:DataReference></xenc:ReferenceList></xenc:EncryptedKey></wsse:Security></soapenv:Header><soapenv:Body><xenc:EncryptedData Id="EncDataId-1246293429" Type="http://www.w3.org/2001/04/xmlenc#Content"><xenc:EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#tripledes-cbc"></xenc:EncryptionMethod><ds:KeyInfo>
<wsse:SecurityTokenReference xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd"><wsse:Reference URI="#EncKeyId-urn:uuid:53A8D861C43C9251E812227056952682"></wsse:Reference></wsse:SecurityTokenReference>
</ds:KeyInfo><xenc:CipherData><xenc:CipherValue>JSjGbM2Jb7+oPGNEwdf9PhqrOIWJnGwyB/HHoZiy3cjvZM2D7U0G5Kgb9Bwz+LRjMFQghSnpHNnS
uwH9EeU8XnJLuI2BUQSb3X3x/FuaHWcjZncOtwg9HllTpD26MTEzuHJFpSnC/gEHcLueXzg324bN
zEwj9XqBvdBgmbRblDfNREcY+9XBrp5OxCWfkaGWoNUDefLBoTRsaTi5heclSxcudFx6lk/riHzZ
SRGQamR1kNbVj9BwJjKmNuEQHpq4ZOgkjB4cPFIHNBW9vwX+yQOgMpp3k5A79GkDDa9qCdaqvrka
nqFkZUrl4rhslRUDFKCnhve0vmDTtDta3o3uuOMxVolWpjn3dt04q5IcaXoHsKesNVDHsA==</xenc:CipherValue></xenc:CipherData></xenc:EncryptedData></soapenv:Body></soapenv:Envelope>

RE: Undeclared namespace prefix

2008-09-28T11:24:49Z

> Questions here: does the XML doc that goes into C14N misses any
> xmlns: declarations at some important positions? If so - where should
> we include these?

With either incl or excl, the ds namespace prefix should be emitted in both
spots, since it's visibly used in that element, and not used anywhere up
above it. So the first/only place it should appear is in the KeyInfo
element.

If you're saying that the XML listed first is directly passed (in DOM form)
into the c14n step, and the output is missing the ds namespace, then it's a
c14n bug.

Otherwise the bug is in the process being used to turn the original XML into
a DOM that you give to the c14n code.

-- Scott