[Issue 960] New - Security does not work with jaxb DOM binding

[capsicum]

https://wsit.dev.java.net/issues/show_bug.cgi?id=960
                 Issue #|960
                 Summary|Security does not work with jaxb DOM binding
               Component|wsit
                 Version|1.2
                Platform|All
              OS/Version|All
                     URL|
                  Status|NEW
       Status whiteboard|
                Keywords|
              Resolution|
              Issue type|DEFECT
                Priority|P3
            Subcomponent|security
             Assigned to|ashutoshshahi
             Reported by|capsicum






------- Additional comments from capsicum@... Tue Jul  8 14:36:14 +0000 2008 -------
Using jaxb DOM binding makes security fail. By looking at the stack trace and
the Metro source code I can see that it's the message integrity check which
fails. I suspect this is because jaxb (or something else?) rewrites namespace
prefixes and adds namespace declarations in the message in transit. I can see
these changes if I do not use security. For instance, if I send a DOM node such
as this in a message:

<foo xmlns="http://example.com/someSchema">
  <bar>
   ...
  </bar>
</foo>

when I receive it will look something like this:

<ns1:foo xmlns:ns1="http://example.com/someSchema"
   xmlsns:ns2="http://some.another.namespace.here/2"
   xmlsns:ns3="http://some.another.namespace.here/3">
  <ns1:bar xmlsns:ns4="http://some.another.namespace.here/4">
   ...
  </ns1:bar>
</ns1:foo>

These added namespaces are all namespaces used in the WSDL.

I am sorry for the poor description of the issue, but it was some time ago I
dealt with this and I couldn't find my old notes on the issue. In the end I
disabled security for then and I will likely have to work around the DOM binding
issue by not using DOM binding unless this is fixed.

If it is not immediately obvious what is wrong, I can put together a small repro
for you. Please let me know.

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@...
For additional commands, e-mail: issues-help@...

[ashutoshshahi]

https://wsit.dev.java.net/issues/show_bug.cgi?id=960






------- Additional comments from ashutoshshahi@... Tue Jul  8 15:20:34 +0000 2008 -------
Yes please, attach a reproducible testcase if you can, and also the stacktrace
that you are seeing.

Ashutosh

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@...
For additional commands, e-mail: issues-help@...

[ashutoshshahi]

https://wsit.dev.java.net/issues/show_bug.cgi?id=960



User ashutoshshahi changed the following:

                What    |Old value                 |New value
================================================================================
        Target milestone|not determined            |1.4
--------------------------------------------------------------------------------




------- Additional comments from ashutoshshahi@... Thu Jul 10 11:36:34 +0000 2008 -------
Setting target milestone to 1.4

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@...
For additional commands, e-mail: issues-help@...

[ashutoshshahi]

https://wsit.dev.java.net/issues/show_bug.cgi?id=960



User ashutoshshahi changed the following:

                What    |Old value                 |New value
================================================================================
                  Status|NEW                       |STARTED
--------------------------------------------------------------------------------




------- Additional comments from ashutoshshahi@... Thu Jul 10 11:52:34 +0000 2008 -------
marking as started

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@...
For additional commands, e-mail: issues-help@...

[capsicum]

https://wsit.dev.java.net/issues/show_bug.cgi?id=960






------- Additional comments from capsicum@... Thu Jul 10 12:00:31 +0000 2008 -------
I'm working on a repro.

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@...
For additional commands, e-mail: issues-help@...

[capsicum]

https://wsit.dev.java.net/issues/show_bug.cgi?id=960






------- Additional comments from capsicum@... Sat Jul 12 13:56:45 +0000 2008 -------
The stack trace I am seeing is this:

javax.xml.crypto.dsig.XMLSignatureException: WSS1717: Error occurred while doing
digest verification of body/payload
javax.xml.ws.WebServiceException: javax.xml.crypto.dsig.XMLSignatureException:
WSS1717: Error occurred while doing digest verification of body/payload
        at
com.sun.xml.ws.security.opt.impl.incoming.processor.StreamingPayLoadDigester.accept(StreamingPayLoadDigester.java:89)
        at com.ctc.wstx.stax.FilteredStreamReader.next(FilteredStreamReader.java:45)
        at
com.sun.xml.ws.security.opt.impl.util.VerifiedMessageXMLStreamReader.next(VerifiedMessageXMLStreamReader.java:68)
        at
com.sun.xml.stream.buffer.stax.StreamReaderBufferCreator.storeElementAndChildrenNoEx(StreamReaderBufferCreator.java:245)
        at
com.sun.xml.stream.buffer.stax.StreamReaderBufferCreator.storeElementAndChildren(StreamReaderBufferCreator.java:177)
        at
com.sun.xml.stream.buffer.stax.StreamReaderBufferCreator.store(StreamReaderBufferCreator.java:142)
        at
com.sun.xml.stream.buffer.stax.StreamReaderBufferCreator.create(StreamReaderBufferCreator.java:82)
        at
com.sun.xml.stream.buffer.MutableXMLStreamBuffer.createFromXMLStreamReader(MutableXMLStreamBuffer.java:113)
        at
com.sun.xml.ws.security.opt.impl.incoming.VerifiedStreamMessage.cacheMessage(VerifiedStreamMessage.java:513)
        at
com.sun.xml.ws.security.opt.impl.incoming.VerifiedStreamMessage.readPayloadAsJAXB(VerifiedStreamMessage.java:259)
        at
com.sun.xml.ws.client.sei.ResponseBuilder$Body.readResponse(ResponseBuilder.java:469)
        at com.sun.xml.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandler.java:121)
        at com.sun.xml.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandler.java:89)
        at com.sun.xml.ws.client.sei.SEIStub.invoke(SEIStub.java:118)
<... My application code here ...>
Caused by: javax.xml.crypto.dsig.XMLSignatureException: WSS1717: Error occurred
while doing digest verification of body/payload
        at
com.sun.xml.ws.security.opt.impl.incoming.processor.StreamingPayLoadDigester.accept(StreamingPayLoadDigester.java:88)
        ... 46 more

I am having trouble isolating it to a small test case. I'll keep working on it.

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@...
For additional commands, e-mail: issues-help@...

[capsicum]

https://wsit.dev.java.net/issues/show_bug.cgi?id=960






------- Additional comments from capsicum@... Sat Jul 12 16:50:58 +0000 2008 -------
Created an attachment (id=670)
Testcase which reproduces the bug


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@...
For additional commands, e-mail: issues-help@...

[capsicum]

https://wsit.dev.java.net/issues/show_bug.cgi?id=960






------- Additional comments from capsicum@... Sat Jul 12 16:53:56 +0000 2008 -------
I uploaded a testcase which reproduces it. It took quite a lot of effort to
extract a smallish test case from the full application. I ended up with this
peculiar case, which I was not able to reduce further.

The test case consists of a simple web service built on a WSDL which has been
customized with JAXB dom bindings. I have duplicated the service so there are
secure and plain variants of it.

The WSDL has been written by hand and I have put the WS-SecurityPolicies in it
by hand, though they have been generated by Netbeans. I think they are right
since the security works if I simplify the schema of the messages being passed.

I am using a server certificate and the client authenticates with username/password.

- You must set the metro.home in the ant build file.

- You also need to modify the keystore location in the client wsit
configuration, shello-client.xml.

-Finally you must adjust the client username and password in the callback class
so it presents a valid login to your glassfish installation.

- I am using the "s1as" alias from the default glassfish installation for the
server. I suppose this should be fine for you.

To build the plain WAR run "ant war". This produces "hello.war" in "build".
Deploy it in glassfish.

To build the secure WAR run "ant secure-war". Deploy "securehello.war".

To run the plain client do "ant run-client".

To run the secure client do "ant run-secureclient"

When the secure service is invoked, I get the exception from the integrity check.

Strangely enough this test case seems to trigger some other bugs in jaxb. (Or am
I doing something wrong?) In the plain invocation, the two parts of the message
get mixed up and one part is lost. I have seen this behaviour previously, but in
the full application where I had a problem with the security failing, I didn't
have this mixed up messages issue if I used a plain service. And if you try to
simplified the schemas slightly, the security will start working, but the
message parts still get mixed up.

I suspect that the bugs are in JAXB rather than WSIT, but I am not sure.

I am running on Linux, but I have reproduced it on Windows as well. (In the full
application. I haven't tried the test case on windows.)

I am using the "official" metro 1.2 binaries and Java 6.

Sometimes I get a warning message like this:

 [wsimport] [WARNING] src-resolve: Cannot resolve the name 'hemsg:Message' to
a(n) 'element declaration' component.
 [wsimport]   line 46 of file:...testcase/hello.wsdl#types?schema2

Strangely enough whether I get this message or not depends on the full file path
and name of the WSDL. When I packed up the zip-file to test it before I uploaded
it here, I ran it in the directory called "testcase" which is included in the
zip. I got the above warning. If I renamed the directory to "t4", I didn't get
the warning. The code seems to work the same though regardless of this.

Another issue, which is perhaps a separate bug in JAXB?, is that the message
content is modified in transit. Lots of namespace prefix declarations are added
and the indentation is changed. The content is still the "same", but since
namespace declarations are significant in typical XML signatures, this behavior
would ruin any signatures I might be using at the application layer. I initially
thought that this was the reason of the integrity check failure, but with a
simplified schema I still get the modified content, but security works.


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@...
For additional commands, e-mail: issues-help@...

[ashutoshshahi]

https://wsit.dev.java.net/issues/show_bug.cgi?id=960






------- Additional comments from ashutoshshahi@... Mon Jul 14 10:28:13 +0000 2008 -------
Hello,
Thanks for the testcase.
I tried the testcase with latest 1.3 nightly and it is working fine for me. Can
you try with the nightly build and see if the issues have been fixed.

Ashutosh

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@...
For additional commands, e-mail: issues-help@...

[capsicum]

https://wsit.dev.java.net/issues/show_bug.cgi?id=960






------- Additional comments from capsicum@... Mon Jul 14 10:51:37 +0000 2008 -------
I tried the 1.3 nightly and I get the same error as before:

run-secureclient:
     [echo] Running the secure client
     [java] Jul 14, 2008 12:42:34 PM
[com.sun.xml.ws.policy.jaxws.PolicyConfigParser]  parse
     [java] INFO: WSP1049: Loaded WSIT configuration from file:
jar:file:/home/erik/workspace/MetroBugRepro/build/secureclient.jar!/META-INF/wsit-client.xml
     [java] Jul 14, 2008 12:42:39 PM
com.sun.xml.ws.security.opt.impl.incoming.processor.StreamingPayLoadDigester accept
     [java] SEVERE: WSS1717: Error occurred while doing digest verification of
body/payload
     [java] Exception in thread "main" javax.xml.ws.WebServiceException:
javax.xml.crypto.dsig.XMLSignatureException: WSS1717: Error occurred while doing
digest verification of body/payload
     [java] at
com.sun.xml.ws.security.opt.impl.incoming.processor.StreamingPayLoadDigester.accept(StreamingPayLoadDigester.java:103)
     [java] at
com.ctc.wstx.stax.FilteredStreamReader.next(FilteredStreamReader.java:45)
     [java] at
com.sun.xml.ws.security.opt.impl.util.VerifiedMessageXMLStreamReader.next(VerifiedMessageXMLStreamReader.java:82)
     [java] at
com.sun.xml.stream.buffer.stax.StreamReaderBufferCreator.storeElementAndChildrenNoEx(StreamReaderBufferCreator.java:245)
     [java] at
com.sun.xml.stream.buffer.stax.StreamReaderBufferCreator.storeElementAndChildren(StreamReaderBufferCreator.java:177)
     [java] at
com.sun.xml.stream.buffer.stax.StreamReaderBufferCreator.store(StreamReaderBufferCreator.java:142)
     [java] at
com.sun.xml.stream.buffer.stax.StreamReaderBufferCreator.create(StreamReaderBufferCreator.java:82)
     [java] at
com.sun.xml.stream.buffer.MutableXMLStreamBuffer.createFromXMLStreamReader(MutableXMLStreamBuffer.java:113)
     [java] at
com.sun.xml.ws.security.opt.impl.incoming.VerifiedStreamMessage.cacheMessage(VerifiedStreamMessage.java:527)
     [java] at
com.sun.xml.ws.security.opt.impl.incoming.VerifiedStreamMessage.readPayload(VerifiedStreamMessage.java:292)
     [java] at
com.sun.xml.ws.client.sei.ResponseBuilder$DocLit.readResponse(ResponseBuilder.java:523)
     [java] at
com.sun.xml.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandler.java:121)
     [java] at
com.sun.xml.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandler.java:89)
     [java] at com.sun.xml.ws.client.sei.SEIStub.invoke(SEIStub.java:118)
     [java] at $Proxy44.hello(Unknown Source)
     [java] at
com.example.hello.client.SecureHelloClient.main(SecureHelloClient.java:26)
     [java] Caused by: javax.xml.crypto.dsig.XMLSignatureException: WSS1717:
Error occurred while doing digest verification of body/payload
     [java] at
com.sun.xml.ws.security.opt.impl.incoming.processor.StreamingPayLoadDigester.accept(StreamingPayLoadDigester.java:102)
     [java] ... 15 more
     [java] Java Result: 1

My java:

$ java -version
java version "1.6.0_07"
Java(TM) SE Runtime Environment (build 1.6.0_07-b06)
Java HotSpot(TM) Client VM (build 10.0-b23, mixed mode)

I am using glassfish "glassfish-installer-v2ur2-b04-linux.jar"

And as I said, I have had the same issue on Windows as well. (I don't remember
exact versions of java and glassfish then.)

What more can I do to help you out?

One thing I was thinking about doing is to modify metro so it writes the
canonicalized XML to a couple of files, so I can see what has changed between
the signature generation and signature verification. I downloaded the metro
source, but it was not immediately obvious to me how to do the modification.
Large parts of the source consisted of zipped distributions of other code, so I
gave up on that then. If you would think that this would be helpful, let me
know, and advise me to where the signature is generated and verified in the
metro source.


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@...
For additional commands, e-mail: issues-help@...

[ashutoshshahi]

https://wsit.dev.java.net/issues/show_bug.cgi?id=960






------- Additional comments from ashutoshshahi@... Mon Jul 14 11:35:05 +0000 2008 -------
I tried the sample with both JDK 5 and 6, and it worked for me. Security sources
are from xwss project. You can check out the sources from xwss.dev.java.net.

If any extra namespaces are added in the xml and they are unused, they will be
removed during canonicalization, before signature verification, so it should not
matter.

Ashutosh

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@...
For additional commands, e-mail: issues-help@...

[capsicum]

https://wsit.dev.java.net/issues/show_bug.cgi?id=960






------- Additional comments from capsicum@... Mon Jul 14 15:02:58 +0000 2008 -------
Hello again. I modified the xwss jar so it writes the canonicalized XML to files
in /tmp which I then had a look at. It looks like it is added namespace
declarations which are braking the signature. Here are two of the files which
were created, which I suspect should be identical:

First file:

<S:Body xmlns:S="http://schemas.xmlsoap.org/soap/envelope/"
xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd"
wsu:Id="_5006"><ns3:HelloResponse
xmlns:ns3="http://example.com/helloService/hello"><ns2:Message
xmlns:ns2="http://example.com/helloService/hellomsg"><ns2:Part1>Hello</ns2:Part1><xacml-context:Response
xmlns:xacml-context="urn:oasis:names:tc:xacml:3.0:core:schema:wd-05">
<xacml-context:Result>
  <xacml-context:Decision>Permit</xacml-context:Decision>
  <xacml-context:Status>
    <xacml-context:StatusCode
Value="urn:oasis:names:tc:xacml:1.0:status:ok"></xacml-context:StatusCode>
  </xacml-context:Status>
</xacml-context:Result>
</xacml-context:Response><hemsg:Part3
xmlns:hemsg="http://example.com/helloService/hellomsg"><ThisElementMatters>
<xacml-ctx:Request xmlns:xacml-ctx="urn:oasis:names:tc:xacml:3.0:core:schema:wd-05">
  <xacml-ctx:Attributes
Category="urn:oasis:names:tc:xacml:1.0:subject-category:access-subject">
    <xacml-ctx:Attribute
AttributeId="urn:oasis:names:tc:xacml:1.0:subject:subject-id">
      <xacml-ctx:AttributeValue
DataType="http://www.w3.org/2001/XMLSchema#string">ytjytjy</xacml-ctx:AttributeValue>
    </xacml-ctx:Attribute>
  </xacml-ctx:Attributes>
</xacml-ctx:Request>
</ThisElementMatters></hemsg:Part3></ns2:Message></ns3:HelloResponse></S:Body>

The second file:

<S:Body xmlns:S="http://schemas.xmlsoap.org/soap/envelope/"
xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd"
wsu:Id="_5006"><ns3:HelloResponse
xmlns:ns3="http://example.com/helloService/hello"><ns2:Message
xmlns:ns2="http://example.com/helloService/hellomsg"><ns2:Part1>Hello</ns2:Part1><xacml-context:Response
xmlns:xacml-context="urn:oasis:names:tc:xacml:3.0:core:schema:wd-05">
<xacml-context:Result>
  <xacml-context:Decision>Permit</xacml-context:Decision>
  <xacml-context:Status>
    <xacml-context:StatusCode
Value="urn:oasis:names:tc:xacml:1.0:status:ok"></xacml-context:StatusCode>
  </xacml-context:Status>
</xacml-context:Result>
</xacml-context:Response><hemsg:Part3
xmlns:hemsg="http://example.com/helloService/hellomsg"><ThisElementMatters xmlns="">
<xacml-ctx:Request xmlns:xacml-ctx="urn:oasis:names:tc:xacml:3.0:core:schema:wd-05">
  <xacml-ctx:Attributes
Category="urn:oasis:names:tc:xacml:1.0:subject-category:access-subject">
    <xacml-ctx:Attribute
AttributeId="urn:oasis:names:tc:xacml:1.0:subject:subject-id">
      <xacml-ctx:AttributeValue
DataType="http://www.w3.org/2001/XMLSchema#string">ytjytjy</xacml-ctx:AttributeValue>
    </xacml-ctx:Attribute>
  </xacml-ctx:Attributes>
</xacml-ctx:Request>
</ThisElementMatters></hemsg:Part3></ns2:Message></ns3:HelloResponse></S:Body>

The only difference is the added declaration 'xmlns=""' to the
<ThisElementMatters> element.

After I saw this, I modified the original XML which I am sending from the
service so that it includes this declaration from the very start. Then it works!

Regarding your statement that namespace declarations do not matter, that is not
true.

It is true that exclusive c14n disregards namespaces coming from ancestor
elements into the context of the signature, but:

- namespace declarations which happen inside the signature are significant,
regardless they are used or not

- namespace prefixes are significant. I think I have observed Metro rewrite XML
so it replaces a default namespace on an element with a prefix "ns6" or such.

If my application would have signed the content which is included in the
message, it would be broken by metro in transit.

Now it appears that metro modifies the message before it is signed, so in most
cases the signature works fine. Here I seem to have discovered a case where a
modification is made after the signature is made.

My example is a bit pathological though. It uses a null default namespace, which
is correct, but is a bit tricky to handle when the XML is included in another
context.

To fix it, the S:Body element should declare 'xmlns=""' so that any XML which
looks like this (uses a null default namespace which has not been declared
inside the included XML) will work. If the included XML declares it's own
default namespace, then the fix would still be ok since the included declaration
will overwrite the one in S:Body.

But I still don't understand why you don't see the error. Are you sure you ran
"ant secure-war" and "ant run-secureclient", not just the plain service I also
included in the test case?


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@...
For additional commands, e-mail: issues-help@...

[ashutoshshahi]

https://wsit.dev.java.net/issues/show_bug.cgi?id=960



User ashutoshshahi changed the following:

                What    |Old value                 |New value
================================================================================
        Target milestone|1.4                       |1.5
--------------------------------------------------------------------------------




------- Additional comments from ashutoshshahi@... Mon Sep  8 09:56:44 +0000 2008 -------
target for 1.5 release

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@...
For additional commands, e-mail: issues-help@...