Problems writing signed document to disk

[Ian Hummel-2]

Hi everyone,

Something weird is going on when I try to write my signed Document to disk...

//doc is a Document which has gone through the signing process...

Transformer xformer = TransformerFactory.newInstance().newTransformer();

DOMSource source = new DOMSource(doc);

StreamResult result = new StreamResult(new File("output.xml"));

xformer.transform(source, result);

DocumentBuilderFactory dbf2 = DocumentBuilderFactory.newInstance();

dbf2.setNamespaceAware(true);

DocumentBuilder db2 = dbf2.newDocumentBuilder();

Document doc2 = db2.parse(new File("output.xml"));

boolean res1 = CryptoUtils.verifyXmlSignature(doc);

boolean res2 = CryptoUtils.verifyXmlSignature(doc2);

System.out.println("RES 1 (raw) was " + res1);

System.out.println("RES 2 (read) was " + res2);

This code prints out:

INFO  org.apache.xml.security.signature.Reference - Verification successful for URI "#IC01"

WARN  org.apache.xml.security.signature.Reference - Verification failed for URI "#IC01"

WARN  CryptoUtils - Signature was invalid

RES 1 (raw) was true

RES 2 (read) was false

any idea why writing the DOM to disk would make subsequent reads to validate it fail?  

The only things I can think of are a) the Transformer is altering the document before writing it to disk or b) there are a lot of elements whose content is base64 encoded and making somehow they are getting mangled when written to disk?

Any help is appreciated!

- ian.

[Ian Hummel-2]

That might be, however the Transformer code to write the DOM to disk  
is right out of the xml signature tutorial, so I hope that is not the  
case...

I read through the list and saw some brief comments that suggest maybe  
one can add attributes (via setAttribute) which do not get added into  
a namespace until the document is written to disk (thus breaking the  
signature)??

Has anyone seen this before?  Any tips to debug?

http://mail-archives.apache.org/mod_mbox/xml-security-dev/200602.mbox/%3c43E3AFE8.9090300@...%3e


On Apr 4, 2008, at 2:44 AM, panther@... wrote:

> Ian,
>
> I did not try it, but maybe your output to file adds "pretty  
> printing"? This means adding white spaces which breaks a signature.
>
> Cheers,
> Jörg Bartholdt
>
>
>
>
>
> Hi everyone,
>
>
> Something weird is going on when I try to write my signed Document  
> to disk...
>
>
> //doc is a Document which has gone through the signing process...
>
> Transformer xformer =  
> TransformerFactory.newInstance().newTransformer();
> DOMSource source = new DOMSource(doc);
> StreamResult result = new StreamResult(new File("output.xml"));
> xformer.transform(source, result);
>
>
>
>
>
> DocumentBuilderFactory dbf2 = DocumentBuilderFactory.newInstance();
> dbf2.setNamespaceAware(true);
> DocumentBuilder db2 = dbf2.newDocumentBuilder();
> Document doc2 = db2.parse(new File("output.xml"));
> boolean res1 = CryptoUtils.verifyXmlSignature(doc);
> boolean res2 = CryptoUtils.verifyXmlSignature(doc2);
> System.out.println("RES 1 (raw) was " + res1);
> System.out.println("RES 2 (read) was " + res2);
>
>
>
>
> This code prints out:
>
>
> INFO  org.apache.xml.security.signature.Reference - Verification  
> successful for URI "#IC01"
> WARN  org.apache.xml.security.signature.Reference - Verification  
> failed for URI "#IC01"
> WARN  CryptoUtils - Signature was invalid
> RES 1 (raw) was true
> RES 2 (read) was false
>
>
>
>
>
>
> any idea why writing the DOM to disk would make subsequent reads to  
> validate it fail?
>
>
> The only things I can think of are a) the Transformer is altering  
> the document before writing it to disk or b) there are a lot of  
> elements whose content is base64 encoded and making somehow they are  
> getting mangled when written to disk?
>
>
> Any help is appreciated!
>
>
> - ian.

[Sean Mullan]

If you attach the signature I might be able to make a guess, but you
really need to debug it and find out what the reference's pre-digested
content is when signing and validating and then compare them to see what
is breaking the signature. Often it's a namespace issue.

--Sean

Ian Hummel wrote:

> Hi everyone,
>
> Something weird is going on when I try to write my signed Document to
> disk...
>
> //doc is a Document which has gone through the signing process...
> Transformer xformer = TransformerFactory.newInstance().newTransformer();
> DOMSource source = new DOMSource(doc);
> StreamResult result = new StreamResult(new File("output.xml"));
> xformer.transform(source, result);
>
>
> DocumentBuilderFactory dbf2 = DocumentBuilderFactory.newInstance();
> dbf2.setNamespaceAware(true);
> DocumentBuilder db2 = dbf2.newDocumentBuilder();
> Document doc2 = db2.parse(new File("output.xml"));
> boolean res1 = CryptoUtils.verifyXmlSignature(doc);
> boolean res2 = CryptoUtils.verifyXmlSignature(doc2);
> System.out.println("RES 1 (raw) was " + res1);
> System.out.println("RES 2 (read) was " + res2);
>
>
> This code prints out:
>
> INFO  org.apache.xml.security.signature.Reference - Verification
> successful for URI "#IC01"
> WARN  org.apache.xml.security.signature.Reference - Verification failed
> for URI "#IC01"
> WARN  CryptoUtils - Signature was invalid
> RES 1 (raw) was true
> RES 2 (read) was false
>
>
>
> any idea why writing the DOM to disk would make subsequent reads to
> validate it fail?  
>
> The only things I can think of are a) the Transformer is altering the
> document before writing it to disk or b) there are a lot of elements
> whose content is base64 encoded and making somehow they are getting
> mangled when written to disk?
>
> Any help is appreciated!
>
> - ian.

[Ian Hummel-2]

I think it is definitely coming down to a namespace issue... The code below writes a single element into my signature, writes the sig to disk, loads it again and tries to verify it.  Then it outputs the pre-digested element, and the parsed-from-disk element and they seem different...

DocumentBuilderFactory documentBuilderFactory = DocumentBuilderFactory.newInstance();

documentBuilderFactory.setNamespaceAware(true);

DocumentBuilder documentBuilder = documentBuilderFactory.newDocumentBuilder();

Document doc = documentBuilder.newDocument();

XMLSignature signature = new XMLSignature(doc, "", XMLSignature.ALGO_ID_SIGNATURE_RSA_SHA1);

doc.appendChild(signature.getElement());

ObjectContainer objectContainer = new ObjectContainer(doc);

Element informationCard = doc.createElementNS(NS_IDENTITY, "InformationCard");

objectContainer.appendChild(informationCard);

objectContainer.setId("IC01");

signature.appendObject(objectContainer);

Transforms transforms = new Transforms(doc);

        transforms.addTransform("http://www.w3.org/2001/10/xml-exc-c14n#");

        signature.addDocument("#IC01", transforms, Constants.ALGO_ID_DIGEST_SHA1);

X509Certificate cert = (X509Certificate) retrieveCertificate(properties);

signature.addKeyInfo(cert);

//signature.addKeyInfo(cert.getPublicKey());

PrivateKey key = (PrivateKey) retrieveKey(properties);

signature.sign(key);

Transformer xformer = TransformerFactory.newInstance().newTransformer();

DOMSource source = new DOMSource(doc);

StreamResult result = new StreamResult(new File("output.crd"));

xformer.transform(source, result);

DocumentBuilderFactory dbf2 = DocumentBuilderFactory.newInstance();

dbf2.setNamespaceAware(true);

DocumentBuilder db2 = dbf2.newDocumentBuilder();

Document doc2 = db2.parse(new File("output.crd"));

System.out.println(CryptoUtils.verifyXmlSignature(doc2));

try {

Element signatureElement = (Element) doc2.getElementsByTagNameNS(Constants.SignatureSpecNS, "Signature").item(0);

    XMLSignature signature2 = new XMLSignature(signatureElement, "");

    ObjectContainer o = signature2.getObjectItem(0);

     Element informationCard2 = (Element)o.getElement().getChildNodes().item(0);

    OutputFormat of = new OutputFormat();

    org.apache.xml.serialize.DOMSerializer ser = SerializerFactory.getSerializerFactory(Method.XML).

    makeSerializer(System.out, of).asDOMSerializer();

    org.apache.xml.serialize.DOMSerializer ser2 = SerializerFactory.getSerializerFactory(Method.XML).

    makeSerializer(System.out, of).asDOMSerializer();

    System.out.println("el1 serialized");

    ser.serialize(informationCard);

    System.out.println();

    System.out.println();

    System.out.println("el2 serialized");

    ser2.serialize(informationCard2);

} catch (XMLSecurityException e) {

}

This outputs:

DEBUG net.parityinc.jumpstart.sts.CryptoUtils - keyInfo is: org.apache.xml.security.keys.KeyInfo@f6fd54

DEBUG net.parityinc.jumpstart.sts.CryptoUtils - Public key is: Sun RSA public key, 1024 bits

  modulus: 111569399812228317974104160667778574453427703386659798836457259422563834830313005221987124756525281662856577689662339654944603341744351755854419363610184584407762310756727959272178338147873650923924948006027542968201887110182779600654460562132129929712009593703603588025273257129489044637731356622650761637013

  public exponent: 65537

WARN  org.apache.xml.security.signature.Reference - Verification failed for URI "#IC01"

WARN  net.parityinc.jumpstart.sts.CryptoUtils - Signature was invalid

false

el1 serialized

<?xml version="1.0" encoding="UTF-8"?>

<InformationCard/>

el2 serialized

<?xml version="1.0" encoding="UTF-8"?>

<InformationCard xmlns="http://schemas.xmlsoap.org/ws/2005/05/identity"/>

Am i not creating/attaching the "InformationCard" node correctly?  Why doesn't the namespace declaration show up in example 1, even though I've created the node via doc.createElementNS(NS_IDENTITY, "InformationCard");

Any help appreciated!

- ian.

On Apr 4, 2008, at 10:20 AM, Sean Mullan wrote:

If you attach the signature I might be able to make a guess, but you
really need to debug it and find out what the reference's pre-digested
content is when signing and validating and then compare them to see what
is breaking the signature. Often it's a namespace issue.

--Sean

Ian Hummel wrote:

Hi everyone,

Something weird is going on when I try to write my signed Document to

disk...

//doc is a Document which has gone through the signing process...

Transformer xformer = TransformerFactory.newInstance().newTransformer();

DOMSource source = new DOMSource(doc);

StreamResult result = new StreamResult(new File("output.xml"));

xformer.transform(source, result);

DocumentBuilderFactory dbf2 = DocumentBuilderFactory.newInstance();

dbf2.setNamespaceAware(true);

DocumentBuilder db2 = dbf2.newDocumentBuilder();

Document doc2 = db2.parse(new File("output.xml"));

boolean res1 = CryptoUtils.verifyXmlSignature(doc);

boolean res2 = CryptoUtils.verifyXmlSignature(doc2);

System.out.println("RES 1 (raw) was " + res1);

System.out.println("RES 2 (read) was " + res2);

This code prints out:

INFO  org.apache.xml.security.signature.Reference - Verification

successful for URI "#IC01"

WARN  org.apache.xml.security.signature.Reference - Verification failed

for URI "#IC01"

WARN  CryptoUtils - Signature was invalid

RES 1 (raw) was true

RES 2 (read) was false

any idea why writing the DOM to disk would make subsequent reads to

validate it fail?

The only things I can think of are a) the Transformer is altering the

document before writing it to disk or b) there are a lot of elements

whose content is base64 encoded and making somehow they are getting

mangled when written to disk?

Any help is appreciated!

- ian.

[Sean Mullan]

Ian Hummel wrote:

> This outputs:
>
> DEBUG net.parityinc.jumpstart.sts.CryptoUtils - keyInfo is:
> org.apache.xml.security.keys.KeyInfo@f6fd54
> DEBUG net.parityinc.jumpstart.sts.CryptoUtils - Public key is: Sun RSA
> public key, 1024 bits
>   modulus:
> 111569399812228317974104160667778574453427703386659798836457259422563834830313005221987124756525281662856577689662339654944603341744351755854419363610184584407762310756727959272178338147873650923924948006027542968201887110182779600654460562132129929712009593703603588025273257129489044637731356622650761637013
>   public exponent: 65537
> WARN  org.apache.xml.security.signature.Reference - Verification failed
> for URI "#IC01"
> WARN  net.parityinc.jumpstart.sts.CryptoUtils - Signature was invalid
> false
> el1 serialized
> <?xml version="1.0" encoding="UTF-8"?>
> <InformationCard/>
>
> el2 serialized
> <?xml version="1.0" encoding="UTF-8"?>
> <InformationCard xmlns="http://schemas.xmlsoap.org/ws/2005/05/identity"/>
>
>
>
> Am i not creating/attaching the "InformationCard" node correctly?  Why
> doesn't the namespace declaration show up in example 1, even though I've
> created the node via doc.createElementNS(NS_IDENTITY, "InformationCard");

You must also add the namespace attribute to the element where it is
defined using the Element.setAttributeNS method. Otherwise the namespace
is not visible when canonicalizing the element.

See: https://issues.apache.org/bugzilla/show_bug.cgi?id=41821

I have seen this same problem come up at least 1/2 dozen times. Since
this is a common mistake, I wonder if we could work around this in our
implementation as it seems xalan is able to figure out where to add the
namespace attribute even though it hasn't been explicitly added by the
application. Will look into this some more, but in the meantime, you
should have a workaround.

--Sean

[Ian Hummel-2]

Ok, thanks Sean.

I hope there is a way things can be made easier in the future... this  
issue has been a huge time sink for me :(

Thanks for the tips,

- Ian.


On Apr 7, 2008, at 3:02 PM, Sean Mullan wrote:

> Ian Hummel wrote:
>
>> This outputs:
>>
>> DEBUG net.parityinc.jumpstart.sts.CryptoUtils - keyInfo is:
>> org.apache.xml.security.keys.KeyInfo@f6fd54
>> DEBUG net.parityinc.jumpstart.sts.CryptoUtils - Public key is: Sun  
>> RSA
>> public key, 1024 bits
>>  modulus:
>> 111569399812228317974104160667778574453427703386659798836457259422563834830313005221987124756525281662856577689662339654944603341744351755854419363610184584407762310756727959272178338147873650923924948006027542968201887110182779600654460562132129929712009593703603588025273257129489044637731356622650761637013
>>  public exponent: 65537
>> WARN  org.apache.xml.security.signature.Reference - Verification  
>> failed
>> for URI "#IC01"
>> WARN  net.parityinc.jumpstart.sts.CryptoUtils - Signature was invalid
>> false
>> el1 serialized
>> <?xml version="1.0" encoding="UTF-8"?>
>> <InformationCard/>
>>
>> el2 serialized
>> <?xml version="1.0" encoding="UTF-8"?>
>> <InformationCard xmlns="http://schemas.xmlsoap.org/ws/2005/05/identity 
>> "/>
>>
>>
>>
>> Am i not creating/attaching the "InformationCard" node correctly?  
>> Why
>> doesn't the namespace declaration show up in example 1, even though  
>> I've
>> created the node via doc.createElementNS(NS_IDENTITY,  
>> "InformationCard");
>
> You must also add the namespace attribute to the element where it is
> defined using the Element.setAttributeNS method. Otherwise the  
> namespace
> is not visible when canonicalizing the element.
>
> See: https://issues.apache.org/bugzilla/show_bug.cgi?id=41821
>
> I have seen this same problem come up at least 1/2 dozen times. Since
> this is a common mistake, I wonder if we could work around this in our
> implementation as it seems xalan is able to figure out where to add  
> the
> namespace attribute even though it hasn't been explicitly added by the
> application. Will look into this some more, but in the meantime, you
> should have a workaround.
>
> --Sean