Threads locked while calling SmbFile.getSecurity

View: New views
7 Messages — Rating Filter:   Alert me  

Threads locked while calling SmbFile.getSecurity

by Marc Brette-2 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

I have an issue where 16 threads calling SmbFile.getSecurity are blocked. When I have only 15 threads, it works fine (note that each thread accesses a different document, but on the same server).
I use JCIFS 1.2.21. And tested on different shared drives (with and without credentials). I initially tested with 1.2.18 with a more complex example so I guess it s not specific to the JCIFS version.
Is it something known ? I saw some thread talking about deadlock in jcifs but it seemed to be in situation unlikely to occur in reality.

Here is my code:
import java.util.List;
import java.util.Arrays;

import jcifs.smb.SmbFile;
import jcifs.smb.NtlmPasswordAuthentication;

public class TestDeadlock
{
   static
   {
      System.setProperty("java.protocol.handler.pkgs", "jcifs");
   }

   public static void main(String[] args) throws InterruptedException
   {
      final List<String> dataset = Arrays.asList(
          "smb://frgr1dev1/Y_Drive/Documentation/javadocs/ant/ant 1.6.2 api/org/apache/tools/zip/package-frame.html"
          , "smb://frgr1dev1/Y_Drive/Documentation/javadocs/ant/ant 1.6.2 api/org/apache/tools/ant/taskdefs/optional/ejb/BorlandDeploymentTool.html"
          , "smb://frgr1dev1/Y_Drive/Documentation/javadocs/ant/ant 1.6.2 api/org/apache/tools/ant/types/Reference.html"
          , "smb://frgr1dev1/Y_Drive/Documentation/javadocs/jdk/jdk1.3.1/javax/rmi/package-use.html"
          , "smb://frgr1dev1/Y_Drive/Documentation/javadocs/jdk/jdk1.3.1/java/io/FilterOutputStream.html"
          , "smb://frgr1dev1/Y_Drive/Documentation/javadocs/jdk/jdk1.3.1/javax/accessibility/AccessibleTableModelChange.html"
          , "smb://frgr1dev1/Y_Drive/Documentation/javadocs/jdk/jdk1.4.2/org/omg/stub/java/"
          , "smb://frgr1dev1/Y_Drive/Documentation/javadocs/jdk/jdk1.3.1/javax/transaction/InvalidTransactionException.html"
          , "smb://frgr1dev1/Y_Drive/Documentation/javadocs/ant/ant 1.6.2 api/org/apache/tools/ant/taskdefs/optional/junit/DOMUtil.NodeListImpl.html"
          , "smb://frgr1dev1/Y_Drive/Documentation/javadocs/jdk/jdk1.4.2/java/awt/geom/AffineTransform.html"
          , "smb://frgr1dev1/Y_Drive/Documentation/javadocs/jdk/jdk1.3.1/javax/naming/ldap/package-frame.html"
          , "smb://frgr1dev1/Y_Drive/Documentation/javadocs/jdk/jdk1.3.1/java/awt/Shape.html"
          , "smb://frgr1dev1/Y_Drive/Documentation/javadocs/jdk/jdk1.4.2/javax/transaction/xa/package-summary.html"
          , "smb://frgr1dev1/Y_Drive/Documentation/javadocs/ant/ant 1.6.2 api/org/apache/tools/ant/taskdefs/optional/script/package-summary.html"
          , "smb://frgr1dev1/Y_Drive/Documentation/javadocs/jdk/jdk1.4.2/javax/net/ServerSocketFactory.html");

      int start = 0;
      int end = 15;
      Thread[] workers = new Thread[end - start];
      for (int i = start; i < end; i++)
      {
         final String url = dataset.get(i);
         workers[i - start] = new Thread(new Runnable()
         {
            public void run()
            {
               try
               {
                  SmbFile f = new SmbFile(url);
                  System.out.println(url + " security = " + f.getSecurity(true)); //This trace never shows up

               } catch (Throwable e)
               {
                  e.printStackTrace();
               }
            }
         });
      }

      for (Thread worker : workers)
      {
         worker.start();
      }

      for (Thread worker : workers)
      {
         worker.join();
      }
   }
}

Here is an extract of the threads stacktrace dump:

"Thread-4" prio=6 tid=0x0033dde8 nid=0x64d4 in Object.wait() [0x031bf000..0x031bf9e8]
    at java.lang.Object.wait(Native Method)
    - waiting on <0x2465e260> (a [Ljava.lang.Object;)
    at java.lang.Object.wait(Object.java:474)
    at jcifs.smb.BufferCache.getBuffers(BufferCache.java:55)
    - locked <0x2465e260> (a [Ljava.lang.Object;)
    at jcifs.smb.SmbTransport.send(SmbTransport.java:547)
    at jcifs.smb.SmbSession.send(SmbSession.java:239)
    - locked <0x2467f780> (a java.lang.Object)
    at jcifs.smb.SmbTree.send(SmbTree.java:109)
    at jcifs.smb.SmbFile.send(SmbFile.java:695)
    at jcifs.smb.TransactNamedPipeOutputStream.write(TransactNamedPipeOutputStream.java:65)
    at jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:59)
    at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:164)
    at jcifs.dcerpc.msrpc.LsaPolicyHandle.<init>(LsaPolicyHandle.java:34)
    at jcifs.smb.SID.resolveSids0(SID.java:136)
    at jcifs.smb.SID.resolveSids(SID.java:179)
    at jcifs.smb.SmbFile.processAces(SmbFile.java:2774)
    at jcifs.smb.SmbFile.getSecurity(SmbFile.java:2808)
    at TestDeadlock$1.run(TestDeadlock.java:84)
    at java.lang.Thread.run(Thread.java:595)

"Thread-3" prio=6 tid=0x0033dc60 nid=0x4494 waiting for monitor entry [0x0317f000..0x0317fce8]
    at jcifs.smb.SmbTree.treeConnect(SmbTree.java:128)
    - waiting to lock <0x2467f780> (a java.lang.Object)
    at jcifs.smb.SmbTree.send(SmbTree.java:64)
    at jcifs.smb.SmbFile.send(SmbFile.java:695)
    at jcifs.smb.TransactNamedPipeOutputStream.write(TransactNamedPipeOutputStream.java:65)
    at jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:59)
    at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:164)
    at jcifs.dcerpc.msrpc.LsaPolicyHandle.<init>(LsaPolicyHandle.java:34)
    at jcifs.smb.SID.resolveSids0(SID.java:136)
    at jcifs.smb.SID.resolveSids(SID.java:179)
    at jcifs.smb.SmbFile.processAces(SmbFile.java:2774)
    at jcifs.smb.SmbFile.getSecurity(SmbFile.java:2808)
    at TestDeadlock$1.run(TestDeadlock.java:84)
    at java.lang.Thread.run(Thread.java:595)

... other threads stack traces are similar to "Thread-3" above.

Regards
-- Marc




Re: Threads locked while calling SmbFile.getSecurity

by Marc Brette-2 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Small change: it is not 16 threads but 15 threads as reflected in the code example.

On Wed, Jun 4, 2008 at 4:14 PM, Marc Brette <marc.brette@...> wrote:
I have an issue where 16 threads calling SmbFile.getSecurity are blocked. When I have only 15 threads, it works fine (note that each thread accesses a different document, but on the same server).
I use JCIFS 1.2.21. And tested on different shared drives (with and without credentials). I initially tested with 1.2.18 with a more complex example so I guess it s not specific to the JCIFS version.
Is it something known ? I saw some thread talking about deadlock in jcifs but it seemed to be in situation unlikely to occur in reality.

Here is my code:
import java.util.List;
import java.util.Arrays;

import jcifs.smb.SmbFile;
import jcifs.smb.NtlmPasswordAuthentication;

public class TestDeadlock
{
   static
   {
      System.setProperty("java.protocol.handler.pkgs", "jcifs");
   }

   public static void main(String[] args) throws InterruptedException
   {
      final List<String> dataset = Arrays.asList(
          "smb://frgr1dev1/Y_Drive/Documentation/javadocs/ant/ant 1.6.2 api/org/apache/tools/zip/package-frame.html"
          , "smb://frgr1dev1/Y_Drive/Documentation/javadocs/ant/ant 1.6.2 api/org/apache/tools/ant/taskdefs/optional/ejb/BorlandDeploymentTool.html"
          , "smb://frgr1dev1/Y_Drive/Documentation/javadocs/ant/ant 1.6.2 api/org/apache/tools/ant/types/Reference.html"
          , "smb://frgr1dev1/Y_Drive/Documentation/javadocs/jdk/jdk1.3.1/javax/rmi/package-use.html"
          , "smb://frgr1dev1/Y_Drive/Documentation/javadocs/jdk/jdk1.3.1/java/io/FilterOutputStream.html"
          , "smb://frgr1dev1/Y_Drive/Documentation/javadocs/jdk/jdk1.3.1/javax/accessibility/AccessibleTableModelChange.html"
          , "smb://frgr1dev1/Y_Drive/Documentation/javadocs/jdk/jdk1.4.2/org/omg/stub/java/"
          , "smb://frgr1dev1/Y_Drive/Documentation/javadocs/jdk/jdk1.3.1/javax/transaction/InvalidTransactionException.html"
          , "smb://frgr1dev1/Y_Drive/Documentation/javadocs/ant/ant 1.6.2 api/org/apache/tools/ant/taskdefs/optional/junit/DOMUtil.NodeListImpl.html"
          , "smb://frgr1dev1/Y_Drive/Documentation/javadocs/jdk/jdk1.4.2/java/awt/geom/AffineTransform.html"
          , "smb://frgr1dev1/Y_Drive/Documentation/javadocs/jdk/jdk1.3.1/javax/naming/ldap/package-frame.html"
          , "smb://frgr1dev1/Y_Drive/Documentation/javadocs/jdk/jdk1.3.1/java/awt/Shape.html"
          , "smb://frgr1dev1/Y_Drive/Documentation/javadocs/jdk/jdk1.4.2/javax/transaction/xa/package-summary.html"
          , "smb://frgr1dev1/Y_Drive/Documentation/javadocs/ant/ant 1.6.2 api/org/apache/tools/ant/taskdefs/optional/script/package-summary.html"
          , "smb://frgr1dev1/Y_Drive/Documentation/javadocs/jdk/jdk1.4.2/javax/net/ServerSocketFactory.html");

      int start = 0;
      int end = 15;
      Thread[] workers = new Thread[end - start];
      for (int i = start; i < end; i++)
      {
         final String url = dataset.get(i);
         workers[i - start] = new Thread(new Runnable()
         {
            public void run()
            {
               try
               {
                  SmbFile f = new SmbFile(url);
                  System.out.println(url + " security = " + f.getSecurity(true)); //This trace never shows up

               } catch (Throwable e)
               {
                  e.printStackTrace();
               }
            }
         });
      }

      for (Thread worker : workers)
      {
         worker.start();
      }

      for (Thread worker : workers)
      {
         worker.join();
      }
   }
}

Here is an extract of the threads stacktrace dump:

"Thread-4" prio=6 tid=0x0033dde8 nid=0x64d4 in Object.wait() [0x031bf000..0x031bf9e8]
    at java.lang.Object.wait(Native Method)
    - waiting on <0x2465e260> (a [Ljava.lang.Object;)
    at java.lang.Object.wait(Object.java:474)
    at jcifs.smb.BufferCache.getBuffers(BufferCache.java:55)
    - locked <0x2465e260> (a [Ljava.lang.Object;)
    at jcifs.smb.SmbTransport.send(SmbTransport.java:547)
    at jcifs.smb.SmbSession.send(SmbSession.java:239)
    - locked <0x2467f780> (a java.lang.Object)
    at jcifs.smb.SmbTree.send(SmbTree.java:109)
    at jcifs.smb.SmbFile.send(SmbFile.java:695)
    at jcifs.smb.TransactNamedPipeOutputStream.write(TransactNamedPipeOutputStream.java:65)
    at jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:59)
    at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:164)
    at jcifs.dcerpc.msrpc.LsaPolicyHandle.<init>(LsaPolicyHandle.java:34)
    at jcifs.smb.SID.resolveSids0(SID.java:136)
    at jcifs.smb.SID.resolveSids(SID.java:179)
    at jcifs.smb.SmbFile.processAces(SmbFile.java:2774)
    at jcifs.smb.SmbFile.getSecurity(SmbFile.java:2808)
    at TestDeadlock$1.run(TestDeadlock.java:84)
    at java.lang.Thread.run(Thread.java:595)

"Thread-3" prio=6 tid=0x0033dc60 nid=0x4494 waiting for monitor entry [0x0317f000..0x0317fce8]
    at jcifs.smb.SmbTree.treeConnect(SmbTree.java:128)
    - waiting to lock <0x2467f780> (a java.lang.Object)
    at jcifs.smb.SmbTree.send(SmbTree.java:64)
    at jcifs.smb.SmbFile.send(SmbFile.java:695)
    at jcifs.smb.TransactNamedPipeOutputStream.write(TransactNamedPipeOutputStream.java:65)
    at jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:59)
    at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:164)
    at jcifs.dcerpc.msrpc.LsaPolicyHandle.<init>(LsaPolicyHandle.java:34)
    at jcifs.smb.SID.resolveSids0(SID.java:136)
    at jcifs.smb.SID.resolveSids(SID.java:179)
    at jcifs.smb.SmbFile.processAces(SmbFile.java:2774)
    at jcifs.smb.SmbFile.getSecurity(SmbFile.java:2808)
    at TestDeadlock$1.run(TestDeadlock.java:84)
    at java.lang.Thread.run(Thread.java:595)

... other threads stack traces are similar to "Thread-3" above.

Regards
-- Marc





Re: Threads locked while calling SmbFile.getSecurity

by Michael B Allen :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

I think you're hitting jcifs.smb.maxBuffers which defaults to 16.

But I have to wonder how you're getting into that scenario and if
increasing that value is the right thing to do.

Mike

On 6/4/08, Marc Brette <marc.brette@...> wrote:

> I have an issue where 16 threads calling SmbFile.getSecurity are blocked.
> When I have only 15 threads, it works fine (note that each thread accesses a
> different document, but on the same server).
> I use JCIFS 1.2.21. And tested on different shared drives (with and without
> credentials). I initially tested with 1.2.18 with a more complex example so
> I guess it s not specific to the JCIFS version.
>  Is it something known ? I saw some thread talking about deadlock in jcifs
> but it seemed to be in situation unlikely to occur in reality.
>
> Here is my code:
> import java.util.List;
> import java.util.Arrays;
>
> import jcifs.smb.SmbFile;
> import jcifs.smb.NtlmPasswordAuthentication;
>
> public class TestDeadlock
> {
>    static
>    {
>       System.setProperty("java.protocol.handler.pkgs",
> "jcifs");
>    }
>
>    public static void main(String[] args) throws InterruptedException
>     {
>       final List<String> dataset = Arrays.asList(
>
> "smb://frgr1dev1/Y_Drive/Documentation/javadocs/ant/ant
> 1.6.2 api/org/apache/tools/zip/package-frame.html"
>           ,
> "smb://frgr1dev1/Y_Drive/Documentation/javadocs/ant/ant
> 1.6.2
> api/org/apache/tools/ant/taskdefs/optional/ejb/BorlandDeploymentTool.html"
>            ,
> "smb://frgr1dev1/Y_Drive/Documentation/javadocs/ant/ant
> 1.6.2 api/org/apache/tools/ant/types/Reference.html"
>           ,
> "smb://frgr1dev1/Y_Drive/Documentation/javadocs/jdk/jdk1.3.1/javax/rmi/package-use.html"
>            ,
> "smb://frgr1dev1/Y_Drive/Documentation/javadocs/jdk/jdk1.3.1/java/io/FilterOutputStream.html"
>           ,
> "smb://frgr1dev1/Y_Drive/Documentation/javadocs/jdk/jdk1.3.1/javax/accessibility/AccessibleTableModelChange.html"
>            ,
> "smb://frgr1dev1/Y_Drive/Documentation/javadocs/jdk/jdk1.4.2/org/omg/stub/java/"
>           ,
> "smb://frgr1dev1/Y_Drive/Documentation/javadocs/jdk/jdk1.3.1/javax/transaction/InvalidTransactionException.html"
>            ,
> "smb://frgr1dev1/Y_Drive/Documentation/javadocs/ant/ant
> 1.6.2
> api/org/apache/tools/ant/taskdefs/optional/junit/DOMUtil.NodeListImpl.html"
>           ,
> "smb://frgr1dev1/Y_Drive/Documentation/javadocs/jdk/jdk1.4.2/java/awt/geom/AffineTransform.html"
>            ,
> "smb://frgr1dev1/Y_Drive/Documentation/javadocs/jdk/jdk1.3.1/javax/naming/ldap/package-frame.html"
>           ,
> "smb://frgr1dev1/Y_Drive/Documentation/javadocs/jdk/jdk1.3.1/java/awt/Shape.html"
>            ,
> "smb://frgr1dev1/Y_Drive/Documentation/javadocs/jdk/jdk1.4.2/javax/transaction/xa/package-summary.html"
>           ,
> "smb://frgr1dev1/Y_Drive/Documentation/javadocs/ant/ant
> 1.6.2
> api/org/apache/tools/ant/taskdefs/optional/script/package-summary.html"
>            ,
> "smb://frgr1dev1/Y_Drive/Documentation/javadocs/jdk/jdk1.4.2/javax/net/ServerSocketFactory.html");
>
>       int start = 0;
>       int end = 15;
>       Thread[] workers = new Thread[end - start];
>        for (int i = start; i < end; i++)
>       {
>          final String url = dataset.get(i);
>          workers[i - start] = new Thread(new Runnable()
>          {
>             public void run()
>             {
>                 try
>                {
>                   SmbFile f = new SmbFile(url);
>                   System.out.println(url + " security = " +
> f.getSecurity(true)); //This trace never shows up
>
>                } catch (Throwable e)
>                 {
>                   e.printStackTrace();
>                }
>             }
>          });
>       }
>
>       for (Thread worker : workers)
>       {
>          worker.start();
>       }
>
>        for (Thread worker : workers)
>       {
>          worker.join();
>       }
>    }
> }
>
> Here is an extract of the threads stacktrace dump:
>
> "Thread-4" prio=6 tid=0x0033dde8 nid=0x64d4 in Object.wait()
> [0x031bf000..0x031bf9e8]
>      at java.lang.Object.wait(Native Method)
>     - waiting on <0x2465e260> (a [Ljava.lang.Object;)
>     at java.lang.Object.wait(Object.java:474)
>     at
> jcifs.smb.BufferCache.getBuffers(BufferCache.java:55)
>      - locked <0x2465e260> (a [Ljava.lang.Object;)
>     at jcifs.smb.SmbTransport.send(SmbTransport.java:547)
>     at jcifs.smb.SmbSession.send(SmbSession.java:239)
>     - locked <0x2467f780> (a java.lang.Object)
>      at jcifs.smb.SmbTree.send(SmbTree.java:109)
>     at jcifs.smb.SmbFile.send(SmbFile.java:695)
>     at
> jcifs.smb.TransactNamedPipeOutputStream.write(TransactNamedPipeOutputStream.java:65)
>     at
> jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:59)
>      at
> jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:164)
>     at
> jcifs.dcerpc.msrpc.LsaPolicyHandle.<init>(LsaPolicyHandle.java:34)
>     at jcifs.smb.SID.resolveSids0(SID.java:136)
>     at jcifs.smb.SID.resolveSids(SID.java:179)
>      at jcifs.smb.SmbFile.processAces(SmbFile.java:2774)
>     at jcifs.smb.SmbFile.getSecurity(SmbFile.java:2808)
>     at TestDeadlock$1.run(TestDeadlock.java:84)
>     at java.lang.Thread.run(Thread.java:595)
>
>  "Thread-3" prio=6 tid=0x0033dc60 nid=0x4494 waiting for monitor entry
> [0x0317f000..0x0317fce8]
>     at jcifs.smb.SmbTree.treeConnect(SmbTree.java:128)
>     - waiting to lock <0x2467f780> (a java.lang.Object)
>      at jcifs.smb.SmbTree.send(SmbTree.java:64)
>     at jcifs.smb.SmbFile.send(SmbFile.java:695)
>     at
> jcifs.smb.TransactNamedPipeOutputStream.write(TransactNamedPipeOutputStream.java:65)
>     at
> jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:59)
>      at
> jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:164)
>     at
> jcifs.dcerpc.msrpc.LsaPolicyHandle.<init>(LsaPolicyHandle.java:34)
>     at jcifs.smb.SID.resolveSids0(SID.java:136)
>     at jcifs.smb.SID.resolveSids(SID.java:179)
>      at jcifs.smb.SmbFile.processAces(SmbFile.java:2774)
>     at jcifs.smb.SmbFile.getSecurity(SmbFile.java:2808)
>     at TestDeadlock$1.run(TestDeadlock.java:84)
>     at java.lang.Thread.run(Thread.java:595)
>
>  ... other threads stack traces are similar to "Thread-3" above.
>
> Regards
> -- Marc
>
>
>
>


--
Michael B Allen
PHP Active Directory SPNEGO SSO
http://www.ioplex.com/

Re: Threads locked while calling SmbFile.getSecurity

by Marc Brette-2 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hi Mike,
On another machine, I actually hit the issue with only 14 threads.
It seems this is machine dependent.
Do you think about a work-around so that I could avoid this limit ?

Note: I didn't try the new versions you shipped since, but it didn't seem to solve the issue.

--Marc

On Wed, Jun 4, 2008 at 6:58 PM, Michael B Allen <ioplex@...> wrote:
I think you're hitting jcifs.smb.maxBuffers which defaults to 16.

But I have to wonder how you're getting into that scenario and if
increasing that value is the right thing to do.

Mike

On 6/4/08, Marc Brette <marc.brette@...> wrote:
> I have an issue where 16 threads calling SmbFile.getSecurity are blocked.
> When I have only 15 threads, it works fine (note that each thread accesses a
> different document, but on the same server).
> I use JCIFS 1.2.21. And tested on different shared drives (with and without
> credentials). I initially tested with 1.2.18 with a more complex example so
> I guess it s not specific to the JCIFS version.
>  Is it something known ? I saw some thread talking about deadlock in jcifs
> but it seemed to be in situation unlikely to occur in reality.
>
> Here is my code:
> import java.util.List;
> import java.util.Arrays;
>
> import jcifs.smb.SmbFile;
> import jcifs.smb.NtlmPasswordAuthentication;
>
> public class TestDeadlock
> {
>    static
>    {
>       System.setProperty("java.protocol.handler.pkgs",
> "jcifs");
>    }
>
>    public static void main(String[] args) throws InterruptedException
>     {
>       final List<String> dataset = Arrays.asList(
>
> "smb://frgr1dev1/Y_Drive/Documentation/javadocs/ant/ant
> 1.6.2 api/org/apache/tools/zip/package-frame.html"
>           ,
> "smb://frgr1dev1/Y_Drive/Documentation/javadocs/ant/ant
> 1.6.2
> api/org/apache/tools/ant/taskdefs/optional/ejb/BorlandDeploymentTool.html"
>            ,
> "smb://frgr1dev1/Y_Drive/Documentation/javadocs/ant/ant
> 1.6.2 api/org/apache/tools/ant/types/Reference.html"
>           ,
> "smb://frgr1dev1/Y_Drive/Documentation/javadocs/jdk/jdk1.3.1/javax/rmi/package-use.html"
>            ,
> "smb://frgr1dev1/Y_Drive/Documentation/javadocs/jdk/jdk1.3.1/java/io/FilterOutputStream.html"
>           ,
> "smb://frgr1dev1/Y_Drive/Documentation/javadocs/jdk/jdk1.3.1/javax/accessibility/AccessibleTableModelChange.html"
>            ,
> "smb://frgr1dev1/Y_Drive/Documentation/javadocs/jdk/jdk1.4.2/org/omg/stub/java/"
>           ,
> "smb://frgr1dev1/Y_Drive/Documentation/javadocs/jdk/jdk1.3.1/javax/transaction/InvalidTransactionException.html"
>            ,
> "smb://frgr1dev1/Y_Drive/Documentation/javadocs/ant/ant
> 1.6.2
> api/org/apache/tools/ant/taskdefs/optional/junit/DOMUtil.NodeListImpl.html"
>           ,
> "smb://frgr1dev1/Y_Drive/Documentation/javadocs/jdk/jdk1.4.2/java/awt/geom/AffineTransform.html"
>            ,
> "smb://frgr1dev1/Y_Drive/Documentation/javadocs/jdk/jdk1.3.1/javax/naming/ldap/package-frame.html"
>           ,
> "smb://frgr1dev1/Y_Drive/Documentation/javadocs/jdk/jdk1.3.1/java/awt/Shape.html"
>            ,
> "smb://frgr1dev1/Y_Drive/Documentation/javadocs/jdk/jdk1.4.2/javax/transaction/xa/package-summary.html"
>           ,
> "smb://frgr1dev1/Y_Drive/Documentation/javadocs/ant/ant
> 1.6.2
> api/org/apache/tools/ant/taskdefs/optional/script/package-summary.html"
>            ,
> "smb://frgr1dev1/Y_Drive/Documentation/javadocs/jdk/jdk1.4.2/javax/net/ServerSocketFactory.html");
>
>       int start = 0;
>       int end = 15;
>       Thread[] workers = new Thread[end - start];
>        for (int i = start; i < end; i++)
>       {
>          final String url = dataset.get(i);
>          workers[i - start] = new Thread(new Runnable()
>          {
>             public void run()
>             {
>                 try
>                {
>                   SmbFile f = new SmbFile(url);
>                   System.out.println(url + " security = " +
> f.getSecurity(true)); //This trace never shows up
>
>                } catch (Throwable e)
>                 {
>                   e.printStackTrace();
>                }
>             }
>          });
>       }
>
>       for (Thread worker : workers)
>       {
>          worker.start();
>       }
>
>        for (Thread worker : workers)
>       {
>          worker.join();
>       }
>    }
> }
>
> Here is an extract of the threads stacktrace dump:
>
> "Thread-4" prio=6 tid=0x0033dde8 nid=0x64d4 in Object.wait()
> [0x031bf000..0x031bf9e8]
>      at java.lang.Object.wait(Native Method)
>     - waiting on <0x2465e260> (a [Ljava.lang.Object;)
>     at java.lang.Object.wait(Object.java:474)
>     at
> jcifs.smb.BufferCache.getBuffers(BufferCache.java:55)
>      - locked <0x2465e260> (a [Ljava.lang.Object;)
>     at jcifs.smb.SmbTransport.send(SmbTransport.java:547)
>     at jcifs.smb.SmbSession.send(SmbSession.java:239)
>     - locked <0x2467f780> (a java.lang.Object)
>      at jcifs.smb.SmbTree.send(SmbTree.java:109)
>     at jcifs.smb.SmbFile.send(SmbFile.java:695)
>     at
> jcifs.smb.TransactNamedPipeOutputStream.write(TransactNamedPipeOutputStream.java:65)
>     at
> jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:59)
>      at
> jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:164)
>     at
> jcifs.dcerpc.msrpc.LsaPolicyHandle.<init>(LsaPolicyHandle.java:34)
>     at jcifs.smb.SID.resolveSids0(SID.java:136)
>     at jcifs.smb.SID.resolveSids(SID.java:179)
>      at jcifs.smb.SmbFile.processAces(SmbFile.java:2774)
>     at jcifs.smb.SmbFile.getSecurity(SmbFile.java:2808)
>     at TestDeadlock$1.run(TestDeadlock.java:84)
>     at java.lang.Thread.run(Thread.java:595)
>
>  "Thread-3" prio=6 tid=0x0033dc60 nid=0x4494 waiting for monitor entry
> [0x0317f000..0x0317fce8]
>     at jcifs.smb.SmbTree.treeConnect(SmbTree.java:128)
>     - waiting to lock <0x2467f780> (a java.lang.Object)
>      at jcifs.smb.SmbTree.send(SmbTree.java:64)
>     at jcifs.smb.SmbFile.send(SmbFile.java:695)
>     at
> jcifs.smb.TransactNamedPipeOutputStream.write(TransactNamedPipeOutputStream.java:65)
>     at
> jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:59)
>      at
> jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:164)
>     at
> jcifs.dcerpc.msrpc.LsaPolicyHandle.<init>(LsaPolicyHandle.java:34)
>     at jcifs.smb.SID.resolveSids0(SID.java:136)
>     at jcifs.smb.SID.resolveSids(SID.java:179)
>      at jcifs.smb.SmbFile.processAces(SmbFile.java:2774)
>     at jcifs.smb.SmbFile.getSecurity(SmbFile.java:2808)
>     at TestDeadlock$1.run(TestDeadlock.java:84)
>     at java.lang.Thread.run(Thread.java:595)
>
>  ... other threads stack traces are similar to "Thread-3" above.
>
> Regards
> -- Marc
>
>
>
>


--
Michael B Allen
PHP Active Directory SPNEGO SSO
http://www.ioplex.com/


Re: Threads locked while calling SmbFile.getSecurity

by Marc Brette-2 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

I resend this email as it didn't appear in the list:

Hi Mike,
On another machine, I actually hit the issue with only 14 threads.
[update: and even 10 threads]
It seems this is machine dependent.
Do you think about a work-around so that I could avoid this limit ?

Note: I didn't try the new versions you shipped since, but it didn't
seem to solve the issue.



On 7/29/08, Marc Brette <marc.brette@...> wrote:

> Hi Mike,
> On another machine, I actually hit the issue with only 14 threads.
> It seems this is machine dependent.
> Do you think about a work-around so that I could avoid this limit ?
>
> Note: I didn't try the new versions you shipped since, but it didn't seem to
> solve the issue.
>
> --Marc
>
> On Wed, Jun 4, 2008 at 6:58 PM, Michael B Allen <ioplex@...> wrote:
>
>> I think you're hitting jcifs.smb.maxBuffers which defaults to 16.
>>
>> But I have to wonder how you're getting into that scenario and if
>> increasing that value is the right thing to do.
>>
>> Mike
>>
>> On 6/4/08, Marc Brette <marc.brette@...> wrote:
>> > I have an issue where 16 threads calling SmbFile.getSecurity are
>> > blocked.
>> > When I have only 15 threads, it works fine (note that each thread
>> accesses a
>> > different document, but on the same server).
>> > I use JCIFS 1.2.21. And tested on different shared drives (with and
>> without
>> > credentials). I initially tested with 1.2.18 with a more complex example
>> so
>> > I guess it s not specific to the JCIFS version.
>> >  Is it something known ? I saw some thread talking about deadlock in
>> jcifs
>> > but it seemed to be in situation unlikely to occur in reality.
>> >
>> > Here is my code:
>> > import java.util.List;
>> > import java.util.Arrays;
>> >
>> > import jcifs.smb.SmbFile;
>> > import jcifs.smb.NtlmPasswordAuthentication;
>> >
>> > public class TestDeadlock
>> > {
>> >    static
>> >    {
>> >       System.setProperty("java.protocol.handler.pkgs",
>> > "jcifs");
>> >    }
>> >
>> >    public static void main(String[] args) throws InterruptedException
>> >     {
>> >       final List<String> dataset = Arrays.asList(
>> >
>> > "smb://frgr1dev1/Y_Drive/Documentation/javadocs/ant/ant
>> > 1.6.2 api/org/apache/tools/zip/package-frame.html"
>> >           ,
>> > "smb://frgr1dev1/Y_Drive/Documentation/javadocs/ant/ant
>> > 1.6.2
>> >
>> api/org/apache/tools/ant/taskdefs/optional/ejb/BorlandDeploymentTool.html"
>> >            ,
>> > "smb://frgr1dev1/Y_Drive/Documentation/javadocs/ant/ant
>> > 1.6.2 api/org/apache/tools/ant/types/Reference.html"
>> >           ,
>> >
>> "smb://frgr1dev1/Y_Drive/Documentation/javadocs/jdk/jdk1.3.1/javax/rmi/package-use.html"
>> >            ,
>> >
>> "smb://frgr1dev1/Y_Drive/Documentation/javadocs/jdk/jdk1.3.1/java/io/FilterOutputStream.html"
>> >           ,
>> >
>> "smb://frgr1dev1/Y_Drive/Documentation/javadocs/jdk/jdk1.3.1/javax/accessibility/AccessibleTableModelChange.html"
>> >            ,
>> >
>> "smb://frgr1dev1/Y_Drive/Documentation/javadocs/jdk/jdk1.4.2/org/omg/stub/java/"
>> >           ,
>> >
>> "smb://frgr1dev1/Y_Drive/Documentation/javadocs/jdk/jdk1.3.1/javax/transaction/InvalidTransactionException.html"
>> >            ,
>> > "smb://frgr1dev1/Y_Drive/Documentation/javadocs/ant/ant
>> > 1.6.2
>> >
>> api/org/apache/tools/ant/taskdefs/optional/junit/DOMUtil.NodeListImpl.html"
>> >           ,
>> >
>> "smb://frgr1dev1/Y_Drive/Documentation/javadocs/jdk/jdk1.4.2/java/awt/geom/AffineTransform.html"
>> >            ,
>> >
>> "smb://frgr1dev1/Y_Drive/Documentation/javadocs/jdk/jdk1.3.1/javax/naming/ldap/package-frame.html"
>> >           ,
>> >
>> "smb://frgr1dev1/Y_Drive/Documentation/javadocs/jdk/jdk1.3.1/java/awt/Shape.html"
>> >            ,
>> >
>> "smb://frgr1dev1/Y_Drive/Documentation/javadocs/jdk/jdk1.4.2/javax/transaction/xa/package-summary.html"
>> >           ,
>> > "smb://frgr1dev1/Y_Drive/Documentation/javadocs/ant/ant
>> > 1.6.2
>> > api/org/apache/tools/ant/taskdefs/optional/script/package-summary.html"
>> >            ,
>> >
>> "smb://frgr1dev1/Y_Drive/Documentation/javadocs/jdk/jdk1.4.2/javax/net/ServerSocketFactory.html");
>> >
>> >       int start = 0;
>> >       int end = 15;
>> >       Thread[] workers = new Thread[end - start];
>> >        for (int i = start; i < end; i++)
>> >       {
>> >          final String url = dataset.get(i);
>> >          workers[i - start] = new Thread(new Runnable()
>> >          {
>> >             public void run()
>> >             {
>> >                 try
>> >                {
>> >                   SmbFile f = new SmbFile(url);
>> >                   System.out.println(url + " security = " +
>> > f.getSecurity(true)); //This trace never shows up
>> >
>> >                } catch (Throwable e)
>> >                 {
>> >                   e.printStackTrace();
>> >                }
>> >             }
>> >          });
>> >       }
>> >
>> >       for (Thread worker : workers)
>> >       {
>> >          worker.start();
>> >       }
>> >
>> >        for (Thread worker : workers)
>> >       {
>> >          worker.join();
>> >       }
>> >    }
>> > }
>> >
>> > Here is an extract of the threads stacktrace dump:
>> >
>> > "Thread-4" prio=6 tid=0x0033dde8 nid=0x64d4 in Object.wait()
>> > [0x031bf000..0x031bf9e8]
>> >      at java.lang.Object.wait(Native Method)
>> >     - waiting on <0x2465e260> (a [Ljava.lang.Object;)
>> >     at java.lang.Object.wait(Object.java:474)
>> >     at
>> > jcifs.smb.BufferCache.getBuffers(BufferCache.java:55)
>> >      - locked <0x2465e260> (a [Ljava.lang.Object;)
>> >     at jcifs.smb.SmbTransport.send(SmbTransport.java:547)
>> >     at jcifs.smb.SmbSession.send(SmbSession.java:239)
>> >     - locked <0x2467f780> (a java.lang.Object)
>> >      at jcifs.smb.SmbTree.send(SmbTree.java:109)
>> >     at jcifs.smb.SmbFile.send(SmbFile.java:695)
>> >     at
>> >
>> jcifs.smb.TransactNamedPipeOutputStream.write(TransactNamedPipeOutputStream.java:65)
>> >     at
>> > jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:59)
>> >      at
>> > jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:164)
>> >     at
>> > jcifs.dcerpc.msrpc.LsaPolicyHandle.<init>(LsaPolicyHandle.java:34)
>> >     at jcifs.smb.SID.resolveSids0(SID.java:136)
>> >     at jcifs.smb.SID.resolveSids(SID.java:179)
>> >      at jcifs.smb.SmbFile.processAces(SmbFile.java:2774)
>> >     at jcifs.smb.SmbFile.getSecurity(SmbFile.java:2808)
>> >     at TestDeadlock$1.run(TestDeadlock.java:84)
>> >     at java.lang.Thread.run(Thread.java:595)
>> >
>> >  "Thread-3" prio=6 tid=0x0033dc60 nid=0x4494 waiting for monitor entry
>> > [0x0317f000..0x0317fce8]
>> >     at jcifs.smb.SmbTree.treeConnect(SmbTree.java:128)
>> >     - waiting to lock <0x2467f780> (a java.lang.Object)
>> >      at jcifs.smb.SmbTree.send(SmbTree.java:64)
>> >     at jcifs.smb.SmbFile.send(SmbFile.java:695)
>> >     at
>> >
>> jcifs.smb.TransactNamedPipeOutputStream.write(TransactNamedPipeOutputStream.java:65)
>> >     at
>> > jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:59)
>> >      at
>> > jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:164)
>> >     at
>> > jcifs.dcerpc.msrpc.LsaPolicyHandle.<init>(LsaPolicyHandle.java:34)
>> >     at jcifs.smb.SID.resolveSids0(SID.java:136)
>> >     at jcifs.smb.SID.resolveSids(SID.java:179)
>> >      at jcifs.smb.SmbFile.processAces(SmbFile.java:2774)
>> >     at jcifs.smb.SmbFile.getSecurity(SmbFile.java:2808)
>> >     at TestDeadlock$1.run(TestDeadlock.java:84)
>> >     at java.lang.Thread.run(Thread.java:595)
>> >
>> >  ... other threads stack traces are similar to "Thread-3" above.
>> >
>> > Regards
>> > -- Marc
>> >
>> >
>> >
>> >
>>
>>
>> --
>> Michael B Allen
>> PHP Active Directory SPNEGO SSO
>> http://www.ioplex.com/
>>
>

Re: Threads locked while calling SmbFile.getSecurity

by Michael B Allen :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On Tue, Aug 5, 2008 at 5:45 AM, Marc Brette <marc.brette@...> wrote:

> I resend this email as it didn't appear in the list:
>
> Hi Mike,
> On another machine, I actually hit the issue with only 14 threads.
> [update: and even 10 threads]
> It seems this is machine dependent.
> Do you think about a work-around so that I could avoid this limit ?
>
> Note: I didn't try the new versions you shipped since, but it didn't
> seem to solve the issue.

If you never use more threads than twice the number of buffers then I
think  you should be fine. So if you want to use 10 threads, set
maxBuffers to 25, for 100 threads set maxBuffers to 105, ... etc.

Note that using a lot of threads does not necessarily equate to more
work getting done. Unless your network is slow, I've found that each
server can only handle about two threads crawling it at the same time.

This deadlock is related to the other connect / disconnect deadlock.
At least I think it would be easy to fix if we used a
ConditionVariable class instead of synchronized blocks.

Mike

--
Michael B Allen
PHP Active Directory SPNEGO SSO
http://www.ioplex.com/

Re: Threads locked while calling SmbFile.getSecurity

by Marc Brette-2 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Thanks. It solves my issue at least on my machine.
(Note: another work-around was to synchronize the call to getSecurity so that only one thread called it).

I also saw that a similar problem was reported here http://lists.samba.org/archive/jcifs/2007-February/007044.html and that you already answered there (I ddin't see this until today).
So thanks for taking the time to answer again.


On Tue, Aug 5, 2008 at 4:31 PM, Michael B Allen <ioplex@...> wrote:
On Tue, Aug 5, 2008 at 5:45 AM, Marc Brette <marc.brette@...> wrote:
> I resend this email as it didn't appear in the list:
>
> Hi Mike,
> On another machine, I actually hit the issue with only 14 threads.
> [update: and even 10 threads]
> It seems this is machine dependent.
> Do you think about a work-around so that I could avoid this limit ?
>
> Note: I didn't try the new versions you shipped since, but it didn't
> seem to solve the issue.

If you never use more threads than twice the number of buffers then I
think  you should be fine. So if you want to use 10 threads, set
maxBuffers to 25, for 100 threads set maxBuffers to 105, ... etc.

Note that using a lot of threads does not necessarily equate to more
work getting done. Unless your network is slow, I've found that each
server can only handle about two threads crawling it at the same time.

This deadlock is related to the other connect / disconnect deadlock.
At least I think it would be easy to fix if we used a
ConditionVariable class instead of synchronized blocks.

Mike

--
Michael B Allen
PHP Active Directory SPNEGO SSO
http://www.ioplex.com/

LightInTheBox - Buy quality products at wholesale price