RowIterator.next() fails when hasNext() returns true

[Fitzgibbons, Brad]

I'm running into this problem occasionally resulting in the exception below.  I'm using Jackrabbit 1.4 (with core at 1.4.2).  I'm using the H2PersistenceManager and almost all configuration values are set to their released default. Document order is turned off so the query is using a LazyScoreNodeIterator.  I'm running in a transactional environment, and my entire execution flow is contained by a single, read-only, JTA transaction.  The single transaction includes executing the query and all interactions with the returned RowIterator.  If I ignore the NoSuchElementException's and continue using the iterator then there are often good results subsequent to the error.  I've noticed that when these errors occur there are often far more results than are returned by RowIterator.getSize().  I realize that getSize() is not meant to be an accurate count of all results, but in most other cases getSize() is greater than the size of the result set (this may not be helpful).  I'm using a SQL query that involves the CONTAINS constraint (so lucene) and ordering by jcr:score.  Does this warrant a bug submission?  Is there any additional information which might help?

org.springframework.web.util.NestedServletException: Request processing failed; nested exception is java.util.NoSuchElementException
        at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:535)
        at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:453)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:114)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:91)
        at com.caucho.server.dispatch.ServletFilterChain.doFilter(ServletFilterChain.java:103)
        at com.caucho.server.webapp.WebAppFilterChain.doFilter(WebAppFilterChain.java:181)
        at com.caucho.server.dispatch.ServletInvocation.service(ServletInvocation.java:266)
        at com.caucho.server.http.HttpRequest.handleRequest(HttpRequest.java:269)
        at com.caucho.server.port.TcpConnection.run(TcpConnection.java:603)
        at com.caucho.util.ThreadPool$Item.runTasks(ThreadPool.java:721)
        at com.caucho.util.ThreadPool$Item.run(ThreadPool.java:643)
        at java.lang.Thread.run(Thread.java:619)
Caused by: java.util.NoSuchElementException
        at org.apache.jackrabbit.core.query.lucene.QueryResultImpl$LazyScoreNodeIterator.getScore(QueryResultImpl.java:351)
        at org.apache.jackrabbit.core.query.lucene.RowIteratorImpl.nextRow(RowIteratorImpl.java:153)
        at com.techtarget.content.data.jcr1.ElementDaoImpl.nextElement(ElementDaoImpl.java:722)
        at com.techtarget.content.data.jcr1.PropertyIterator.next(PropertyIterator.java:47)
        at com.techtarget.content.data.jcr1.PropertyIterator.next(PropertyIterator.java:21)
        at com.techtarget.content.web.controller.ProfileController.resourceQueryAndPrint(ProfileController.java:125)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.springframework.web.bind.annotation.support.HandlerMethodInvoker.doInvokeMethod(HandlerMethodInvoker.java:305)
        at org.springframework.web.bind.annotation.support.HandlerMethodInvoker.invokeHandlerMethod(HandlerMethodInvoker.java:131)
        at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter.invokeHandlerMethod(AnnotationMethodHandlerAdapter.java:308)
        at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter.handle(AnnotationMethodHandlerAdapter.java:295)
        at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:874)
        at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:808)
        at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:523)
        at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:453)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:114)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:91)
        at com.caucho.server.dispatch.ServletFilterChain.doFilter(ServletFilterChain.java:103)
        at com.caucho.server.webapp.WebAppFilterChain.doFilter(WebAppFilterChain.java:181)
        at com.caucho.server.dispatch.ServletInvocation.service(ServletInvocation.java:266)
        at com.caucho.server.http.HttpRequest.handleRequest(HttpRequest.java:269)
        at com.caucho.server.port.TcpConnection.run(TcpConnection.java:603)
        at com.caucho.util.ThreadPool$Item.runTasks(ThreadPool.java:721)
        at com.caucho.util.ThreadPool$Item.run(ThreadPool.java:643)
        at java.lang.Thread.run(Thread.java:619)

Brad Fitzgibbons
Senior Software Engineer
Tech Target
The Most Targeted IT Media
http://www.techtarget.com/

[Marcel Reutegger]

Hi,

this seems very strange because LazyScoreNodeIterator always pre-fetches the
next node. so whenever you call hasNext() and it returns true
LazyScoreNodeIterator already has the next node and does not have to fetch it on
nextRow().

Can you please double check that you always call hasNext() before you call
nextRow() ?

regards
  marcel

Fitzgibbons, Brad wrote:
> I'm running into this problem occasionally resulting in the exception below.  I'm using Jackrabbit 1.4 (with core at 1.4.2).  I'm using the H2PersistenceManager and almost all configuration values are set to their released default. Document order is turned off so the query is using a LazyScoreNodeIterator.  I'm running in a transactional environment, and my entire execution flow is contained by a single, read-only, JTA transaction.  The single transaction includes executing the query and all interactions with the returned RowIterator.  If I ignore the NoSuchElementException's and continue using the iterator then there are often good results subsequent to the error.  I've noticed that when these errors occur there are often far more results than are returned by RowIterator.getSize().  I realize that getSize() is not meant to be an accurate count of all results, but in most other cases getSize() is greater than the size of the result set (this may not be helpful).  I'm using
 a SQL query that involves the CONTAINS constraint (so lucene) and ordering by jcr:score.  Does this warrant a bug submission?  Is there any additional information which might help?

>
> org.springframework.web.util.NestedServletException: Request processing failed; nested exception is java.util.NoSuchElementException
>         at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:535)
>         at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:453)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:114)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:91)
>         at com.caucho.server.dispatch.ServletFilterChain.doFilter(ServletFilterChain.java:103)
>         at com.caucho.server.webapp.WebAppFilterChain.doFilter(WebAppFilterChain.java:181)
>         at com.caucho.server.dispatch.ServletInvocation.service(ServletInvocation.java:266)
>         at com.caucho.server.http.HttpRequest.handleRequest(HttpRequest.java:269)
>         at com.caucho.server.port.TcpConnection.run(TcpConnection.java:603)
>         at com.caucho.util.ThreadPool$Item.runTasks(ThreadPool.java:721)
>         at com.caucho.util.ThreadPool$Item.run(ThreadPool.java:643)
>         at java.lang.Thread.run(Thread.java:619)
> Caused by: java.util.NoSuchElementException
>         at org.apache.jackrabbit.core.query.lucene.QueryResultImpl$LazyScoreNodeIterator.getScore(QueryResultImpl.java:351)
>         at org.apache.jackrabbit.core.query.lucene.RowIteratorImpl.nextRow(RowIteratorImpl.java:153)
>         at com.techtarget.content.data.jcr1.ElementDaoImpl.nextElement(ElementDaoImpl.java:722)
>         at com.techtarget.content.data.jcr1.PropertyIterator.next(PropertyIterator.java:47)
>         at com.techtarget.content.data.jcr1.PropertyIterator.next(PropertyIterator.java:21)
>         at com.techtarget.content.web.controller.ProfileController.resourceQueryAndPrint(ProfileController.java:125)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at org.springframework.web.bind.annotation.support.HandlerMethodInvoker.doInvokeMethod(HandlerMethodInvoker.java:305)
>         at org.springframework.web.bind.annotation.support.HandlerMethodInvoker.invokeHandlerMethod(HandlerMethodInvoker.java:131)
>         at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter.invokeHandlerMethod(AnnotationMethodHandlerAdapter.java:308)
>         at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter.handle(AnnotationMethodHandlerAdapter.java:295)
>         at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:874)
>         at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:808)
>         at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:523)
>         at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:453)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:114)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:91)
>         at com.caucho.server.dispatch.ServletFilterChain.doFilter(ServletFilterChain.java:103)
>         at com.caucho.server.webapp.WebAppFilterChain.doFilter(WebAppFilterChain.java:181)
>         at com.caucho.server.dispatch.ServletInvocation.service(ServletInvocation.java:266)
>         at com.caucho.server.http.HttpRequest.handleRequest(HttpRequest.java:269)
>         at com.caucho.server.port.TcpConnection.run(TcpConnection.java:603)
>         at com.caucho.util.ThreadPool$Item.runTasks(ThreadPool.java:721)
>         at com.caucho.util.ThreadPool$Item.run(ThreadPool.java:643)
>         at java.lang.Thread.run(Thread.java:619)
>
> Brad Fitzgibbons
> Senior Software Engineer
> Tech Target
> The Most Targeted IT Media
> http://www.techtarget.com/
>
>
>

[Fitzgibbons, Brad]

Yes.  My loop breaks down to this:

while (iterator.hasNext()) {
   Object obj = iterator.next();
   ...
}

However, if I do this instead I get (what appears to be) "normal" behavior (except that getSize() is often much less than the actual result set size):

while (iterator.hasNext()) {
    try {
        Object obj = iterator.next();
    } catch (NoSuchElementException excp) { /* masked */ }
}

Thanks!
brad

-----Original Message-----
From: Marcel Reutegger [mailto:marcel.reutegger@...]
Sent: Monday, July 14, 2008 4:35 AM
To: users@...
Subject: Re: RowIterator.next() fails when hasNext() returns true

Hi,

this seems very strange because LazyScoreNodeIterator always pre-fetches the next node. so whenever you call hasNext() and it returns true LazyScoreNodeIterator already has the next node and does not have to fetch it on nextRow().

Can you please double check that you always call hasNext() before you call
nextRow() ?

regards
  marcel

Fitzgibbons, Brad wrote:

> I'm running into this problem occasionally resulting in the exception
> below.  I'm using Jackrabbit 1.4 (with core at 1.4.2).  I'm using the
> H2PersistenceManager and almost all configuration values are set to
> their released default. Document order is turned off so the query is
> using a LazyScoreNodeIterator.  I'm running in a transactional
> environment, and my entire execution flow is contained by a single,
> read-only, JTA transaction.  The single transaction includes executing
> the query and all interactions with the returned RowIterator.  If I
> ignore the NoSuchElementException's and continue using the iterator
> then there are often good results subsequent to the error.  I've
> noticed that when these errors occur there are often far more results
> than are returned by RowIterator.getSize().  I realize that getSize()
> is not meant to be an accurate count of all results, but in most other
> cases getSize() is greater than the size of the result set (this may
> not be helpful).  I'm using

 a SQL query that involves the CONTAINS constraint (so lucene) and ordering by jcr:score.  Does this warrant a bug submission?  Is there any additional information which might help?

>
> org.springframework.web.util.NestedServletException: Request processing failed; nested exception is java.util.NoSuchElementException
>         at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:535)
>         at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:453)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:114)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:91)
>         at com.caucho.server.dispatch.ServletFilterChain.doFilter(ServletFilterChain.java:103)
>         at com.caucho.server.webapp.WebAppFilterChain.doFilter(WebAppFilterChain.java:181)
>         at com.caucho.server.dispatch.ServletInvocation.service(ServletInvocation.java:266)
>         at com.caucho.server.http.HttpRequest.handleRequest(HttpRequest.java:269)
>         at com.caucho.server.port.TcpConnection.run(TcpConnection.java:603)
>         at com.caucho.util.ThreadPool$Item.runTasks(ThreadPool.java:721)
>         at com.caucho.util.ThreadPool$Item.run(ThreadPool.java:643)
>         at java.lang.Thread.run(Thread.java:619)
> Caused by: java.util.NoSuchElementException
>         at org.apache.jackrabbit.core.query.lucene.QueryResultImpl$LazyScoreNodeIterator.getScore(QueryResultImpl.java:351)
>         at org.apache.jackrabbit.core.query.lucene.RowIteratorImpl.nextRow(RowIteratorImpl.java:153)
>         at com.techtarget.content.data.jcr1.ElementDaoImpl.nextElement(ElementDaoImpl.java:722)
>         at com.techtarget.content.data.jcr1.PropertyIterator.next(PropertyIterator.java:47)
>         at com.techtarget.content.data.jcr1.PropertyIterator.next(PropertyIterator.java:21)
>         at com.techtarget.content.web.controller.ProfileController.resourceQueryAndPrint(ProfileController.java:125)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at org.springframework.web.bind.annotation.support.HandlerMethodInvoker.doInvokeMethod(HandlerMethodInvoker.java:305)
>         at org.springframework.web.bind.annotation.support.HandlerMethodInvoker.invokeHandlerMethod(HandlerMethodInvoker.java:131)
>         at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter.invokeHandlerMethod(AnnotationMethodHandlerAdapter.java:308)
>         at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter.handle(AnnotationMethodHandlerAdapter.java:295)
>         at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:874)
>         at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:808)
>         at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:523)
>         at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:453)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:114)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:91)
>         at com.caucho.server.dispatch.ServletFilterChain.doFilter(ServletFilterChain.java:103)
>         at com.caucho.server.webapp.WebAppFilterChain.doFilter(WebAppFilterChain.java:181)
>         at com.caucho.server.dispatch.ServletInvocation.service(ServletInvocation.java:266)
>         at com.caucho.server.http.HttpRequest.handleRequest(HttpRequest.java:269)
>         at com.caucho.server.port.TcpConnection.run(TcpConnection.java:603)
>         at com.caucho.util.ThreadPool$Item.runTasks(ThreadPool.java:721)
>         at com.caucho.util.ThreadPool$Item.run(ThreadPool.java:643)
>         at java.lang.Thread.run(Thread.java:619)
>
> Brad Fitzgibbons
> Senior Software Engineer
> Tech Target
> The Most Targeted IT Media
> http://www.techtarget.com/
>
>
>

[Marcel Reutegger]

Hi,

hmm, let's check something else... what about your class ProfileController, is
it thread-safe? is it possible that two threads concurrently call hasNext/next?
is PropertyIterator in ProfileController an instance variable?

regards
  marcel

Fitzgibbons, Brad wrote:

> Yes.  My loop breaks down to this:
>
> while (iterator.hasNext()) {
>    Object obj = iterator.next();
>    ...
> }
>
> However, if I do this instead I get (what appears to be) "normal" behavior (except that getSize() is often much less than the actual result set size):
>
> while (iterator.hasNext()) {
>     try {
>         Object obj = iterator.next();
>     } catch (NoSuchElementException excp) { /* masked */ }
> }
>
> Thanks!
> brad
>
> -----Original Message-----
> From: Marcel Reutegger [mailto:marcel.reutegger@...]
> Sent: Monday, July 14, 2008 4:35 AM
> To: users@...
> Subject: Re: RowIterator.next() fails when hasNext() returns true
>
> Hi,
>
> this seems very strange because LazyScoreNodeIterator always pre-fetches the next node. so whenever you call hasNext() and it returns true LazyScoreNodeIterator already has the next node and does not have to fetch it on nextRow().
>
> Can you please double check that you always call hasNext() before you call
> nextRow() ?
>
> regards
>   marcel
>
> Fitzgibbons, Brad wrote:
>> I'm running into this problem occasionally resulting in the exception
>> below.  I'm using Jackrabbit 1.4 (with core at 1.4.2).  I'm using the
>> H2PersistenceManager and almost all configuration values are set to
>> their released default. Document order is turned off so the query is
>> using a LazyScoreNodeIterator.  I'm running in a transactional
>> environment, and my entire execution flow is contained by a single,
>> read-only, JTA transaction.  The single transaction includes executing
>> the query and all interactions with the returned RowIterator.  If I
>> ignore the NoSuchElementException's and continue using the iterator
>> then there are often good results subsequent to the error.  I've
>> noticed that when these errors occur there are often far more results
>> than are returned by RowIterator.getSize().  I realize that getSize()
>> is not meant to be an accurate count of all results, but in most other
>> cases getSize() is greater than the size of the result set (this may
>> not be helpful).  I'm using
>  a SQL query that involves the CONTAINS constraint (so lucene) and ordering by jcr:score.  Does this warrant a bug submission?  Is there any additional information which might help?
>> org.springframework.web.util.NestedServletException: Request processing failed; nested exception is java.util.NoSuchElementException
>>         at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:535)
>>         at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:453)
>>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:114)
>>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:91)
>>         at com.caucho.server.dispatch.ServletFilterChain.doFilter(ServletFilterChain.java:103)
>>         at com.caucho.server.webapp.WebAppFilterChain.doFilter(WebAppFilterChain.java:181)
>>         at com.caucho.server.dispatch.ServletInvocation.service(ServletInvocation.java:266)
>>         at com.caucho.server.http.HttpRequest.handleRequest(HttpRequest.java:269)
>>         at com.caucho.server.port.TcpConnection.run(TcpConnection.java:603)
>>         at com.caucho.util.ThreadPool$Item.runTasks(ThreadPool.java:721)
>>         at com.caucho.util.ThreadPool$Item.run(ThreadPool.java:643)
>>         at java.lang.Thread.run(Thread.java:619)
>> Caused by: java.util.NoSuchElementException
>>         at org.apache.jackrabbit.core.query.lucene.QueryResultImpl$LazyScoreNodeIterator.getScore(QueryResultImpl.java:351)
>>         at org.apache.jackrabbit.core.query.lucene.RowIteratorImpl.nextRow(RowIteratorImpl.java:153)
>>         at com.techtarget.content.data.jcr1.ElementDaoImpl.nextElement(ElementDaoImpl.java:722)
>>         at com.techtarget.content.data.jcr1.PropertyIterator.next(PropertyIterator.java:47)
>>         at com.techtarget.content.data.jcr1.PropertyIterator.next(PropertyIterator.java:21)
>>         at com.techtarget.content.web.controller.ProfileController.resourceQueryAndPrint(ProfileController.java:125)
>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>         at java.lang.reflect.Method.invoke(Method.java:597)
>>         at org.springframework.web.bind.annotation.support.HandlerMethodInvoker.doInvokeMethod(HandlerMethodInvoker.java:305)
>>         at org.springframework.web.bind.annotation.support.HandlerMethodInvoker.invokeHandlerMethod(HandlerMethodInvoker.java:131)
>>         at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter.invokeHandlerMethod(AnnotationMethodHandlerAdapter.java:308)
>>         at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter.handle(AnnotationMethodHandlerAdapter.java:295)
>>         at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:874)
>>         at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:808)
>>         at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:523)
>>         at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:453)
>>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:114)
>>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:91)
>>         at com.caucho.server.dispatch.ServletFilterChain.doFilter(ServletFilterChain.java:103)
>>         at com.caucho.server.webapp.WebAppFilterChain.doFilter(WebAppFilterChain.java:181)
>>         at com.caucho.server.dispatch.ServletInvocation.service(ServletInvocation.java:266)
>>         at com.caucho.server.http.HttpRequest.handleRequest(HttpRequest.java:269)
>>         at com.caucho.server.port.TcpConnection.run(TcpConnection.java:603)
>>         at com.caucho.util.ThreadPool$Item.runTasks(ThreadPool.java:721)
>>         at com.caucho.util.ThreadPool$Item.run(ThreadPool.java:643)
>>         at java.lang.Thread.run(Thread.java:619)
>>
>> Brad Fitzgibbons
>> Senior Software Engineer
>> Tech Target
>> The Most Targeted IT Media
>> http://www.techtarget.com/
>>
>>
>>
>
>

[Fitzgibbons, Brad]

Yes, it's threadsafe.  Each visit to the URL invokes the resourceQueryAndPrint() method.  This method is wrapped in a JTA transaction, and results in the following actions:
    (1) Execute a new SQL query on the JCR session (using spring-modules to provide session).
    (2) Create a new instance of PropertyIterator which wraps the RowIterator from the query.
    (3) Perform the loop as shown and print some of the metadata from each element in the result set.
        (a) Each call to PropertyIterator.hasNext() delegates to RowIterator.hasNext().
        (a) Each call to PropertyIterator.next() will marshall data from the Row returned by RowIterator.next() into a DTO.

Here's an example of the SQL query (with actual property names and node primary type changed):
    select A,B,C,D from custom_type where (A='A1' or A='A2' or A='A3') and (contains(B, 'XXX') or contains(C, 'XXX')) order by D

Changing the properties in the select clause or the order by clause (but leaving the where clause as-is) will sometimes produce iterators that do not have the hasNext/next behavior I've described.


-----Original Message-----
From: Marcel Reutegger [mailto:marcel.reutegger@...]
Sent: Monday, July 14, 2008 8:40 AM
To: users@...
Subject: Re: RowIterator.next() fails when hasNext() returns true

Hi,

hmm, let's check something else... what about your class ProfileController, is it thread-safe? is it possible that two threads concurrently call hasNext/next?
is PropertyIterator in ProfileController an instance variable?

regards
  marcel

Fitzgibbons, Brad wrote:

> Yes.  My loop breaks down to this:
>
> while (iterator.hasNext()) {
>    Object obj = iterator.next();
>    ...
> }
>
> However, if I do this instead I get (what appears to be) "normal" behavior (except that getSize() is often much less than the actual result set size):
>
> while (iterator.hasNext()) {
>     try {
>         Object obj = iterator.next();
>     } catch (NoSuchElementException excp) { /* masked */ } }
>
> Thanks!
> brad
>
> -----Original Message-----
> From: Marcel Reutegger [mailto:marcel.reutegger@...]
> Sent: Monday, July 14, 2008 4:35 AM
> To: users@...
> Subject: Re: RowIterator.next() fails when hasNext() returns true
>
> Hi,
>
> this seems very strange because LazyScoreNodeIterator always pre-fetches the next node. so whenever you call hasNext() and it returns true LazyScoreNodeIterator already has the next node and does not have to fetch it on nextRow().
>
> Can you please double check that you always call hasNext() before you
> call
> nextRow() ?
>
> regards
>   marcel
>
> Fitzgibbons, Brad wrote:
>> I'm running into this problem occasionally resulting in the exception
>> below.  I'm using Jackrabbit 1.4 (with core at 1.4.2).  I'm using the
>> H2PersistenceManager and almost all configuration values are set to
>> their released default. Document order is turned off so the query is
>> using a LazyScoreNodeIterator.  I'm running in a transactional
>> environment, and my entire execution flow is contained by a single,
>> read-only, JTA transaction.  The single transaction includes
>> executing the query and all interactions with the returned
>> RowIterator.  If I ignore the NoSuchElementException's and continue
>> using the iterator then there are often good results subsequent to
>> the error.  I've noticed that when these errors occur there are often
>> far more results than are returned by RowIterator.getSize().  I
>> realize that getSize() is not meant to be an accurate count of all
>> results, but in most other cases getSize() is greater than the size
>> of the result set (this may not be helpful).  I'm using
>  a SQL query that involves the CONTAINS constraint (so lucene) and ordering by jcr:score.  Does this warrant a bug submission?  Is there any additional information which might help?
>> org.springframework.web.util.NestedServletException: Request processing failed; nested exception is java.util.NoSuchElementException
>>         at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:535)
>>         at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:453)
>>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:114)
>>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:91)
>>         at com.caucho.server.dispatch.ServletFilterChain.doFilter(ServletFilterChain.java:103)
>>         at com.caucho.server.webapp.WebAppFilterChain.doFilter(WebAppFilterChain.java:181)
>>         at com.caucho.server.dispatch.ServletInvocation.service(ServletInvocation.java:266)
>>         at com.caucho.server.http.HttpRequest.handleRequest(HttpRequest.java:269)
>>         at com.caucho.server.port.TcpConnection.run(TcpConnection.java:603)
>>         at com.caucho.util.ThreadPool$Item.runTasks(ThreadPool.java:721)
>>         at com.caucho.util.ThreadPool$Item.run(ThreadPool.java:643)
>>         at java.lang.Thread.run(Thread.java:619)
>> Caused by: java.util.NoSuchElementException
>>         at org.apache.jackrabbit.core.query.lucene.QueryResultImpl$LazyScoreNodeIterator.getScore(QueryResultImpl.java:351)
>>         at org.apache.jackrabbit.core.query.lucene.RowIteratorImpl.nextRow(RowIteratorImpl.java:153)
>>         at com.techtarget.content.data.jcr1.ElementDaoImpl.nextElement(ElementDaoImpl.java:722)
>>         at com.techtarget.content.data.jcr1.PropertyIterator.next(PropertyIterator.java:47)
>>         at com.techtarget.content.data.jcr1.PropertyIterator.next(PropertyIterator.java:21)
>>         at com.techtarget.content.web.controller.ProfileController.resourceQueryAndPrint(ProfileController.java:125)
>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>         at java.lang.reflect.Method.invoke(Method.java:597)
>>         at org.springframework.web.bind.annotation.support.HandlerMethodInvoker.doInvokeMethod(HandlerMethodInvoker.java:305)
>>         at org.springframework.web.bind.annotation.support.HandlerMethodInvoker.invokeHandlerMethod(HandlerMethodInvoker.java:131)
>>         at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter.invokeHandlerMethod(AnnotationMethodHandlerAdapter.java:308)
>>         at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter.handle(AnnotationMethodHandlerAdapter.java:295)
>>         at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:874)
>>         at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:808)
>>         at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:523)
>>         at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:453)
>>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:114)
>>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:91)
>>         at com.caucho.server.dispatch.ServletFilterChain.doFilter(ServletFilterChain.java:103)
>>         at com.caucho.server.webapp.WebAppFilterChain.doFilter(WebAppFilterChain.java:181)
>>         at com.caucho.server.dispatch.ServletInvocation.service(ServletInvocation.java:266)
>>         at com.caucho.server.http.HttpRequest.handleRequest(HttpRequest.java:269)
>>         at com.caucho.server.port.TcpConnection.run(TcpConnection.java:603)
>>         at com.caucho.util.ThreadPool$Item.runTasks(ThreadPool.java:721)
>>         at com.caucho.util.ThreadPool$Item.run(ThreadPool.java:643)
>>         at java.lang.Thread.run(Thread.java:619)
>>
>> Brad Fitzgibbons
>> Senior Software Engineer
>> Tech Target
>> The Most Targeted IT Media
>> http://www.techtarget.com/
>>
>>
>>
>
>

[Fitzgibbons, Brad]

Here's one more thing I thought of that might be helpful.  I can create a query where I expect exactly one result that exhibits the hasNext/next behavior.  However, when I mask it as shown below I still get just the one expected result.  This goes to what I meant by "normal" behavior when masking the error.

Thanks for your help!
brad

-----Original Message-----
From: Fitzgibbons, Brad
Sent: Monday, July 14, 2008 9:04 AM
To: 'users@...'
Subject: RE: RowIterator.next() fails when hasNext() returns true

Yes, it's threadsafe.  Each visit to the URL invokes the resourceQueryAndPrint() method.  This method is wrapped in a JTA transaction, and results in the following actions:
    (1) Execute a new SQL query on the JCR session (using spring-modules to provide session).
    (2) Create a new instance of PropertyIterator which wraps the RowIterator from the query.
    (3) Perform the loop as shown and print some of the metadata from each element in the result set.
        (a) Each call to PropertyIterator.hasNext() delegates to RowIterator.hasNext().
        (a) Each call to PropertyIterator.next() will marshall data from the Row returned by RowIterator.next() into a DTO.

Here's an example of the SQL query (with actual property names and node primary type changed):
    select A,B,C,D from custom_type where (A='A1' or A='A2' or A='A3') and (contains(B, 'XXX') or contains(C, 'XXX')) order by D

Changing the properties in the select clause or the order by clause (but leaving the where clause as-is) will sometimes produce iterators that do not have the hasNext/next behavior I've described.


-----Original Message-----
From: Marcel Reutegger [mailto:marcel.reutegger@...]
Sent: Monday, July 14, 2008 8:40 AM
To: users@...
Subject: Re: RowIterator.next() fails when hasNext() returns true

Hi,

hmm, let's check something else... what about your class ProfileController, is it thread-safe? is it possible that two threads concurrently call hasNext/next?
is PropertyIterator in ProfileController an instance variable?

regards
  marcel

Fitzgibbons, Brad wrote:

> Yes.  My loop breaks down to this:
>
> while (iterator.hasNext()) {
>    Object obj = iterator.next();
>    ...
> }
>
> However, if I do this instead I get (what appears to be) "normal" behavior (except that getSize() is often much less than the actual result set size):
>
> while (iterator.hasNext()) {
>     try {
>         Object obj = iterator.next();
>     } catch (NoSuchElementException excp) { /* masked */ } }
>
> Thanks!
> brad
>
> -----Original Message-----
> From: Marcel Reutegger [mailto:marcel.reutegger@...]
> Sent: Monday, July 14, 2008 4:35 AM
> To: users@...
> Subject: Re: RowIterator.next() fails when hasNext() returns true
>
> Hi,
>
> this seems very strange because LazyScoreNodeIterator always pre-fetches the next node. so whenever you call hasNext() and it returns true LazyScoreNodeIterator already has the next node and does not have to fetch it on nextRow().
>
> Can you please double check that you always call hasNext() before you
> call
> nextRow() ?
>
> regards
>   marcel
>
> Fitzgibbons, Brad wrote:
>> I'm running into this problem occasionally resulting in the exception
>> below.  I'm using Jackrabbit 1.4 (with core at 1.4.2).  I'm using the
>> H2PersistenceManager and almost all configuration values are set to
>> their released default. Document order is turned off so the query is
>> using a LazyScoreNodeIterator.  I'm running in a transactional
>> environment, and my entire execution flow is contained by a single,
>> read-only, JTA transaction.  The single transaction includes
>> executing the query and all interactions with the returned
>> RowIterator.  If I ignore the NoSuchElementException's and continue
>> using the iterator then there are often good results subsequent to
>> the error.  I've noticed that when these errors occur there are often
>> far more results than are returned by RowIterator.getSize().  I
>> realize that getSize() is not meant to be an accurate count of all
>> results, but in most other cases getSize() is greater than the size
>> of the result set (this may not be helpful).  I'm using
>  a SQL query that involves the CONTAINS constraint (so lucene) and ordering by jcr:score.  Does this warrant a bug submission?  Is there any additional information which might help?
>> org.springframework.web.util.NestedServletException: Request processing failed; nested exception is java.util.NoSuchElementException
>>         at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:535)
>>         at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:453)
>>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:114)
>>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:91)
>>         at com.caucho.server.dispatch.ServletFilterChain.doFilter(ServletFilterChain.java:103)
>>         at com.caucho.server.webapp.WebAppFilterChain.doFilter(WebAppFilterChain.java:181)
>>         at com.caucho.server.dispatch.ServletInvocation.service(ServletInvocation.java:266)
>>         at com.caucho.server.http.HttpRequest.handleRequest(HttpRequest.java:269)
>>         at com.caucho.server.port.TcpConnection.run(TcpConnection.java:603)
>>         at com.caucho.util.ThreadPool$Item.runTasks(ThreadPool.java:721)
>>         at com.caucho.util.ThreadPool$Item.run(ThreadPool.java:643)
>>         at java.lang.Thread.run(Thread.java:619)
>> Caused by: java.util.NoSuchElementException
>>         at org.apache.jackrabbit.core.query.lucene.QueryResultImpl$LazyScoreNodeIterator.getScore(QueryResultImpl.java:351)
>>         at org.apache.jackrabbit.core.query.lucene.RowIteratorImpl.nextRow(RowIteratorImpl.java:153)
>>         at com.techtarget.content.data.jcr1.ElementDaoImpl.nextElement(ElementDaoImpl.java:722)
>>         at com.techtarget.content.data.jcr1.PropertyIterator.next(PropertyIterator.java:47)
>>         at com.techtarget.content.data.jcr1.PropertyIterator.next(PropertyIterator.java:21)
>>         at com.techtarget.content.web.controller.ProfileController.resourceQueryAndPrint(ProfileController.java:125)
>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>         at java.lang.reflect.Method.invoke(Method.java:597)
>>         at org.springframework.web.bind.annotation.support.HandlerMethodInvoker.doInvokeMethod(HandlerMethodInvoker.java:305)
>>         at org.springframework.web.bind.annotation.support.HandlerMethodInvoker.invokeHandlerMethod(HandlerMethodInvoker.java:131)
>>         at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter.invokeHandlerMethod(AnnotationMethodHandlerAdapter.java:308)
>>         at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter.handle(AnnotationMethodHandlerAdapter.java:295)
>>         at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:874)
>>         at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:808)
>>         at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:523)
>>         at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:453)
>>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:114)
>>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:91)
>>         at com.caucho.server.dispatch.ServletFilterChain.doFilter(ServletFilterChain.java:103)
>>         at com.caucho.server.webapp.WebAppFilterChain.doFilter(WebAppFilterChain.java:181)
>>         at com.caucho.server.dispatch.ServletInvocation.service(ServletInvocation.java:266)
>>         at com.caucho.server.http.HttpRequest.handleRequest(HttpRequest.java:269)
>>         at com.caucho.server.port.TcpConnection.run(TcpConnection.java:603)
>>         at com.caucho.util.ThreadPool$Item.runTasks(ThreadPool.java:721)
>>         at com.caucho.util.ThreadPool$Item.run(ThreadPool.java:643)
>>         at java.lang.Thread.run(Thread.java:619)
>>
>> Brad Fitzgibbons
>> Senior Software Engineer
>> Tech Target
>> The Most Targeted IT Media
>> http://www.techtarget.com/
>>
>>
>>
>
>

[Marcel Reutegger]

Hi Brad,

are you able to extract this into simple test program without dependencies to
your application? that would be great.

regards
  marcel

Fitzgibbons, Brad wrote:

> Yes, it's threadsafe.  Each visit to the URL invokes the resourceQueryAndPrint() method.  This method is wrapped in a JTA transaction, and results in the following actions:
>     (1) Execute a new SQL query on the JCR session (using spring-modules to provide session).
>     (2) Create a new instance of PropertyIterator which wraps the RowIterator from the query.
>     (3) Perform the loop as shown and print some of the metadata from each element in the result set.
>         (a) Each call to PropertyIterator.hasNext() delegates to RowIterator.hasNext().
>         (a) Each call to PropertyIterator.next() will marshall data from the Row returned by RowIterator.next() into a DTO.
>
> Here's an example of the SQL query (with actual property names and node primary type changed):
>     select A,B,C,D from custom_type where (A='A1' or A='A2' or A='A3') and (contains(B, 'XXX') or contains(C, 'XXX')) order by D
>
> Changing the properties in the select clause or the order by clause (but leaving the where clause as-is) will sometimes produce iterators that do not have the hasNext/next behavior I've described.
>
>
> -----Original Message-----
> From: Marcel Reutegger [mailto:marcel.reutegger@...]
> Sent: Monday, July 14, 2008 8:40 AM
> To: users@...
> Subject: Re: RowIterator.next() fails when hasNext() returns true
>
> Hi,
>
> hmm, let's check something else... what about your class ProfileController, is it thread-safe? is it possible that two threads concurrently call hasNext/next?
> is PropertyIterator in ProfileController an instance variable?
>
> regards
>   marcel
>
> Fitzgibbons, Brad wrote:
>> Yes.  My loop breaks down to this:
>>
>> while (iterator.hasNext()) {
>>    Object obj = iterator.next();
>>    ...
>> }
>>
>> However, if I do this instead I get (what appears to be) "normal" behavior (except that getSize() is often much less than the actual result set size):
>>
>> while (iterator.hasNext()) {
>>     try {
>>         Object obj = iterator.next();
>>     } catch (NoSuchElementException excp) { /* masked */ } }
>>
>> Thanks!
>> brad
>>
>> -----Original Message-----
>> From: Marcel Reutegger [mailto:marcel.reutegger@...]
>> Sent: Monday, July 14, 2008 4:35 AM
>> To: users@...
>> Subject: Re: RowIterator.next() fails when hasNext() returns true
>>
>> Hi,
>>
>> this seems very strange because LazyScoreNodeIterator always pre-fetches the next node. so whenever you call hasNext() and it returns true LazyScoreNodeIterator already has the next node and does not have to fetch it on nextRow().
>>
>> Can you please double check that you always call hasNext() before you
>> call
>> nextRow() ?
>>
>> regards
>>   marcel
>>
>> Fitzgibbons, Brad wrote:
>>> I'm running into this problem occasionally resulting in the exception
>>> below.  I'm using Jackrabbit 1.4 (with core at 1.4.2).  I'm using the
>>> H2PersistenceManager and almost all configuration values are set to
>>> their released default. Document order is turned off so the query is
>>> using a LazyScoreNodeIterator.  I'm running in a transactional
>>> environment, and my entire execution flow is contained by a single,
>>> read-only, JTA transaction.  The single transaction includes
>>> executing the query and all interactions with the returned
>>> RowIterator.  If I ignore the NoSuchElementException's and continue
>>> using the iterator then there are often good results subsequent to
>>> the error.  I've noticed that when these errors occur there are often
>>> far more results than are returned by RowIterator.getSize().  I
>>> realize that getSize() is not meant to be an accurate count of all
>>> results, but in most other cases getSize() is greater than the size
>>> of the result set (this may not be helpful).  I'm using
>>  a SQL query that involves the CONTAINS constraint (so lucene) and ordering by jcr:score.  Does this warrant a bug submission?  Is there any additional information which might help?
>>> org.springframework.web.util.NestedServletException: Request processing failed; nested exception is java.util.NoSuchElementException
>>>         at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:535)
>>>         at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:453)
>>>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:114)
>>>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:91)
>>>         at com.caucho.server.dispatch.ServletFilterChain.doFilter(ServletFilterChain.java:103)
>>>         at com.caucho.server.webapp.WebAppFilterChain.doFilter(WebAppFilterChain.java:181)
>>>         at com.caucho.server.dispatch.ServletInvocation.service(ServletInvocation.java:266)
>>>         at com.caucho.server.http.HttpRequest.handleRequest(HttpRequest.java:269)
>>>         at com.caucho.server.port.TcpConnection.run(TcpConnection.java:603)
>>>         at com.caucho.util.ThreadPool$Item.runTasks(ThreadPool.java:721)
>>>         at com.caucho.util.ThreadPool$Item.run(ThreadPool.java:643)
>>>         at java.lang.Thread.run(Thread.java:619)
>>> Caused by: java.util.NoSuchElementException
>>>         at org.apache.jackrabbit.core.query.lucene.QueryResultImpl$LazyScoreNodeIterator.getScore(QueryResultImpl.java:351)
>>>         at org.apache.jackrabbit.core.query.lucene.RowIteratorImpl.nextRow(RowIteratorImpl.java:153)
>>>         at com.techtarget.content.data.jcr1.ElementDaoImpl.nextElement(ElementDaoImpl.java:722)
>>>         at com.techtarget.content.data.jcr1.PropertyIterator.next(PropertyIterator.java:47)
>>>         at com.techtarget.content.data.jcr1.PropertyIterator.next(PropertyIterator.java:21)
>>>         at com.techtarget.content.web.controller.ProfileController.resourceQueryAndPrint(ProfileController.java:125)
>>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>         at java.lang.reflect.Method.invoke(Method.java:597)
>>>         at org.springframework.web.bind.annotation.support.HandlerMethodInvoker.doInvokeMethod(HandlerMethodInvoker.java:305)
>>>         at org.springframework.web.bind.annotation.support.HandlerMethodInvoker.invokeHandlerMethod(HandlerMethodInvoker.java:131)
>>>         at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter.invokeHandlerMethod(AnnotationMethodHandlerAdapter.java:308)
>>>         at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter.handle(AnnotationMethodHandlerAdapter.java:295)
>>>         at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:874)
>>>         at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:808)
>>>         at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:523)
>>>         at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:453)
>>>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:114)
>>>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:91)
>>>         at com.caucho.server.dispatch.ServletFilterChain.doFilter(ServletFilterChain.java:103)
>>>         at com.caucho.server.webapp.WebAppFilterChain.doFilter(WebAppFilterChain.java:181)
>>>         at com.caucho.server.dispatch.ServletInvocation.service(ServletInvocation.java:266)
>>>         at com.caucho.server.http.HttpRequest.handleRequest(HttpRequest.java:269)
>>>         at com.caucho.server.port.TcpConnection.run(TcpConnection.java:603)
>>>         at com.caucho.util.ThreadPool$Item.runTasks(ThreadPool.java:721)
>>>         at com.caucho.util.ThreadPool$Item.run(ThreadPool.java:643)
>>>         at java.lang.Thread.run(Thread.java:619)
>>>
>>> Brad Fitzgibbons
>>> Senior Software Engineer
>>> Tech Target
>>> The Most Targeted IT Media
>>> http://www.techtarget.com/
>>>
>>>
>>>
>>
>
>

[Fitzgibbons, Brad]

I'll try to find some time to spin something up, but it's likely to be awhile before I can find the bandwidth to do so.  As soon as I can I'll submit a repeatable, non-proprietary example of the problem.

As an aside, I've thoroughly enjoyed working with the JCR standard and Jackrabbit.  Thanks for all the work you and others have put into this.

brad

-----Original Message-----
From: Marcel Reutegger [mailto:marcel.reutegger@...]
Sent: Wednesday, July 16, 2008 5:16 AM
To: users@...
Subject: Re: RowIterator.next() fails when hasNext() returns true

Hi Brad,

are you able to extract this into simple test program without dependencies to your application? that would be great.

regards
  marcel

Fitzgibbons, Brad wrote:

> Yes, it's threadsafe.  Each visit to the URL invokes the resourceQueryAndPrint() method.  This method is wrapped in a JTA transaction, and results in the following actions:
>     (1) Execute a new SQL query on the JCR session (using spring-modules to provide session).
>     (2) Create a new instance of PropertyIterator which wraps the RowIterator from the query.
>     (3) Perform the loop as shown and print some of the metadata from each element in the result set.
>         (a) Each call to PropertyIterator.hasNext() delegates to RowIterator.hasNext().
>         (a) Each call to PropertyIterator.next() will marshall data from the Row returned by RowIterator.next() into a DTO.
>
> Here's an example of the SQL query (with actual property names and node primary type changed):
>     select A,B,C,D from custom_type where (A='A1' or A='A2' or A='A3')
> and (contains(B, 'XXX') or contains(C, 'XXX')) order by D
>
> Changing the properties in the select clause or the order by clause (but leaving the where clause as-is) will sometimes produce iterators that do not have the hasNext/next behavior I've described.
>
>
> -----Original Message-----
> From: Marcel Reutegger [mailto:marcel.reutegger@...]
> Sent: Monday, July 14, 2008 8:40 AM
> To: users@...
> Subject: Re: RowIterator.next() fails when hasNext() returns true
>
> Hi,
>
> hmm, let's check something else... what about your class ProfileController, is it thread-safe? is it possible that two threads concurrently call hasNext/next?
> is PropertyIterator in ProfileController an instance variable?
>
> regards
>   marcel
>
> Fitzgibbons, Brad wrote:
>> Yes.  My loop breaks down to this:
>>
>> while (iterator.hasNext()) {
>>    Object obj = iterator.next();
>>    ...
>> }
>>
>> However, if I do this instead I get (what appears to be) "normal" behavior (except that getSize() is often much less than the actual result set size):
>>
>> while (iterator.hasNext()) {
>>     try {
>>         Object obj = iterator.next();
>>     } catch (NoSuchElementException excp) { /* masked */ } }
>>
>> Thanks!
>> brad
>>
>> -----Original Message-----
>> From: Marcel Reutegger [mailto:marcel.reutegger@...]
>> Sent: Monday, July 14, 2008 4:35 AM
>> To: users@...
>> Subject: Re: RowIterator.next() fails when hasNext() returns true
>>
>> Hi,
>>
>> this seems very strange because LazyScoreNodeIterator always pre-fetches the next node. so whenever you call hasNext() and it returns true LazyScoreNodeIterator already has the next node and does not have to fetch it on nextRow().
>>
>> Can you please double check that you always call hasNext() before you
>> call
>> nextRow() ?
>>
>> regards
>>   marcel
>>
>> Fitzgibbons, Brad wrote:
>>> I'm running into this problem occasionally resulting in the
>>> exception below.  I'm using Jackrabbit 1.4 (with core at 1.4.2).
>>> I'm using the H2PersistenceManager and almost all configuration
>>> values are set to their released default. Document order is turned
>>> off so the query is using a LazyScoreNodeIterator.  I'm running in a
>>> transactional environment, and my entire execution flow is contained
>>> by a single, read-only, JTA transaction.  The single transaction
>>> includes executing the query and all interactions with the returned
>>> RowIterator.  If I ignore the NoSuchElementException's and continue
>>> using the iterator then there are often good results subsequent to
>>> the error.  I've noticed that when these errors occur there are
>>> often far more results than are returned by RowIterator.getSize().
>>> I realize that getSize() is not meant to be an accurate count of all
>>> results, but in most other cases getSize() is greater than the size
>>> of the result set (this may not be helpful).  I'm using
>>  a SQL query that involves the CONTAINS constraint (so lucene) and ordering by jcr:score.  Does this warrant a bug submission?  Is there any additional information which might help?
>>> org.springframework.web.util.NestedServletException: Request processing failed; nested exception is java.util.NoSuchElementException
>>>         at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:535)
>>>         at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:453)
>>>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:114)
>>>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:91)
>>>         at com.caucho.server.dispatch.ServletFilterChain.doFilter(ServletFilterChain.java:103)
>>>         at com.caucho.server.webapp.WebAppFilterChain.doFilter(WebAppFilterChain.java:181)
>>>         at com.caucho.server.dispatch.ServletInvocation.service(ServletInvocation.java:266)
>>>         at com.caucho.server.http.HttpRequest.handleRequest(HttpRequest.java:269)
>>>         at com.caucho.server.port.TcpConnection.run(TcpConnection.java:603)
>>>         at com.caucho.util.ThreadPool$Item.runTasks(ThreadPool.java:721)
>>>         at com.caucho.util.ThreadPool$Item.run(ThreadPool.java:643)
>>>         at java.lang.Thread.run(Thread.java:619)
>>> Caused by: java.util.NoSuchElementException
>>>         at org.apache.jackrabbit.core.query.lucene.QueryResultImpl$LazyScoreNodeIterator.getScore(QueryResultImpl.java:351)
>>>         at org.apache.jackrabbit.core.query.lucene.RowIteratorImpl.nextRow(RowIteratorImpl.java:153)
>>>         at com.techtarget.content.data.jcr1.ElementDaoImpl.nextElement(ElementDaoImpl.java:722)
>>>         at com.techtarget.content.data.jcr1.PropertyIterator.next(PropertyIterator.java:47)
>>>         at com.techtarget.content.data.jcr1.PropertyIterator.next(PropertyIterator.java:21)
>>>         at com.techtarget.content.web.controller.ProfileController.resourceQueryAndPrint(ProfileController.java:125)
>>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>         at java.lang.reflect.Method.invoke(Method.java:597)
>>>         at org.springframework.web.bind.annotation.support.HandlerMethodInvoker.doInvokeMethod(HandlerMethodInvoker.java:305)
>>>         at org.springframework.web.bind.annotation.support.HandlerMethodInvoker.invokeHandlerMethod(HandlerMethodInvoker.java:131)
>>>         at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter.invokeHandlerMethod(AnnotationMethodHandlerAdapter.java:308)
>>>         at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter.handle(AnnotationMethodHandlerAdapter.java:295)
>>>         at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:874)
>>>         at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:808)
>>>         at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:523)
>>>         at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:453)
>>>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:114)
>>>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:91)
>>>         at com.caucho.server.dispatch.ServletFilterChain.doFilter(ServletFilterChain.java:103)
>>>         at com.caucho.server.webapp.WebAppFilterChain.doFilter(WebAppFilterChain.java:181)
>>>         at com.caucho.server.dispatch.ServletInvocation.service(ServletInvocation.java:266)
>>>         at com.caucho.server.http.HttpRequest.handleRequest(HttpRequest.java:269)
>>>         at com.caucho.server.port.TcpConnection.run(TcpConnection.java:603)
>>>         at com.caucho.util.ThreadPool$Item.runTasks(ThreadPool.java:721)
>>>         at com.caucho.util.ThreadPool$Item.run(ThreadPool.java:643)
>>>         at java.lang.Thread.run(Thread.java:619)
>>>
>>> Brad Fitzgibbons
>>> Senior Software Engineer
>>> Tech Target
>>> The Most Targeted IT Media
>>> http://www.techtarget.com/
>>>
>>>
>>>
>>
>
>