[ monetdb-Bugs-1759552 ] Mil optimizer bug

[SourceForge.net]

Bugs item #1759552, was opened at 2007-07-24 13:07
Message generated for change (Comment added) made by sf-robot
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=482468&aid=1759552&group_id=56967

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: PF/compiler
Group: Pathfinder CVS Head
>Status: Closed
Resolution: Out of Date
Priority: 5
Private: No
Submitted By: Arthur van Bunningen (arthurvb)
Assigned to: Stefan Manegold (stmane)
Summary: Mil optimizer bug

Initial Comment:
When executing attached query (ca_module.xq) monetdb returns the following error:

==========================
8272 8273 8274 8275 8276 8277 8278 8279 8280 8281 8282 8283 8284 8285 8286 8287 8288 8289 8290 8291 8292 8293 8294 8295 8296 8297 8298 8299 8300 8301 8302 8303 8304 8305 8306 8307 8308 8309 8310 8311 8312 8313 8314 8315 8316 8317 8318 8319 8320 8321 8322 8323 8324 8325 8326 8327 8328 8329 8332 8333 8334 8335 8336 8337 8338 8339 19080 19081 19082 19083 19084 19085 19086 !ERROR: interpret: no matching MIL operator to 'leftjoin(void, BAT[oid,oid])'.
MAPI  = monetdb@localhost:50000
QUERY = printf("1 ");
        var tID := 9999@0; # start counter at an arbitrary number
        printf("2 ");
        var tijah_tID   := new(void,oid).seqbase(0@0);
        printf("3 ");
        var tijah_frag  := new(void,oid).seqbase(0@0);
        printf("4 ");
        var tijah_pre   := new(void,oid).seqbase(0@0);
        printf("5 ");
        var tijah_score := new(void,dbl).seqbase(0@0);
        printf("6 ");
        var tijah_resultsz := new(lng,lng);
        printf("7 ");
        var tijah_lock  := lock_nil; # pftijah collection lock
        printf("8 ");
        module("probxml");
        printf("9 ");
        var newid_counter := 1LL; # va
ERROR = !MAYBE YOU MEAN:
        !       leftjoin(BAT[any::1,any::2], BAT[any::2,any::3]) : BAT[any::1,any::3]
        !       leftjoin(any::1, BAT[any::1,any::2]) : any::2
        !       leftjoin(oid, any::1) : any::1
        !       leftjoin(BAT[void,oid], any::1) : any::1
        !       leftjoin(BAT[oid,oid], any::1) : BAT[oid,any::1]
==========================

This happens in the following code:
==========================

                                             { # expand ()
printf("8326 ");
                                              var expOid_iter := expOid.leftfetchjoin(v_iter009);
printf("8327 ");
                                              var iter_expOid := expOid_iter.reverse();
printf("8328 ");
                                              var oidMap_expOid := outer010.chk_order().leftjoin(iter_expOid.chk_order());
printf("8329 ");
                                              if (not(ordered(reverse(outer010)) and ordered(iter_expOid) and                         ordered(reverse(iter_expOid)))) {
printf("8330 ");
                                               var temp_sort := oidMap_expOid.hmark(0@0).CTrefine(oidMap_expOid.tmark(0@0)).mirror();
printf("8331 ");
                                              }
printf("8332 ");
                                              var expOid_oidMap := oidMap_expOid.reverse();
printf("8333 ");
                                              v_iter010 := expOid_iter;
printf("8334 ");
                                             } # end of expand ()
printf("8335 ");
                                             # join ()
printf("8336 ");
                                             var cnt := count(v_iter010)*2;
printf("8337 ");
                                             var new_v_iter := v_iter010;
printf("8338 ");
                                             v_iter010 := bat(void,oid,cnt).seqbase(0@0).access(BAT_APPEND).append(new_v_iter);
printf("8339 ");
                                             var new_v_vid := oidNew_expOid.leftjoin(v_vid009);
==========================


Further inspection learns that this is caused by the "oidNew_expOid" variable which is not initialized.
When looking at the code in mil_print_summer for expand  and join I saw the initialization code is generated but seems to be optimized out by the mil_opt optimizer in compiler/mil/mil_opt.c . When running the compiler without optimizer the query runs fine.

Attachements:
ca_module.xq : the query
mil_opt.mil : the optimized code which fails (generated with extra line numbers for debugging)
mil_not_opt.mil : the non-optimized code which succeeds.

----------------------------------------------------------------------

>Comment By: SourceForge Robot (sf-robot)
Date: 2008-07-25 02:20

Message:
Logged In: YES
user_id=1312539
Originator: NO

This Tracker item was closed automatically by the system. It was
previously set to a Pending status, and the original submitter
did not respond within 365 days (the time period specified by
the administrator of this Tracker).

----------------------------------------------------------------------

Comment By: Stefan Manegold (stmane)
Date: 2007-07-25 15:36

Message:
Logged In: YES
user_id=572415
Originator: NO

Problem seems to be fixed by Sjoerd's mil_opt fixes last week.

No test added (yet?) as the provided test requires probxml (configure
--enable-probxml), which is not standard, yet(?).


----------------------------------------------------------------------

Comment By: Arthur van Bunningen (arthurvb)
Date: 2007-07-25 14:10

Message:
Logged In: YES
user_id=1045637
Originator: YES

Confirmed, it works here again as well.
Thank you for your help!,

Arthur

----------------------------------------------------------------------

Comment By: Stefan Manegold (stmane)
Date: 2007-07-24 20:40

Message:
Logged In: YES
user_id=572415
Originator: NO

Works for me with the latest CVS code (development trunk), at least after
my compilation fix (see my latest checkin):

$ Mserver </dev/null
# MonetDB Server v4.19.0
# based on GDK   v1.19.0
# Copyright (c) 1993-2007, CWI. All rights reserved.
# Compiled for x86_64-redhat-linux-gnu/64bit with 64bit OIDs; dynamically
linked.
# Visit http://monetdb.cwi.nl/ for further information.

$ pf /tmp/ca_module.xq | Mserver --set monet_prompt=''
--dbinit='module(pathfinder);' > /tmp/ca_module.out.O1
$ pf -O0 /tmp/ca_module.xq | Mserver --set monet_prompt=''
--dbinit='module(pathfinder);' > /tmp/ca_module.out.O0
$ diff /tmp/ca_module.out.O[01]

See attached file to check the output.

File Added: ca_module.out.O1

----------------------------------------------------------------------

Comment By: Arthur van Bunningen (arthurvb)
Date: 2007-07-24 20:22

Message:
Logged In: YES
user_id=1045637
Originator: YES

File Added: rules.xml

----------------------------------------------------------------------

Comment By: Arthur van Bunningen (arthurvb)
Date: 2007-07-24 20:21

Message:
Logged In: YES
user_id=1045637
Originator: YES

File Added: movies.xml

----------------------------------------------------------------------

Comment By: Arthur van Bunningen (arthurvb)
Date: 2007-07-24 20:21

Message:
Logged In: YES
user_id=1045637
Originator: YES

File Added: context_demo3.xml

----------------------------------------------------------------------

Comment By: Stefan Manegold (stmane)
Date: 2007-07-24 20:12

Message:
Logged In: YES
user_id=572415
Originator: NO

Arthur,

could you also make the required documents available?
otherwise, I cannot run your test query...

Stefan


----------------------------------------------------------------------

Comment By: Arthur van Bunningen (arthurvb)
Date: 2007-07-24 19:36

Message:
Logged In: YES
user_id=1045637
Originator: YES

Just checked with a fresh version (using monetdb-install.sh
--nightly=current --enable-xquery), it seems that the unwanted behavior
still exists.


----------------------------------------------------------------------

Comment By: Arthur van Bunningen (arthurvb)
Date: 2007-07-24 18:29

Message:
Logged In: YES
user_id=1045637
Originator: YES

Sorry, I forgot to mention that the pathfinder module should be compiled
with "--enable-probxml=yes" to run the XQuery. As can be seen from the
attachments, the MIL is actually generated. The version I used I checked
out last Friday, I will try again with a newer version.

----------------------------------------------------------------------

Comment By: Stefan Manegold (stmane)
Date: 2007-07-24 17:25

Message:
Logged In: YES
user_id=572415
Originator: NO

Arthur,

I just tried to add your attached query as a test, but it does not even
successfully pass the compiler:

$ pf /tmp/ca_module.xq
bad usage of XML namespaces: at (20,5-20,57): unknown namespace in
qualified function name pxmlsup:deep-equal
# halted in
/ufs/manegold/_/scratch0/Monet/Testing/Current/source/pathfinder/compiler/semantics/ns.c
(ns_resolve), line 851

Stefan


----------------------------------------------------------------------

Comment By: Stefan Manegold (stmane)
Date: 2007-07-24 14:48

Message:
Logged In: YES
user_id=572415
Originator: NO

Arthur,

did you try the latest development version of after Friday Jul 20?

Sjoerd fixed a potentially related MIL optimizer bug on Friday...

Stefan


----------------------------------------------------------------------

Comment By: Arthur van Bunningen (arthurvb)
Date: 2007-07-24 13:09

Message:
Logged In: YES
user_id=1045637
Originator: YES

File Added: mil_not_opt.mil.gz

----------------------------------------------------------------------

Comment By: Arthur van Bunningen (arthurvb)
Date: 2007-07-24 13:08

Message:
Logged In: YES
user_id=1045637
Originator: YES

File Added: mil_opt.mil.gz

----------------------------------------------------------------------

You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=482468&aid=1759552&group_id=56967

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Monetdb-bugs mailing list
Monetdb-bugs@...
https://lists.sourceforge.net/lists/listinfo/monetdb-bugs