Re: [sc-users] Routine / AppClock broken

[Scott Wilson-3]

Moving this to dev.

Hmm this might also be an opportunity to clean up and/or deprecate some of the ways of playing a routine. There are rather a lot of variants, which at a certain point decreases usability rather than increasing it.

S.

On 22 Jul 2008, at 09:46, Scott Wilson wrote:

This is definitely a bug not a feature, IMO, so I think it would be best to fix Clock:play (not that I have a solution to hand... ;-)

At the very least the current implementation is extremely counterintuitive.

S.

On 22 Jul 2008, at 03:35, James Harkins wrote:

Or, perhaps best, we change fork to use clock.sched(0, Routine(this)) instead of relying on play (which in its current state is a big mistake).

hjh

On Jul 21, 2008, at 10:30 PM, James Harkins wrote:

In any case, one workaround is always to write 0.yield as the first thing in your forked function.

The other way is to schedule explicitly, instead of relying on what is really nothing more than syntactic sugar. This works perfectly well without any changes:

AppClock.sched(0, Routine({ ... }))

Or, if you can't go entirely sugar-free:

AppClock.sched(0, r {  })

hjh

: H. James Harkins

: jamshark70@...

: http://www.dewdrop-world.net

.::!:.:.......:.::........:..!.::.::...:..:...:.:.:.:..:

"Come said the Muse,

Sing me a song no poet has yet chanted,

Sing me the universal."  -- Whitman

[James Harkins-2]

Moving to dev...

On Jul 22, 2008, at 2:18 AM, Sciss wrote:

hooo looks complicated. ok, i understand that fork { ... } ends up in Meta_Clock:play

this.sched(task.value(beats, seconds, this), task)

and hence the problem of task.value. but then still:

c = Condition.new;

AppClock.sched(0, r {  ("Can? " ++ thisThread.canCallOS).postln; 1.wait; ("Can? " ++ thisThread.canCallOS).postln; c.wait; ("Can? " ++ thisThread.canCallOS).postln; })

c.test = true; c.signal;

the condition wakes up the routine again on the wrong clock....

That's a separate issue -- I wasn't working on that problem last night :)

It looks to me like AppClock is something like a "virtual clock" -- its tasks run under control of the associated Scheduler object, so technically its tasks have to wake up on a "real" clock (i.e., SystemClock), but the mechanism of waking up the tasks is different. The upshot is that the routine's clock variable is wrong (even though canCallOS returns the right thing) -- then, when Condition reschedules the task on what the routine thinks is the right clock, it's actually the wrong clock.

Unfortunately I don't see a solution to this. The c++ backend overwrites the routine's clock variable upon "awake":

g->thread->beats.uf = slot[1].uf;

g->thread->seconds.uf = slot[2].uf;

g->thread->clock.ucopy = slot[3].ucopy;

g->gc->GCWrite(g->thread, slot+3);

And, since it looks to me like the back and has no knowledge of AppClock, slot[3].ucopy is never going to be AppClock and the routine will always wake up thinking it's on SystemClock.

Trying to change that carries a real risk of breaking core stuff... I don't have any ideas here at the moment.

I tend to agree about Clock:play -- it's wack, let's take out that .value nonsense.

hjh

: H. James Harkins

: jamshark70@...

: http://www.dewdrop-world.net

.::!:.:.......:.::........:..!.::.::...:..:...:.:.:.:..:

"Come said the Muse,

Sing me a song no poet has yet chanted,

Sing me the universal."  -- Whitman

[James Harkins-2]

Actually I thought of something for the Condition issue. The idea is
to have Condition remember not only the thread, but also whether that
thread could call the OS at the time it was placed under control of
the Condition. Then, if that saved value is true, reschedule the
thread on AppClock. My reasoning is that canCallOS is the only thing
that knows whether it's AppClock or not (since the thread's clock
variable is necessarily wrong for AppClock).

Also signal is a near exact quotation of unhang, so why not have
signal call unhang instead of naughty copy-paste programming?

Also, what is the time variable doing in unhang? It appears to be
vestigial - I'd say, dump it.

This version of Condition passes the sciss test (yay!).

Condition {
        var <>test, waitingThreads;
       
        *new { arg test=false;
                ^super.newCopyArgs(test, Array(8))
        }
        wait {
                if (test.value.not, {
                        waitingThreads = waitingThreads.add([thisThread.canCallOS, thisThread]);
                        nil.yield;
                });
        }
        hang { arg value;
                // ignore the test, just wait
                waitingThreads = waitingThreads.add([thisThread.canCallOS, thisThread]);
                value.yield;
        }
       
        signal {
// var tempWaitingThreads, time;
                if (test.value, {
                        this.unhang;
                });
        }
        unhang {
                var tempWaitingThreads, time;
                // ignore the test, just resume all waiting threads
                time = thisThread.seconds;
                tempWaitingThreads = waitingThreads;
                waitingThreads = nil;
                tempWaitingThreads.do({ arg thread;
                        if(thread[0]) {
                                AppClock.sched(0, thread[1]);
                        } {
                                thread[1].clock.sched(0, thread[1]);
                        };
                });
        }
}

hjh


--
James Harkins /// dewdrop world
jamshark70@...
http://www.dewdrop-world.net

"Come said the Muse,
Sing me a song no poet has yet chanted,
Sing me the universal." -- Whitman

_______________________________________________
sc-dev mailing list

info (subscribe and unsubscribe): http://swiki.hfbk-hamburg.de:8888/MusicTechnology/880
archive: http://www.listarc.bham.ac.uk/marchives/sc-dev/
search: http://www.listarc.bham.ac.uk/lists/sc-dev/search/

[Julian Rohrhuber]

>  (since the thread's clock
>variable is necessarily wrong for AppClock).

why is this so? It seems to me the source of the problem and a
workaround seems unappealing.

>Also signal is a near exact quotation of unhang, so why not have
>signal call unhang instead of naughty copy-paste programming?
>
>Also, what is the time variable doing in unhang? It appears to be
>vestigial - I'd say, dump it.

yes, please remove time.

>This version of Condition passes the sciss test (yay!).
>
>Condition {
> var <>test, waitingThreads;
>
> *new { arg test=false;
> ^super.newCopyArgs(test, Array(8))
> }
> wait {
> if (test.value.not, {
> waitingThreads =
>waitingThreads.add([thisThread.canCallOS, thisThread]);
> nil.yield;
> });
> }
> hang { arg value;
> // ignore the test, just wait
> waitingThreads =
>waitingThreads.add([thisThread.canCallOS, thisThread]);
> value.yield;
> }
>
> signal {
>// var tempWaitingThreads, time;
> if (test.value, {
> this.unhang;
> });
> }
> unhang {
> var tempWaitingThreads, time;
> // ignore the test, just resume all waiting threads
> time = thisThread.seconds;
> tempWaitingThreads = waitingThreads;
> waitingThreads = nil;
> tempWaitingThreads.do({ arg thread;
> if(thread[0]) {
> AppClock.sched(0, thread[1]);
> } {
> thread[1].clock.sched(0, thread[1]);
> };
> });
> }
>}
>
>hjh
>
>
>--
>James Harkins /// dewdrop world
>jamshark70@...
>http://www.dewdrop-world.net
>
>"Come said the Muse,
>Sing me a song no poet has yet chanted,
>Sing me the universal." -- Whitman
>
>_______________________________________________
>sc-dev mailing list
>
>info (subscribe and unsubscribe):
>http://swiki.hfbk-hamburg.de:8888/MusicTechnology/880
>archive: http://www.listarc.bham.ac.uk/marchives/sc-dev/
>search: http://www.listarc.bham.ac.uk/lists/sc-dev/search/

--





.

_______________________________________________
sc-dev mailing list

info (subscribe and unsubscribe): http://swiki.hfbk-hamburg.de:8888/MusicTechnology/880
archive: http://www.listarc.bham.ac.uk/marchives/sc-dev/
search: http://www.listarc.bham.ac.uk/lists/sc-dev/search/

[James Harkins-2]

On Tue, Jul 22, 2008 at 2:39 PM, Julian Rohrhuber
<rohrhuber@...> wrote:
>>  (since the thread's clock
>> variable is necessarily wrong for AppClock).
>
> why is this so? It seems to me the source of the problem and a workaround
> seems unappealing.

I guess you missed my email earlier this morning?

It's deep in the c++ layer. If my read of the source is correct, a
true fix will be extremely difficult and probably quite risky. So, if
you want to muck around with it and break the virtual machine, go
ahead, knock yourself out. I'll just make sure never to update my
sources again! :-p

hjh


--
James Harkins /// dewdrop world
jamshark70@...
http://www.dewdrop-world.net

"Come said the Muse,
Sing me a song no poet has yet chanted,
Sing me the universal." -- Whitman

_______________________________________________
sc-dev mailing list

info (subscribe and unsubscribe): http://swiki.hfbk-hamburg.de:8888/MusicTechnology/880
archive: http://www.listarc.bham.ac.uk/marchives/sc-dev/
search: http://www.listarc.bham.ac.uk/lists/sc-dev/search/

[Julian Rohrhuber]

>On Tue, Jul 22, 2008 at 2:39 PM, Julian Rohrhuber
><rohrhuber@...> wrote:
>>>   (since the thread's clock
>>>  variable is necessarily wrong for AppClock).
>>
>>  why is this so? It seems to me the source of the problem and a workaround
>>  seems unappealing.
>
>I guess you missed my email earlier this morning?
>
>It's deep in the c++ layer. If my read of the source is correct, a
>true fix will be extremely difficult and probably quite risky. So, if
>you want to muck around with it and break the virtual machine, go
>ahead, knock yourself out. I'll just make sure never to update my
>sources again! :-p

?? I don't get your joke.

Sorry, back to the topic - yes I did not see the the earlier posts. I
would suggest to first fix the .play methods as was suggested earlier.

Regarding the awake issue, maybe the awake-message could be fixed?

Routine-awake { arg inBeats, inSeconds, inClock;
                var temp = inBeats; // prevent optimization
                this.clock = inClock ? this.clock;
                ^this.next(inBeats)
}

it seems to be called only in Scheduler.

But - where is the c-code called that you are referring to?
--





.

_______________________________________________
sc-dev mailing list

info (subscribe and unsubscribe): http://swiki.hfbk-hamburg.de:8888/MusicTechnology/880
archive: http://www.listarc.bham.ac.uk/marchives/sc-dev/
search: http://www.listarc.bham.ac.uk/lists/sc-dev/search/

[James Harkins-2]

On Tue, Jul 22, 2008 at 5:15 PM, Julian Rohrhuber
<rohrhuber@...> wrote:
> ?? I don't get your joke.

A bit of an exaggeration... I would choose not to update my sources
until all of the inevitable bugs from the fix were worked out :)

> Regarding the awake issue, maybe the awake-message could be fixed?
>
> Routine-awake { arg inBeats, inSeconds, inClock;
>                var temp = inBeats; // prevent optimization
>                this.clock = inClock ? this.clock;
>                ^this.next(inBeats)
> }

This is worth testing. I'm not sure whether the c++ stuff runs before
or after this (but I suspect it's probably before).

> But - where is the c-code called that you are referring to?

It's in initAwakeMessage -- I think this is in PyrInterpreter.cpp
(could be wrong, going from memory now).
hjh


--
James Harkins /// dewdrop world
jamshark70@...
http://www.dewdrop-world.net

"Come said the Muse,
Sing me a song no poet has yet chanted,
Sing me the universal." -- Whitman

_______________________________________________
sc-dev mailing list

info (subscribe and unsubscribe): http://swiki.hfbk-hamburg.de:8888/MusicTechnology/880
archive: http://www.listarc.bham.ac.uk/marchives/sc-dev/
search: http://www.listarc.bham.ac.uk/lists/sc-dev/search/

[Julian Rohrhuber]

>
>>  Regarding the awake issue, maybe the awake-message could be fixed?
>>
>>  Routine-awake { arg inBeats, inSeconds, inClock;
>>                 var temp = inBeats; // prevent optimization
>>                 this.clock = inClock ? this.clock;
>>                 ^this.next(inBeats)
>>  }
>
>This is worth testing. I'm not sure whether the c++ stuff runs before
>or after this (but I suspect it's probably before).
>
>>  But - where is the c-code called that you are referring to?
>
>It's in initAwakeMessage -- I think this is in PyrInterpreter.cpp
>(could be wrong, going from memory now).

yes, I was wondering where this message was called from sclang.

As far as I see this is called only a number of times in the
beginning, after a recompile, in

compileSucceeded()
--





.

_______________________________________________
sc-dev mailing list

info (subscribe and unsubscribe): http://swiki.hfbk-hamburg.de:8888/MusicTechnology/880
archive: http://www.listarc.bham.ac.uk/marchives/sc-dev/
search: http://www.listarc.bham.ac.uk/lists/sc-dev/search/

[James Harkins-2]

On Jul 22, 2008, at 5:15 PM, Julian Rohrhuber wrote:

Routine-awake { arg inBeats, inSeconds, inClock;

var temp = inBeats; // prevent optimization

this.clock = inClock ? this.clock;

^this.next(inBeats)

}

it seems to be called only in Scheduler.

No, doesn't work. Using the sciss test:

c = Condition.new;

AppClock.sched(0, r {

("Can? " ++ thisThread.canCallOS).postln;

thisThread.clock.debug("thisThread.clock");

1.wait;

("Can? " ++ thisThread.canCallOS).postln;

thisThread.clock.debug("thisThread.clock");

c.wait;

("Can? " ++ thisThread.canCallOS).postln;

thisThread.clock.debug("thisThread.clock");

});

c.test = true; c.signal;

I also put a debugging statement into Routine:awake to print out the clock that is passed in. Here's the output:

>> Routine:awake: class AppClock

Can? true

thisThread.clock: class SystemClock

>> Routine:awake: class AppClock

Can? true

thisThread.clock: class SystemClock

a Condition

>> Routine:awake: class SystemClock

Can? false

thisThread.clock: class SystemClock

The clock passed into Routine:awake is right, but the thread's clock inside the routine is not. So I checked the source for prRoutineResume (which handles 'next' for routines) and (drumroll please) the thread's clock is set here as well.

thread->beats.uf = g->thread->beats.uf;

thread->seconds.uf = g->thread->seconds.uf;

thread->clock.ucopy = g->thread->clock.ucopy;

g->gc->GCWrite(thread, &g->thread->clock);

So there is nothing we can do within the language. No matter what we set the clock to, the interpreter will overwrite it before allowing the routine to resume.

Down the c++ rabbit hole then... I'm not going to touch that myself. First we have to establish whether the backend regards AppClock the same way it sees SystemClock or TempoClock -- and, because of AppClock's language-side implementation, I have reason to doubt it.

hjh

: H. James Harkins

: jamshark70@...

: http://www.dewdrop-world.net

.::!:.:.......:.::........:..!.::.::...:..:...:.:.:.:..:

"Come said the Muse,

Sing me a song no poet has yet chanted,

Sing me the universal."  -- Whitman

[Julian Rohrhuber]

>
>The clock passed into Routine:awake is right, but the thread's clock
>inside the routine is not. So I checked the source for
>prRoutineResume (which handles 'next' for routines) and (drumroll
>please) the thread's clock is set here as well.
>
> thread->beats.uf = g->thread->beats.uf;
> thread->seconds.uf = g->thread->seconds.uf;
> thread->clock.ucopy = g->thread->clock.ucopy;
> g->gc->GCWrite(thread, &g->thread->clock);
>
>So there is nothing we can do within the language. No matter what we
>set the clock to, the interpreter will overwrite it before allowing
>the routine to resume.
>
>Down the c++ rabbit hole then... I'm not going to touch that myself.
>First we have to establish whether the backend regards AppClock the
>same way it sees SystemClock or TempoClock -- and, because of
>AppClock's language-side implementation, I have reason to doubt it.

Maybe it is also that in switchToThread() the parent thread's clock
is not saved. This is not a problem in TempoClock, because it sets
the clock explicitly from the primitive prTempoClock_Sched.

Btw. the simplest test I have found is this:

AppClock.sched(0, r { thisThread.clock.debug("thisThread.clock"); })
AppClock.sched(0, r { thisThread.clock  = AppClock; 1.wait;
thisThread.clock.debug("thisThread.clock"); })

// see that resuming restores the old thread clock.
TempoClock.default.sched(0, r { thisThread.clock  = SystemClock;
1.wait; thisThread.clock.debug("thisThread.clock"); })


I'm convinced this is the bug and should be solved somehow (not a
workaround in the Scheduler). I've added a comment to the bug report:
http://sourceforge.net/tracker/index.php?func=detail&aid=2023852&group_id=54622&atid=474248
--





.

_______________________________________________
sc-dev mailing list

info (subscribe and unsubscribe): http://swiki.hfbk-hamburg.de:8888/MusicTechnology/880
archive: http://www.listarc.bham.ac.uk/marchives/sc-dev/
search: http://www.listarc.bham.ac.uk/lists/sc-dev/search/

[Julian Rohrhuber]

So it turned out that AppClock.tick is not called from the right
thread, or rather that its clock is not the AppClock. The attached
diff fixes this problem by providing a thread and resetting its
clock. This solves Sciss's issue about Condition with AppClock.

Please someone commit this fix, I don't have access to svn currently.
--





.

[James Harkins-2]

Done. Very clever - I like it.

hjh

On Jul 24, 2008, at 8:28 AM, Julian Rohrhuber wrote:

So it turned out that AppClock.tick is not called from the right thread, or rather that its clock is not the AppClock. The attached diff fixes this problem by providing a thread and resetting its clock. This solves Sciss's issue about Condition with AppClock.

Please someone commit this fix, I don't have access to svn currently.

: H. James Harkins

: jamshark70@...

: http://www.dewdrop-world.net

.::!:.:.......:.::........:..!.::.::...:..:...:.:.:.:..:

"Come said the Muse,

Sing me a song no poet has yet chanted,

Sing me the universal."  -- Whitman