Nabble - Squeak - Exupery

Exupery update preparing for the 0.14 release

2008-06-15T12:59:50Z

The Exupery 0.14 release is in final testing now. The major
improvement is to register allocator performance. The register
allocator should also produce slightly better code due to the
changes. There's also now support for cascades, the last major missing
language feature.

The main gain in this release is register allocation is much faster on
large methods. Now it takes about 50% of the compilation time for all
methods, it used to take almost all the time for large methods. This
means compilation time is now roughly linear with method size.

The major improvements for 0.14 were done by mid April. The release
has been delayed due to upgrading to 3.10. Exupery's testing uncovered
two bugs in 3.10 and four tests were failing due to changes in the
base image.

Two tests have been deleted as they were trying to compile methods
that have been removed. Those methods were involved in bugs found by
the stress test. There should be unit tests that cover any changes to
the compiler required to fix them.

Two tests were failing because of cascades. Cascades have been added
so they pass again. The other option was to either delete the tests or
release with failing tests on 3.10.

All that's left to do before releasing is run the stress test again
and do some Seaside development with Exupery running. Both were done
successfully before upgrading to the latest squeak-dev 3.10 images.

Bryce
_______________________________________________
Exupery mailing list
Exupery@...
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/exupery

Re: Problem with ColouringRegisterAllocator

2008-06-05T02:32:57Z

The bug was in my code

was

machine
^ machine ifNil: [ MachineX86 new]

instead of:

machine
^ machine ifNil: [ machine := MachineX86 new]

2008/6/2 <bryce@...>:

> Igor Stasenko writes:
> > in method addInterferenceEdge: firstRegister to: secondRegister
> >
> > is stumbled upon error: key not found when it does:
> >
> > secondNode := interferenceGraph at: secondRegister.
> >
> > an interferenceGraph keys = a Set(t9 t11 t14 t10 t2 t24 t22 t19 t25
> > t3 t8 t15 eax t20 t21 t16 t11 edx t17 t6 t3 t26 t12 t2 t4 t21 ecx t16
> > t12 t9 t23 t1 t5 t20 t13 t15 t14 t5 t27 t1 t6 t19 eax t13 t4 t17 t18
> > t28 t7 t8 t18 t10 t7)
> >
> > and secondRegister = eax.
> >
> > It looks like MedMachineRegister needs #= and #hash methods to make
> > things working correctly with dictionaries.
> >
> > I'll try add these methods, lets see if issue will disappear.
>
> There should only ever be one instance of each register. The bug is
> creating the second one. That said, you may find it easier to
> impleemnt = and hash than track it down. Using identity is a
> deliberate design decision.
>
> Having multiple versions of the same item may cause problems. I
> think you may be OK with duplicate registers but am not sure.
>
> > Btw, i don't know why or where it creates another instance of same
> > register (MedMachineRegister name: #eax), in my code i using:
> > 'machine registerNamed: #eax', which should create instance only once,
> > and then return same instance for consequent calls.
>
> Bryce
> _______________________________________________
> Exupery mailing list
> Exupery@...
> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/exupery
>

--
Best regards,
Igor Stasenko AKA sig.
_______________________________________________
Exupery mailing list
Exupery@...
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/exupery

Problem with ColouringRegisterAllocator

2008-06-02T13:11:34Z

Igor Stasenko writes:
> in method addInterferenceEdge: firstRegister to: secondRegister
>
> is stumbled upon error: key not found when it does:
>
> secondNode := interferenceGraph at: secondRegister.
>
> an interferenceGraph keys = a Set(t9 t11 t14 t10 t2 t24 t22 t19 t25
> t3 t8 t15 eax t20 t21 t16 t11 edx t17 t6 t3 t26 t12 t2 t4 t21 ecx t16
> t12 t9 t23 t1 t5 t20 t13 t15 t14 t5 t27 t1 t6 t19 eax t13 t4 t17 t18
> t28 t7 t8 t18 t10 t7)
>
> and secondRegister = eax.
>
> It looks like MedMachineRegister needs #= and #hash methods to make
> things working correctly with dictionaries.
>
> I'll try add these methods, lets see if issue will disappear.

There should only ever be one instance of each register. The bug is
creating the second one. That said, you may find it easier to
impleemnt = and hash than track it down. Using identity is a
deliberate design decision.

Having multiple versions of the same item may cause problems. I
think you may be OK with duplicate registers but am not sure.

> Btw, i don't know why or where it creates another instance of same
> register (MedMachineRegister name: #eax), in my code i using:
> 'machine registerNamed: #eax', which should create instance only once,
> and then return same instance for consequent calls.

Bryce
_______________________________________________
Exupery mailing list
Exupery@...
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/exupery

Problem with ColouringRegisterAllocator

2008-06-01T11:25:06Z

in method addInterferenceEdge: firstRegister to: secondRegister

is stumbled upon error: key not found when it does:

secondNode := interferenceGraph at: secondRegister.

an interferenceGraph keys = a Set(t9 t11 t14 t10 t2 t24 t22 t19 t25
t3 t8 t15 eax t20 t21 t16 t11 edx t17 t6 t3 t26 t12 t2 t4 t21 ecx t16
t12 t9 t23 t1 t5 t20 t13 t15 t14 t5 t27 t1 t6 t19 eax t13 t4 t17 t18
t28 t7 t8 t18 t10 t7)

and secondRegister = eax.

It looks like MedMachineRegister needs #= and #hash methods to make
things working correctly with dictionaries.

I'll try add these methods, lets see if issue will disappear.

Btw, i don't know why or where it creates another instance of same
register (MedMachineRegister name: #eax), in my code i using:
'machine registerNamed: #eax', which should create instance only once,
and then return same instance for consequent calls.

--
Best regards,
Igor Stasenko AKA sig.
_______________________________________________
Exupery mailing list
Exupery@...
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/exupery

Re: Register spilling mechanism

2008-04-24T13:16:37Z

Igor Stasenko writes:
> Don't worry about it. I'm planning to overthrow the rule of C and make
> every bit in VM be implemented as native methods
> which can be compiled at run time. :)
> So, i care little about what calling convention C or other libraries having.
> All foreign calls will be handled by FFI class(es) (compiled as
> anything else as well), while inside i'll have methods compiled to
> machine code and i'm free to choose any calling convention for them,
> as well as choose own stack layout which will be convenient for GC and
> object memory.
> That's why i asked how i can control these aspects with Exupery.

If an signal comes in from the OS, then the signal will be executed on
the machine (C) stack, so the stack pointer must always be correct. If
not you risk having your C stack trashed by the signal.

> >
> > > Can be there any contracts between Exupery and caller, where i can
> > > state that code should compile under certain rules, like:
> > > - specify a set of registers, which should be preserved after given
> > > code done working
> > > - specify, what registers code expects to be preserved when doing calls
> > > - specify, what registers should be reverted just before ret/non-local
> > > jump instructions
> >
> > It would be possible to control what you want to, but there's no
> > published interface. The intermediate languages will change as
> > Exupery evolves.
> >
>
> Thank you for explanation. I will look forward for any updates
> concerning these features.
> Maybe, as temporary solution, i can patch register allocator to get
> some feedback on how many stack space i need, and then
> re-run it again.
> Or i can try to measure maximum number of live temps in code by own.

The best bet would be to read the final stack height from the register
allocator. Though for now it'll always be 64 bytes big. Look
at SpillInserter and MachineX86>>newSpillSlot. That's the code
responsible for managing the stack.

Also look at the loadStack and the storeStack instructions as they
generate the Smalltalk stack loading and storing instructions in the
register allocator after all spills have been made. This is to avoid
loading spilled values from the Smalltalk stack.

Bryce
_______________________________________________
Exupery mailing list
Exupery@...
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/exupery

Re: Register spilling mechanism

2008-04-23T17:08:09Z

2008/4/24 <bryce@...>:

>
> Igor Stasenko writes:
> > 2008/4/22 <bryce@...>:
> > >
> > > Igor Stasenko writes:
> > > > More broader explanation why i need this..
> > > > I want to allow direct stack manipulation for native methods to some extent.
> > > >
> > > > In code, one can write:
> > > >
> > > > (object1 expression1) push.
> > > > (object2 expression2) push.
> > > > address call.
> > > >
> > > > Now i need to make sure that Exupery will produce correct stack frame
> > > > for a call, regardless what code inlined in expression1/expression2
> > > > it should not interfere with top stack layout, which should be:
> > > >
> > > > <result of expression1>
> > > > <result of expression2>
> > > > <return address>
> > > >
> > > > when entering routine at (address call).
> > >
> > > Are you talking about the C stack or the Smalltalk stack?
> > >
> > > If you're talking about the Smalltalk stack then in Exupery it's only
> > > dealt with in the front half of Exupery. It's handled in the
> > > ByteCodeReader and IntermediateSimplifier. Exupery uses exactly the
> > > same stack locations as the interpreter so doesn't need to check the
> > > stack height. It'll break for exactly the same methods that will cause
> > > the interpreter to crash. Exupery does model the stack in both classes
> > > so it would be fairly easy to monitor stack height if you were
> > > interested.
> > >
> >
> > No, forget about Smalltalk stack, i plan to use Exupery at lower
> > levels to produce machine code fed by my own compiler.
> >
> > > The C stack is maintained by Exupery, and is currently a fixed size
> > > but will be dynamically sized sometime in the future. Exupery will
> > > fail to compile if it uses more C stack than exists. The amount of C
> > > stack used isn't known until after register allocation because that's
> > > where registers are spilled to.
> > >
> >
> > I don't know what you mean under 'C stack'.
> > Isn't Exupery compiles directly to machine code? Then why you speaking
> > about C stack at all? ;)
>
> The machine stack which C also uses. The stack follow's C's
> conventions for calls and returns. It's the stack that the C
> in the interpreter uses.
>
>
> > As i understand (correct me if i'm wrong) by fixed size stack you
> > mean, that after done register allocation, Exupery determines the
> > exact size of stack, and then it inserts an instruction in method's
> > preamble to allocate it, like:
> >
> > push %ebp
> > mod %ebp, %esp
> > sub %esp, stacksize
> >
> > and at return point it does reverse:
> >
> > add %esp, stacksize
> > pop %ebp
> > ret
> >
> > or, does just 'ret' , in case if caller are responsible from cleaning
> > the stack and can restore %ebp.
>
> It generates sequences like you suggest but they're generated
> as low level intermediate then instruction selected before
> register allocation. So the size is set before the number of
> spills is known. I've been using a fixed size stack frame as
> at temporary measure, it's been good enough for now, there isn't
> a great deal of pressure on that memory especially as Exupery
> can now spill directly into the context frame.
>
> If you're planning on generating C style calls in code be careful.
> If a GC happens as part of the call any object may be moved. Exupery
> doesn't really save the C (machine) stack because it doesn't have any
> state at any time it would risk making calls, it moves all the state
> back into real objects so the GC can see it.
>

Don't worry about it. I'm planning to overthrow the rule of C and make
every bit in VM be implemented as native methods
which can be compiled at run time. :)
So, i care little about what calling convention C or other libraries having.
All foreign calls will be handled by FFI class(es) (compiled as
anything else as well), while inside i'll have methods compiled to
machine code and i'm free to choose any calling convention for them,
as well as choose own stack layout which will be convenient for GC and
object memory.
That's why i asked how i can control these aspects with Exupery.

>
> > Can be there any contracts between Exupery and caller, where i can
> > state that code should compile under certain rules, like:
> > - specify a set of registers, which should be preserved after given
> > code done working
> > - specify, what registers code expects to be preserved when doing calls
> > - specify, what registers should be reverted just before ret/non-local
> > jump instructions
>
> It would be possible to control what you want to, but there's no
> published interface. The intermediate languages will change as
> Exupery evolves.
>

Thank you for explanation. I will look forward for any updates
concerning these features.
Maybe, as temporary solution, i can patch register allocator to get
some feedback on how many stack space i need, and then
re-run it again.
Or i can try to measure maximum number of live temps in code by own.

>
>
> Bryce
> _______________________________________________
> Exupery mailing list
> Exupery@...
> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/exupery
>

--
Best regards,
Igor Stasenko AKA sig.
_______________________________________________
Exupery mailing list
Exupery@...
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/exupery

Re: Register spilling mechanism

2008-04-23T14:34:21Z

Igor Stasenko writes:
> 2008/4/22 <bryce@...>:
> >
> > Igor Stasenko writes:
> > > More broader explanation why i need this..
> > > I want to allow direct stack manipulation for native methods to some extent.
> > >
> > > In code, one can write:
> > >
> > > (object1 expression1) push.
> > > (object2 expression2) push.
> > > address call.
> > >
> > > Now i need to make sure that Exupery will produce correct stack frame
> > > for a call, regardless what code inlined in expression1/expression2
> > > it should not interfere with top stack layout, which should be:
> > >
> > > <result of expression1>
> > > <result of expression2>
> > > <return address>
> > >
> > > when entering routine at (address call).
> >
> > Are you talking about the C stack or the Smalltalk stack?
> >
> > If you're talking about the Smalltalk stack then in Exupery it's only
> > dealt with in the front half of Exupery. It's handled in the
> > ByteCodeReader and IntermediateSimplifier. Exupery uses exactly the
> > same stack locations as the interpreter so doesn't need to check the
> > stack height. It'll break for exactly the same methods that will cause
> > the interpreter to crash. Exupery does model the stack in both classes
> > so it would be fairly easy to monitor stack height if you were
> > interested.
> >
>
> No, forget about Smalltalk stack, i plan to use Exupery at lower
> levels to produce machine code fed by my own compiler.
>
> > The C stack is maintained by Exupery, and is currently a fixed size
> > but will be dynamically sized sometime in the future. Exupery will
> > fail to compile if it uses more C stack than exists. The amount of C
> > stack used isn't known until after register allocation because that's
> > where registers are spilled to.
> >
>
> I don't know what you mean under 'C stack'.
> Isn't Exupery compiles directly to machine code? Then why you speaking
> about C stack at all? ;)

The machine stack which C also uses. The stack follow's C's
conventions for calls and returns. It's the stack that the C
in the interpreter uses.

> As i understand (correct me if i'm wrong) by fixed size stack you
> mean, that after done register allocation, Exupery determines the
> exact size of stack, and then it inserts an instruction in method's
> preamble to allocate it, like:
>
> push %ebp
> mod %ebp, %esp
> sub %esp, stacksize
>
> and at return point it does reverse:
>
> add %esp, stacksize
> pop %ebp
> ret
>
> or, does just 'ret' , in case if caller are responsible from cleaning
> the stack and can restore %ebp.

It generates sequences like you suggest but they're generated
as low level intermediate then instruction selected before
register allocation. So the size is set before the number of
spills is known. I've been using a fixed size stack frame as
at temporary measure, it's been good enough for now, there isn't
a great deal of pressure on that memory especially as Exupery
can now spill directly into the context frame.

If you're planning on generating C style calls in code be careful.
If a GC happens as part of the call any object may be moved. Exupery
doesn't really save the C (machine) stack because it doesn't have any
state at any time it would risk making calls, it moves all the state
back into real objects so the GC can see it.

> Can be there any contracts between Exupery and caller, where i can
> state that code should compile under certain rules, like:
> - specify a set of registers, which should be preserved after given
> code done working
> - specify, what registers code expects to be preserved when doing calls
> - specify, what registers should be reverted just before ret/non-local
> jump instructions

It would be possible to control what you want to, but there's no
published interface. The intermediate languages will change as
Exupery evolves.

Bryce
_______________________________________________
Exupery mailing list
Exupery@...
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/exupery

Re: Register spilling mechanism

2008-04-22T15:56:53Z

2008/4/22 <bryce@...>:

>
> Igor Stasenko writes:
> > More broader explanation why i need this..
> > I want to allow direct stack manipulation for native methods to some extent.
> >
> > In code, one can write:
> >
> > (object1 expression1) push.
> > (object2 expression2) push.
> > address call.
> >
> > Now i need to make sure that Exupery will produce correct stack frame
> > for a call, regardless what code inlined in expression1/expression2
> > it should not interfere with top stack layout, which should be:
> >
> > <result of expression1>
> > <result of expression2>
> > <return address>
> >
> > when entering routine at (address call).
>
> Are you talking about the C stack or the Smalltalk stack?
>
> If you're talking about the Smalltalk stack then in Exupery it's only
> dealt with in the front half of Exupery. It's handled in the
> ByteCodeReader and IntermediateSimplifier. Exupery uses exactly the
> same stack locations as the interpreter so doesn't need to check the
> stack height. It'll break for exactly the same methods that will cause
> the interpreter to crash. Exupery does model the stack in both classes
> so it would be fairly easy to monitor stack height if you were
> interested.
>

No, forget about Smalltalk stack, i plan to use Exupery at lower
levels to produce machine code fed by my own compiler.

> The C stack is maintained by Exupery, and is currently a fixed size
> but will be dynamically sized sometime in the future. Exupery will
> fail to compile if it uses more C stack than exists. The amount of C
> stack used isn't known until after register allocation because that's
> where registers are spilled to.
>

I don't know what you mean under 'C stack'.
Isn't Exupery compiles directly to machine code? Then why you speaking
about C stack at all? ;)

As i understand (correct me if i'm wrong) by fixed size stack you
mean, that after done register allocation, Exupery determines the
exact size of stack, and then it inserts an instruction in method's
preamble to allocate it, like:

push %ebp
mod %ebp, %esp
sub %esp, stacksize

and at return point it does reverse:

add %esp, stacksize
pop %ebp
ret

or, does just 'ret' , in case if caller are responsible from cleaning
the stack and can restore %ebp.

And by dynamic stack size, you mean that instead of preallocating
fixed amount of bytes on stack, in future, it can use pushes/pops in
code based on demand?

I having nothing against both models, just need a safe way how i can
manipulate stack directly without being endangered by register
allocator to corrupt stack layout which developer expecting to have.

Let me elaborate the problem in the way how i see it.
Suppose my compiler generates the code in form, what we call 'Exupery
low-level intermediate'.
So, there are a 'virtual' machine instructions arranged in blocks and
using N temporary registers.
Now, after register allocator did its job, we are know, that we need
extra K bytes on stack to hold temporary values which is not fit in
CPU registers at some points of code.
And now, an instruction selector for particular temps should use
instructions to store/read values on stack.
But also there are need to generate instructions to preallocate the
stack space to hold these values and deallocate it at some point,
where they no longer needed.
Now, who is controlling these actions? And can outer subsystem
influence where to put stack allocation instruction, for instance?

Can be there any contracts between Exupery and caller, where i can
state that code should compile under certain rules, like:
- specify a set of registers, which should be preserved after given
code done working
- specify, what registers code expects to be preserved when doing calls
- specify, what registers should be reverted just before ret/non-local
jump instructions

> It's also possible to spill directly into the context (into the
> Smalltalk stack) which is done for stack registers. That's new in
> 0.14 and will be extended to temporaries as well.
>
> Bryce
> _______________________________________________
> Exupery mailing list
> Exupery@...
> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/exupery
>

--
Best regards,
Igor Stasenko AKA sig.
_______________________________________________
Exupery mailing list
Exupery@...
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/exupery

Re: Register spilling mechanism

2008-04-22T13:39:13Z

Igor Stasenko writes:
> More broader explanation why i need this..
> I want to allow direct stack manipulation for native methods to some extent.
>
> In code, one can write:
>
> (object1 expression1) push.
> (object2 expression2) push.
> address call.
>
> Now i need to make sure that Exupery will produce correct stack frame
> for a call, regardless what code inlined in expression1/expression2
> it should not interfere with top stack layout, which should be:
>
> <result of expression1>
> <result of expression2>
> <return address>
>
> when entering routine at (address call).

Are you talking about the C stack or the Smalltalk stack?

If you're talking about the Smalltalk stack then in Exupery it's only
dealt with in the front half of Exupery. It's handled in the
ByteCodeReader and IntermediateSimplifier. Exupery uses exactly the
same stack locations as the interpreter so doesn't need to check the
stack height. It'll break for exactly the same methods that will cause
the interpreter to crash. Exupery does model the stack in both classes
so it would be fairly easy to monitor stack height if you were
interested.

The C stack is maintained by Exupery, and is currently a fixed size
but will be dynamically sized sometime in the future. Exupery will
fail to compile if it uses more C stack than exists. The amount of C
stack used isn't known until after register allocation because that's
where registers are spilled to.

It's also possible to spill directly into the context (into the
Smalltalk stack) which is done for stack registers. That's new in
0.14 and will be extended to temporaries as well.

Bryce
_______________________________________________
Exupery mailing list
Exupery@...
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/exupery

Re: Debugging a bug that wasn't.

2008-04-22T12:19:07Z

Igor Stasenko writes:
> Okay. So, the problem that forcing to collect garbage after single
> allocation seem too stressing for your tests.
> Then why not divide test on parts - one for testing GC, another one
> for testing interrupts which causing active process switching?

There's no bug hence the title of this thread. For a week I thought
there was a bug because with one instance it was appearing and
disappearing based on whether I compiled the code or not. That was
just a consequence of a very close race.

Bryce
_______________________________________________
Exupery mailing list
Exupery@...
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/exupery

Re: Register spilling mechanism

2008-04-22T06:01:16Z

More broader explanation why i need this..
I want to allow direct stack manipulation for native methods to some extent.

In code, one can write:

(object1 expression1) push.
(object2 expression2) push.
address call.

Now i need to make sure that Exupery will produce correct stack frame
for a call, regardless what code inlined in expression1/expression2
it should not interfere with top stack layout, which should be:

<result of expression1>
<result of expression2>
<return address>

when entering routine at (address call).

--
Best regards,
Igor Stasenko AKA sig.
_______________________________________________
Exupery mailing list
Exupery@...
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/exupery

Register spilling mechanism

2008-04-22T05:44:08Z

I'm currently want to make an inlining for methods of specific format
(lets call them 'native' methods),
these methods should work as ST methods (same calling convention &
stack layout when you entering method), but
these methods don't do polymorphic sends unless specified exclusively,
instead all sends are inlined statically with compiler.
I don't want to describe much details here, just want you to help me
how to deal with register spilling when inlining.

Suppose i inlining a send like:

object message: a+b with: c+d with: a+d with: ....

so, all arguments to method now should use machine temporary registers
instead of stack, because it's more effective.

What i fear of that if register allocator sees that there is not
enough registers in CPU to fit them all it starts using stack for
storing temp values
and there is some situations where i need to know exact stack depth
for a method.
Is there any way in Exupery to get know, how many registers will be
spilled on stack, so this information can be used by my compiler?

--
Best regards,
Igor Stasenko AKA sig.
_______________________________________________
Exupery mailing list
Exupery@...
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/exupery

Re: Debugging a bug that wasn't.

2008-04-21T15:18:01Z

2008/4/20 <bryce@...>:

> Igor Stasenko writes:
> > GC signals a semaphore after finishing (don't remember - is weak
> > finalization process using it or different one).
> > I'm not sure, can it somehow interfere with given case?
> >
> > If weak finalization loop creates new objects, it can lead to
> > exclusive looping in it, because user processes running with lower
> > priority.
> >
>
> The problem has nothing to do with finalisation. The issue is running
> a GC for every allocation slows down execution a lot. Enough that
> there's a race to get through the test code before all 10 profiling
> threads start profiling. If they all start profiling then they can
> consume all available CPU.
>

Okay. So, the problem that forcing to collect garbage after single
allocation seem too stressing for your tests.
Then why not divide test on parts - one for testing GC, another one
for testing interrupts which causing active process switching?

>
>
> Bryce
> _______________________________________________
> Exupery mailing list
> Exupery@...
> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/exupery
>

--
Best regards,
Igor Stasenko AKA sig.
_______________________________________________
Exupery mailing list
Exupery@...
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/exupery

Re: Debugging a bug that wasn't.

2008-04-20T13:51:08Z

Igor Stasenko writes:
> GC signals a semaphore after finishing (don't remember - is weak
> finalization process using it or different one).
> I'm not sure, can it somehow interfere with given case?
>
> If weak finalization loop creates new objects, it can lead to
> exclusive looping in it, because user processes running with lower
> priority.
>

The problem has nothing to do with finalisation. The issue is running
a GC for every allocation slows down execution a lot. Enough that
there's a race to get through the test code before all 10 profiling
threads start profiling. If they all start profiling then they can
consume all available CPU.

Bryce
_______________________________________________
Exupery mailing list
Exupery@...
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/exupery

Re: Debugging a bug that wasn't.

2008-04-20T06:25:45Z

2008/4/20 <bryce@...>:

>
> I'm testing and debugging before the 0.14 release. The major benefit
> is faster compile times and better generated code. There's around a
> 10% gain or loss for the macro benchmarks depending on what's
> compiled.
>
> The bug was triggered by the below code. Originally it was compiling
> "^[42] value". The set-up shown below is what generates the fault
> without needing to compile the code. The fault was odd because it
> wasn't crashing, and wasn't stuck in an infinite loop but the image
> had locked up.
>
> I was investigating by interrupting execution with gdb and looking at
> both the Smalltalk stack using "p printCallStack()" and the C stack
> using "where". Normally, if compiled code was corrupting the object
> memory it would crash fairly quickly. That it kept executing, which
> could be seen because the Smalltalk stack kept changing was weird. If
> the GC's got a corrupt view of the object memory then normally it'll
> end up really corrupting it when it decides to try to interpret the
> middle of an object as an object header.
>
> The "Error signal." at the front of the test is because running the
> test will lock up the image. This allows the rest of the suite to
> be run.
>
> testInterruptCausesCrashes
> "Crashing is a failure"
> | processes |
> Error signal.
> self compileExpression: 'self createPoint'.
> processes := (1 to: 10) collect:
> [:each| [[(ExuperyProfiler
> spyOn: [(Delay forMilliseconds: 500) wait])
> queueCompilationCommandsOn: SharedQueue2 new] repeat] fork.
> "Force a garbage collect on every allocation"
> SmalltalkImage current vmParameterAt: 5 put: 1.
> 10000 timesRepeat: [self example].
> Exupery log: 'after running example'.
> "Reset garbage collects to a sensible value"
> SmalltalkImage current vmParameterAt: 5 put: 4000.
> processes do: [:each| each terminate].
>
> createPoint
> "^ 10 @ 20"
>
> What I think is happening is the 10 profiling processes end up
> consuming all the time and starving the user level process so it
> never completes and never resets the garbage collector's collection
> rate to a sane value. Having produced the lockup twice without any
> compiled code, I'm reasonably happy that this is the case.
>

GC signals a semaphore after finishing (don't remember - is weak
finalization process using it or different one).
I'm not sure, can it somehow interfere with given case?

If weak finalization loop creates new objects, it can lead to
exclusive looping in it, because user processes running with lower
priority.

finalizationProcess

[true] whileTrue:
[FinalizationSemaphore wait.
FinalizationLock critical:
[FinalizationDependents do:
[:weakDependent |
weakDependent ifNotNil:
[weakDependent finalizeValues.
"***Following statement is required to keep weakDependent
from holding onto its value as garbage.***"
weakDependent := nil]]]
ifError:
[:msg :rcvr | rcvr error: msg].
].

see, if FinalizationSemaphore having excess signals after executing
cycle once, it continues with execution.
I think better to place FinalizationSemaphore initSignals at the end
of weak finalization, so it wouldn't be triggered by garbage
collection which may be happen in weak finalization routines.

> Bryce
> _______________________________________________
> Exupery mailing list
> Exupery@...
> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/exupery
>

--
Best regards,
Igor Stasenko AKA sig.
_______________________________________________
Exupery mailing list
Exupery@...
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/exupery

Debugging a bug that wasn't.

2008-04-20T02:38:45Z

I'm testing and debugging before the 0.14 release. The major benefit
is faster compile times and better generated code. There's around a
10% gain or loss for the macro benchmarks depending on what's
compiled.

The bug was triggered by the below code. Originally it was compiling
"^[42] value". The set-up shown below is what generates the fault
without needing to compile the code. The fault was odd because it
wasn't crashing, and wasn't stuck in an infinite loop but the image
had locked up.

I was investigating by interrupting execution with gdb and looking at
both the Smalltalk stack using "p printCallStack()" and the C stack
using "where". Normally, if compiled code was corrupting the object
memory it would crash fairly quickly. That it kept executing, which
could be seen because the Smalltalk stack kept changing was weird. If
the GC's got a corrupt view of the object memory then normally it'll
end up really corrupting it when it decides to try to interpret the
middle of an object as an object header.

The "Error signal." at the front of the test is because running the
test will lock up the image. This allows the rest of the suite to
be run.

testInterruptCausesCrashes
"Crashing is a failure"
| processes |
Error signal.
self compileExpression: 'self createPoint'.
processes := (1 to: 10) collect:
[:each| [[(ExuperyProfiler
spyOn: [(Delay forMilliseconds: 500) wait])
queueCompilationCommandsOn: SharedQueue2 new] repeat] fork.
"Force a garbage collect on every allocation"
SmalltalkImage current vmParameterAt: 5 put: 1.
10000 timesRepeat: [self example].
Exupery log: 'after running example'.
"Reset garbage collects to a sensible value"
SmalltalkImage current vmParameterAt: 5 put: 4000.
processes do: [:each| each terminate].

createPoint
"^ 10 @ 20"

What I think is happening is the 10 profiling processes end up
consuming all the time and starving the user level process so it
never completes and never resets the garbage collector's collection
rate to a sane value. Having produced the lockup twice without any
compiled code, I'm reasonably happy that this is the case.

Bryce
_______________________________________________
Exupery mailing list
Exupery@...
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/exupery

Deciding how to fix a bug.

2008-04-12T14:40:47Z

I've started the final release testing for Exupery 0.14. There are two
bugs captured with tests so far. The first has something to do with
interrupts, it needs a few running profilers to cause the bug to
happen. I'm hoping this one is the source of the unrepeatable bugs
that were in the the previous release, if so solving it may allow
Exupery to run for much longer as a background compiler.

The second bug is caused by the current work. What's happening is the
currentContext register is being spilled but is being used by code
generated in the register allocator to load the stack. It may also be
needed by spill code to spill back into the stack.

The simplest fix is to avoid spilling the currentContext. I doubt that
spilling the currentContext is a good idea as it'll make spills of
stack registers much more complex. Not spilling currentContext is
probably the right thing to do unless the spill heuristics are
improved to avoid spilling currentContext then spilling many stack
registers that will use it to load themselves from the stack.

A theoretically better fix would be to get the register allocator to
reload the currentContext if required when generating spill code that
needs it. This is nicer as it doesn't require making another register
special. There are a few practical problems to make sure that this
doesn't lead to very bad code when we decide to spill currentContext
then spill other registers. Also, the only gain to handling
currentContext as a real register is allowing it to be spilled when
appropriate which could be very nice if it was a special register that
would be loaded from it's real location rather than spilled from the
stack.

My feeling is the right solution for now is to avoid spilling
currentContext and leave the better solution until a time when it's
appropriate to develop a more complete solution including both proper
spill heuristics and reloading spilled registers from the
interpreter's variables.

The background compiler compiled over 1200 methods before running into
one where the currentContext was spilled. So at worst, forcing it into
a register will only make 1 method in a thousand worse, and may make
it better.

Bryce
_______________________________________________
Exupery mailing list
Exupery@...
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/exupery

Floating point support

2008-03-16T14:37:00Z

Igor Stasenko writes:
> Hello Bryce,
>
> i'd like to ask you, how do you envision adding floating point support
> to Exupery
> to make it able to use FPU.
>
> The problem i see, is that it would require adding a type to registers
> to be able to use float-type values in intermediate code
> representation.
>
> Or else, how it would look like, when i need to encode floating point
> operations like:
>
> temp1 := floatValue1.
> temp2 := floatValue2.
> temp3 := temp1 + temp2. "here compiler should generate FPU
> instructions instead of integer addition"
>
> Or maybe i'm looking at problem at wrong angle?
> Maybe easier to make intermediate in format like:
>
> unaryFloatOp(argmunentAddress, resultAddress)
> binaryFloatOp(argmunent1Address, argument2Address, resultAddress)
> compareFloatsOp(arg1address, arg2address)
>
> then compiler don't have to deal with float registers (in register
> spilling code and other optimization patterns).

I'll probably create floating point operations and have floating point
registers. The operations will be to make instruction selection easy,
the registers to play nicely with the register allocator. Floating
point registers are a separate register bank independent from the
integer registers.

I'm planning on supporting SSE2 first. It should be widely enough
deployed, and it's easier to deal with than the legacy x87 stack based
floating point instruction set. SSE2 is also the default in 64 bit
mode as the number of SSE2 registers was doubled from 8 to 16 but the
x87 register set left unchanged.

> Another thing, is support of byte-wide operations, like
> loading/storing byte at specific address. And of course being able to
> do some operations with byte-sized values.
> How do you plan to support this?

With an 8 bit mem operation. Make operations that make it easy to load
and store a 8 bit quantity. Operate on it in 32 bit registers.

> The reason, why i'm asking about this, is that i'm currently busy with
> this: http://wiki.squeak.org/squeak/6041
>
> As you maybe remember i had plans to do such system before. And now i
> spent some time to push it to the point, where it can become a reality
> :)

I remember.

Bryce
_______________________________________________
Exupery mailing list
Exupery@...
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/exupery

Floating point support

2008-03-16T06:05:06Z

Hello Bryce,

i'd like to ask you, how do you envision adding floating point support
to Exupery
to make it able to use FPU.

The problem i see, is that it would require adding a type to registers
to be able to use float-type values in intermediate code
representation.

Or else, how it would look like, when i need to encode floating point
operations like:

temp1 := floatValue1.
temp2 := floatValue2.
temp3 := temp1 + temp2. "here compiler should generate FPU
instructions instead of integer addition"

Or maybe i'm looking at problem at wrong angle?
Maybe easier to make intermediate in format like:

unaryFloatOp(argmunentAddress, resultAddress)
binaryFloatOp(argmunent1Address, argument2Address, resultAddress)
compareFloatsOp(arg1address, arg2address)

then compiler don't have to deal with float registers (in register
spilling code and other optimization patterns).

Another thing, is support of byte-wide operations, like
loading/storing byte at specific address. And of course being able to
do some operations with byte-sized values.
How do you plan to support this?

The reason, why i'm asking about this, is that i'm currently busy with
this: http://wiki.squeak.org/squeak/6041

As you maybe remember i had plans to do such system before. And now i
spent some time to push it to the point, where it can become a reality
:)

--
Best regards,
Igor Stasenko AKA sig.
_______________________________________________
Exupery mailing list
Exupery@...
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/exupery

The next step towards Exupery 0.14

2008-01-08T12:02:26Z

The next step after improving the spill heuristics is likely
to be one of the following:
* Spilling context registers directly to the context
* Adding a few more primitives (String>>at: and return true, false, or
nil)
* More work on speeding up compilation
* More bug chasing.

There's two reasons to want to spill context registers directly
into the context rather than onto the C stack. First it'll reduce
memory traffic for sends with many arguments or a deep stack. At
the moment the entire stack is copied from the context into registers
when the method is entered (and possibly spilled on the C stack)
then copied back onto the context on exit. For send heavy code this
can lead to code that reads the stack into registers to immediately
write it back out again. This is especially wasteful if that section
of code doesn't use any of the stack variables copied. The second
reason is to allow temporaries and arguments to be held in registers,
if they're moved into registers then it's likely there will be more
spilling as the x86 only has 6 registers available for general use.

The compiler benchmark can try to inline String>>at: and the large
explorers benchmark can try to inline return true and false. Not
having these primitives may be causing Exupery to be slower for some
parts of both benchmarks than the interpreter. These primitives are
inlined into the main interpretation function so are especially
efficient to interpret.

Compilation of large methods has been sped up significantly but medium
and small methods have only smaller gains. Further performance gains
would be nice, if only to make it faster to run larger acceptance
tests.

Exupery is currently reliable just enough to develop with. I saw
a crash after a few hours of Seaside development. This is barely
good enough as it's easy to recover using the changes file. I don't
know how to reproduce that crash so haven't tried to debug it.

Bryce
_______________________________________________
Exupery mailing list
Exupery@...
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/exupery

Re: Compiler heuristics

2007-12-28T08:02:03Z

Igor Stasenko writes:
> On 26/12/2007, bryce@... <bryce@...> wrote:
>
> > What you're talking about sounds very much like specialisation which
> > was done by Craig Chambers in a Self VM before they started using
> > dynamic type feedback.
> >
> > The harder part in your case is figuring out what = should mean.
> > There are 126 implementers of = in my image. That said, you could
> > look in the PICs to see what classes have been used previously then
> > just deal with those cases allowing for an un-optimised fall-back
> > position. If we could guarantee that = was side effect free then
> > specialisation would be a lot easier.
> >
>
> Suppose, code provided is a code generated by compiler (to deal with oops)
> As for smallintegers, you can check a tag bit only once and then skip
> check in later code, because you already know that it's a
> smallinteger.
> It's also, can be possible for objects (if we know what = doing).
> In general, we can reduce the problem to proving that, if we got
> result from #= message and receiver is not changed, then we should
> expect to have similar result when we call #= later.
> So, we are free to replace send of #= by result of previous invocation.

If we're talking about type checking (e.g. the SmallInteger tag bit)
then the optimisation is specialisation.

For Exupery, I'll probably only look to optimise such cases after
inlining. And the more general optimisation will be left until after
specialisation to remove type checking.

A common case could be doing arithmatic on floats from a float array,
we'd want to specialise the code for the float array case then remove
all but one checks to make sure it is a float array.

Bryce
_______________________________________________
Exupery mailing list
Exupery@...
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/exupery

Re: Compiler heuristics

2007-12-26T18:27:29Z

On 26/12/2007, bryce@... <bryce@...> wrote:

> What you're talking about sounds very much like specialisation which
> was done by Craig Chambers in a Self VM before they started using
> dynamic type feedback.
>
> The harder part in your case is figuring out what = should mean.
> There are 126 implementers of = in my image. That said, you could
> look in the PICs to see what classes have been used previously then
> just deal with those cases allowing for an un-optimised fall-back
> position. If we could guarantee that = was side effect free then
> specialisation would be a lot easier.
>

Suppose, code provided is a code generated by compiler (to deal with oops)
As for smallintegers, you can check a tag bit only once and then skip
check in later code, because you already know that it's a
smallinteger.
It's also, can be possible for objects (if we know what = doing).
In general, we can reduce the problem to proving that, if we got
result from #= message and receiver is not changed, then we should
expect to have similar result when we call #= later.
So, we are free to replace send of #= by result of previous invocation.

--
Best regards,
Igor Stasenko AKA sig.
_______________________________________________
Exupery mailing list
Exupery@...
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/exupery

Compiler heuristics

2007-12-26T06:23:08Z

Igor Stasenko writes:
> It's a theoretical question, rather than practical concerning Exupery,
> but i'd like to know, what you know/thinking about special compiler
> heuristics concerning simplifying expressions, when do inlining.
>
> The problem, in basic can be demonstrated by following example.
>
> Suppose you having a code:
>
> mainMethod
> object = 0 ifTrue: [ object method1 ] ifFalse: [ object method2]
>
> and now method bodies:
>
> method1
> object = 0 ifTrue: [ do something] ifFalse: [ do something else ]
>
> method2
> object ~= 0 ifTrue: [ do something] ifFalse: [ do something else ]
>
> Now, the question is, when you inlining method1/2 into mainMethod
> you can see, that tests in inlined methods become redundant.
> If we inline both methods, code will look like:
>
> mainMethod
> object = 0 ifTrue: [
> object = 0 ifTrue: [ do something]
> ifFalse: [ do something else ] ]
> ifFalse: [
> object ~= 0 ifTrue: [ do something]
> ifFalse: [ do something else ] ]
>
> Is there any techniques, which can help in removing such redundancy,
> when analyzing code by compiler?

What you're talking about sounds very much like specialisation which
was done by Craig Chambers in a Self VM before they started using
dynamic type feedback.

The harder part in your case is figuring out what = should mean.
There are 126 implementers of = in my image. That said, you could
look in the PICs to see what classes have been used previously then
just deal with those cases allowing for an un-optimised fall-back
position. If we could guarantee that = was side effect free then
specialisation would be a lot easier.

Bryce
_______________________________________________
Exupery mailing list
Exupery@...
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/exupery

Re: Compiler heuristics

2007-12-24T19:14:12Z

> and now method bodies:
>
> method1
> object = 0 ifTrue: [ do something] ifFalse: [ do something else ]
>
> method2
> object ~= 0 ifTrue: [ do something] ifFalse: [ do something else ]
>
oops, i mistaken, obviously method bodies should use self:

method1
self = 0 ifTrue: [ do something] ifFalse: [ do something else ]

method2
self ~= 0 ifTrue: [ do something] ifFalse: [ do something else ]

--
Best regards,
Igor Stasenko AKA sig.
_______________________________________________
Exupery mailing list
Exupery@...
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/exupery

Compiler heuristics

2007-12-24T19:11:00Z

It's a theoretical question, rather than practical concerning Exupery,
but i'd like to know, what you know/thinking about special compiler
heuristics concerning simplifying expressions, when do inlining.

The problem, in basic can be demonstrated by following example.

Suppose you having a code:

mainMethod
object = 0 ifTrue: [ object method1 ] ifFalse: [ object method2]

and now method bodies:

method1
object = 0 ifTrue: [ do something] ifFalse: [ do something else ]

method2
object ~= 0 ifTrue: [ do something] ifFalse: [ do something else ]

Now, the question is, when you inlining method1/2 into mainMethod
you can see, that tests in inlined methods become redundant.
If we inline both methods, code will look like:

mainMethod
object = 0 ifTrue: [
object = 0 ifTrue: [ do something]
ifFalse: [ do something else ] ]
ifFalse: [
object ~= 0 ifTrue: [ do something]
ifFalse: [ do something else ] ]

Is there any techniques, which can help in removing such redundancy,
when analyzing code by compiler?

--
Best regards,
Igor Stasenko AKA sig.
_______________________________________________
Exupery mailing list
Exupery@...
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/exupery

Re: Thinking about Exupery 0.14

2007-12-21T13:00:46Z

Igor Stasenko writes:
>
> I suspect that main bottleneck in largeExplorers is not
> compiled/bytecode code, but memory allocations and GC.
> So, i doubt that you can gain any performance increase here.
>

Below's the raw numbers, this is from largeExplorers but with the
profiling compiler turned up to compile a bit more code. About 60% of
the time is going into the interpreter, compiled code, and primitives
that should be natively compiled. That's enough time to provide a
decent speed improvement. 70% is the normal amount spent in the
interpreter. The GC is probably only consuming about 5% of the time.

CPU: AMD64 processors, speed 1000 MHz (estimated)
Counted CPU_CLK_UNHALTED events (Cycles outside of halt state) with a unit mask of 0x00 (No unit mask) count 1000000
Counted LS_BUFFER_FULL events (LS Buffer 2 Full) with a unit mask of 0x00 (No unit mask) count 100000
Counted RETIRED_BRANCHES_MISPREDICTED events (Retired branches mispredicted) with a unit mask of 0x00 (No unit mask) count 100000
Counted RETIRED_INSNS events (Retired instructions (includes exceptions, interrupts, re-syncs)) with a unit mask of 0x00 (No unit mask) count 1000000
samples % samples % samples % samples % image name app name symbol name
1792637 57.4476 65779 12.9048 391686 84.8932 1970781 58.4824 squeak squeak interpret
223739 7.1700 150 0.0294 211 0.0457 376848 11.1829 BitBltPlugin BitBltPlugin alphaBlendwith
110588 3.5439 48505 9.5159 1302 0.2822 104008 3.0864 BitBltPlugin BitBltPlugin copyBits
76361 2.4471 124873 24.4982 3679 0.7974 64529 1.9149 libc-2.4.so libc-2.4.so (no symbols)
65089 2.0859 91160 17.8842 2155 0.4671 12648 0.3753 no-vmlinux no-vmlinux (no symbols)
60351 1.9340 51072 10.0195 2297 0.4978 31543 0.9360 anon (tgid:6681 range:0xb1c0d000-0xb7b6c000) squeak (no symbols)
52940 1.6965 5 9.8e-04 1632 0.3537 82896 2.4599 B2DPlugin B2DPlugin fillSpanfromto
45634 1.4624 12633 2.4784 2849 0.6175 59950 1.7790 BitBltPlugin BitBltPlugin copyLoopPixMap
39297 1.2593 1 2.0e-04 161 0.0349 12604 0.3740 squeak squeak sweepPhase
34504 1.1057 31 0.0061 3079 0.6673 10196 0.3026 squeak squeak lookupMethodInClass
31797 1.0190 19 0.0037 3408 0.7386 39560 1.1739 squeak squeak markAndTrace
28467 0.9123 11 0.0022 1757 0.3808 15187 0.4507 squeak squeak updatePointersInRangeFromto
28316 0.9074 197 0.0386 2484 0.5384 22647 0.6720 BitBltPlugin BitBltPlugin loadBitBltFromwarping
26469 0.8482 7 0.0014 739 0.1602 39972 1.1862 squeak squeak finalizeReference
26380 0.8454 15 0.0029 342 0.0741 38994 1.1571 squeak squeak updatePointersInRootObjectsFromto
24235 0.7766 727 0.1426 522 0.1131 29049 0.8620 squeak squeak exuperyIsNativeContext
17343 0.5558 7 0.0014 790 0.1712 9012 0.2674 squeak squeak positive32BitValueOf
15935 0.5107 5546 1.0880 336 0.0728 22525 0.6684 squeak squeak allocateheaderSizeh1h2h3doFillwith
15559 0.4986 2712 0.5321 1191 0.2581 25552 0.7582 BitBltPlugin BitBltPlugin pixPaintwith
14371 0.4605 11 0.0022 3301 0.7155 19116 0.5673 squeak squeak commonAt
13348 0.4278 466 0.0914 383 0.0830 6048 0.1795 squeak squeak lookupSelectorclass

The anon block is the native code. What's interesting is the
instructions per clock is about 0.5 while the intepreter's
instructions per clock is a little over one. The native code has less
branch mispredicts but much more memory traffic. About 8% of the time
the native code has the load store unit's buffer full and is probably
stalled waiting for a memory request to finish.

Based on the profiling I've done I'm fairly confident that one of the
reasons why the macro benchmarks are not often showing a performance
improvement on an Athon 64 is due to excess spill code causing too
much memory traffic. The register allocator is not handling heavy
register pressure well and I doubt the spill heuristics are ideal for
larger methods.

Bryce
_______________________________________________
Exupery mailing list
Exupery@...
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/exupery

Re: Thinking about Exupery 0.14

2007-12-18T13:23:50Z

On 18/12/2007, bryce@... <bryce@...> wrote:

> Igor Stasenko writes:
> > On 17/12/2007, bryce@... <bryce@...> wrote:
> > > arithmaticLoopBenchmark 1396 compiled 128 ratio: 10.906
> > > bytecodeBenchmark 2111 compiled 460 ratio: 4.589
> > > sendBenchmark 1637 compiled 668 ratio: 2.451
> > > doLoopsBenchmark 1081 compiled 715 ratio: 1.512
> > > pointCreation 1245 compiled 1317 ratio: 0.945
> > > largeExplorers 728 compiled 715 ratio: 1.018
> > > compilerBenchmark 483 compiled 489 ratio: 0.988
> > > Cumulative Time 1125 compiled 537 ratio 2.093
> > >
> > > ExuperyBenchmarks>>arithmeticLoop 249ms
> > > SmallInteger>>benchmark 1112ms
> > > InstructionStream>>interpretExtension:in:for: 113460ms
> > > Average 3155.360
>
> First, from the numbers above, I'd say that having a method that takes
> 2 minutes to compile is currently the biggest practical problem. The
> second set of numbers is a compilation time benchmark. The second
> biggest problem is that a 2.4 times increase in send speed is not
> transferring through to the two macro-benchmarks (largeExplorers and
> compilerBenchmark).
>

I suspect that main bottleneck in largeExplorers is not
compiled/bytecode code, but memory allocations and GC.
So, i doubt that you can gain any performance increase here.

> >
> > Do you make any difference between calling compiling method and , for
> > instance, a primitive function?
>
> The sender doesn't know if it's sending to a primitive or to a full
> method. If Exupery compiles a primitive then it executes in the
> senders context, just like the interpreter,
>
> > As i remember, you compiling methods to some form of a routine, which
> > can be called using cdecl convention.
> > But on top of that, knowing the fact that you calling a compiled
> > method you can use some register optimizations like passing arguments
> > in it, and in general by knowing where you changing registers, you can
> > predict what of them are changing after call, and what will stay
> > unchanged.
> > And, of course, nothing stops you from using own calling convention to
> > make code working faster. There's also a MMX/SSE registers which can
> > be used for different purposes.
> > All of the above, depending on choices, can greatly improve sends speed.
> > Just want to know, what you thinking about it.
>
> Currently Exupery uses C's calling conventions combined with the
> interpreters handling of contexts, there's plenty of room to improve
> this but I doubt that raw send speed is why the macro benchmarks
> aren't performing.
>
> Also full method inlining will change the value of other send
> optimisations by removing most of the common sends. It's the best
> optimisation for common sends. 1.0 is a base to add full method
> inlining too.
>
> > And small trick when compiling SmallInteger methods: you already know
> > that receiver is a smallinteger. So, by using that knowledge, some
> > tests can be omitted.
> > In same manner you can deal with compiling methods for classes which
> > have byte/reference indexed instances.
>
> Exupery compiles a method for each receiver so this is possible but
> not done yet. It'll get even more interesting when combined with full
> method inlining, then common self sends will become completely free.
>

Yes, and for such matter i started coding some classes to translate ST
method source to set of lambda-sends.
Lambdas are very simple and yet powerful way to represent abstract algorithm.
By having only few rules - substitution and reduction, i can transform
method code to any form.
And it's very helpful in case of full method inlining.
Method can be represented as a labmda having single free variable <context>:

lambda method(context),

where any contextual parts/actions represented as messages to context, like:

receiver: Lambda context receiver.
receiver class: Lambda context receiver class.
argument1 : Lambda context argumentAt: 1
push: Lambda context push: arg
return: Lambda context return: expression.
e.t.c.

But now, when you going to inline method, things become interesting,
because, if you know the receiver's class, then you can reduce some
labmdas at compile stage, like any accesses to receiver's class,
results of receiver method lookups e.t.c.

And even more, if you have a way how to determine if method(s) are
referentially transparent, then you can reduce some of methods to
returned results at compile time.

Like having:

isInteger
^ true

and then, in some method, when you encounter
self isInteger ifTrue: [ codeA ] ifFalse: [ codeB ]

you can reduce the whole expression to codeA, or codeB , depending on
class of receiver.
So, in given example, knowing the class of receiver, you can eliminate
two sends: #isInteger and #ifTrue:ifFalse:.
I think, that making compiler be able to detect and do such reductions
will render any other kinds of optimizations much less important.

As i know, you are translating methods by taking their bytecode.
I'm not sure, if the above is possible by compiling from bytecode. I
think it would be much easier to operate with abstract parse trees
(with lambda-sends as nodes).

> Bryce
> _______________________________________________
> Exupery mailing list
> Exupery@...
> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/exupery
>

--
Best regards,
Igor Stasenko AKA sig.
_______________________________________________
Exupery mailing list
Exupery@...
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/exupery