[squeak-dev] Threaded ODBC

[Rob Rothwell-2]

Hello,

I was just wondering if anyone knew what work would be required to provide a threaded ODBC connection in Squeak similar to what is available for VisualWorks or Smalltalk/X.  The ability to not freeze the VM while running a query would be very useful for me, but I don't even know where to start.

Or, is it a bigger issue than just ODBC, something buried deep within the VM and/or the FFI?

Thanks,

Rob

[Igor Stasenko]

2008/5/10 Rob Rothwell <r.j.rothwell@...>:

> Hello,
>
> I was just wondering if anyone knew what work would be required to provide a
> threaded ODBC connection in Squeak similar to what is available for
> VisualWorks or Smalltalk/X.  The ability to not freeze the VM while running
> a query would be very useful for me, but I don't even know where to start.
>
> Or, is it a bigger issue than just ODBC, something buried deep within the VM
> and/or the FFI?
>

Someone could make a plugin, which will call ODBC from separate native
thread, without blocking VM thread and blocking only  caller Process.
It can be even me :)

Also see an GSoC OpenDBX project, not taken by anyone, but i seen
someone recently started to code initial bindings to it.
http://wiki.squeak.org/squeak/6036
Who is it? Please raise the hand! :)

> Thanks,
>
> Rob
>


--
Best regards,
Igor Stasenko AKA sig.

[Klaus D. Witzel]

On Sat, 10 May 2008 20:06:22 +0200, Igor Stasenko wrote:

> 2008/5/10 Rob Rothwell wrote:
>> Hello,
>>
>> I was just wondering if anyone knew what work would be required to  
>> provide a
>> threaded ODBC connection in Squeak similar to what is available for
>> VisualWorks or Smalltalk/X.  The ability to not freeze the VM while  
>> running
>> a query would be very useful for me, but I don't even know where to  
>> start.
>>
>> Or, is it a bigger issue than just ODBC, something buried deep within  
>> the VM
>> and/or the FFI?
>>
>
> Someone could make a plugin, which will call ODBC from separate native
> thread, without blocking VM thread and blocking only  caller Process.
> It can be even me :)
>
> Also see an GSoC OpenDBX project, not taken by anyone, but i seen
> someone recently started to code initial bindings to it.
> http://wiki.squeak.org/squeak/6036
> Who is it? Please raise the hand! :)

Did you mean

- http://www.squeaksource.com/SqueakDBX.html

/Klaus

>> Thanks,
>>
>> Rob
>>
>
>

[Rob Rothwell-2]

On 5/10/08, Igor Stasenko <siguctua@...> wrote:

Someone could make a plugin, which will call ODBC from separate native
thread, without blocking VM thread and blocking only  caller Process.
It can be even me :)

Hmm...my Windows threading capablilities are slim to non-existent, and I have yet to create a Squeak plugin.  However, I think I could review the "C Way" of ODBC from MSDN and the Smalltalk/X inline C as a start if there is any way I could be useful at all!

Is this really something you would be interested in?  This is a big enough issue for me to keep considering "one of the other Smalltalks" over, but I keep coming back to Squeak (in spite of it being rough around the edges) because of the truly brilliant, engaging community.  

I keep looking for something I am actually good enough at to help out with and contribute to, and once again, I am a little out of my depth, but I could dust off my ancient memories of C (I would honestly be better in straight assembly using something like FASM) and try to help out if you are interested and I wouldn't get in the way too much...

Thanks again,

Rob
 

[Rob Rothwell-2]

Some parts of this message have been removed. Learn more about Nabble's security policy.

On 5/10/08, Klaus D. Witzel <klaus.witzel@...> wrote:

Also see an GSoC OpenDBX project, not taken by anyone, but i seen
someone recently started to code initial bindings to it.
http://wiki.squeak.org/squeak/6036
Who is it? Please raise the hand! :)

Did you mean

- http://www.squeaksource.com/SqueakDBX.html

Does anyone have any experience with this?  What makes it different from the standard ODBC?

Rob

[Igor Stasenko]

2008/5/10 Rob Rothwell <r.j.rothwell@...>:

> On 5/10/08, Klaus D. Witzel <klaus.witzel@...> wrote:
>>>
>>> Also see an GSoC OpenDBX project, not taken by anyone, but i seen
>>> someone recently started to code initial bindings to it.
>>> http://wiki.squeak.org/squeak/6036
>>> Who is it? Please raise the hand! :)
>>
>> Did you mean
>>
>> - http://www.squeaksource.com/SqueakDBX.html
>
> Does anyone have any experience with this?  What makes it different from the
> standard ODBC?
>

You can read about it here
http://www.linuxnetworks.de/doc/index.php/OpenDBX


> Rob
>
>
>
>



--
Best regards,
Igor Stasenko AKA sig.

[Igor Stasenko]

2008/5/10 Klaus D. Witzel <klaus.witzel@...>:

> On Sat, 10 May 2008 20:06:22 +0200, Igor Stasenko wrote:
>
>> 2008/5/10 Rob Rothwell wrote:
>>>
>>> Hello,
>>>
>>> I was just wondering if anyone knew what work would be required to
>>> provide a
>>> threaded ODBC connection in Squeak similar to what is available for
>>> VisualWorks or Smalltalk/X.  The ability to not freeze the VM while
>>> running
>>> a query would be very useful for me, but I don't even know where to
>>> start.
>>>
>>> Or, is it a bigger issue than just ODBC, something buried deep within the
>>> VM
>>> and/or the FFI?
>>>
>>
>> Someone could make a plugin, which will call ODBC from separate native
>> thread, without blocking VM thread and blocking only  caller Process.
>> It can be even me :)
>>
>> Also see an GSoC OpenDBX project, not taken by anyone, but i seen
>> someone recently started to code initial bindings to it.
>> http://wiki.squeak.org/squeak/6036
>> Who is it? Please raise the hand! :)
>
> Did you mean
>
> - http://www.squeaksource.com/SqueakDBX.html
>

Yes.
It is likely, that i will need to use DB connection with MS-Sql and
MySql servers from squeak, so i'm interested in progress of this as
well.



--
Best regards,
Igor Stasenko AKA sig.

[Rob Rothwell-2]

On 5/10/08, Igor Stasenko <siguctua@...> wrote:

You can read about it here
http://www.linuxnetworks.de/doc/index.php/OpenDBX

So...a Linux implementation?

I am stuck with Windows at work!
 
Rob

[Rob Rothwell-2]

On 5/10/08, Igor Stasenko <siguctua@...> wrote:

Yes.
It is likely, that i will need to use DB connection with MS-Sql and
MySql servers from squeak, so i'm interested in progress of this as
well.

Well...it would be fun to actually contribute something useful.  I'm not sure what the next step would be, short of moving off the list and beginning the conversation!

I guess I am most interested in making Squeak..."business useful" for "non-genious" users like myself!

Rob

[Igor Stasenko]

2008/5/10 Rob Rothwell <r.j.rothwell@...>:

> On 5/10/08, Igor Stasenko <siguctua@...> wrote:
>>
>> Yes.
>> It is likely, that i will need to use DB connection with MS-Sql and
>> MySql servers from squeak, so i'm interested in progress of this as
>> well.
>
> Well...it would be fun to actually contribute something useful.  I'm not
> sure what the next step would be, short of moving off the list and beginning
> the conversation!
>
> I guess I am most interested in making Squeak..."business useful" for
> "non-genious" users like myself!
>

I can help with windoze port of openDbx. Lets hope its wouldn't be too hard.
Also i can help with making generic binding of openDBX to squeak by
implementing a plugin.
But i'm not yet ready to start with it :)

> Rob
>



--
Best regards,
Igor Stasenko AKA sig.

[Rob Rothwell-2]

On 5/10/08, Igor Stasenko <siguctua@...> wrote:

2008/5/10 Rob Rothwell <r.j.rothwell@...>:

I can help with windoze port of openDbx. Lets hope its wouldn't be too hard.
Also i can help with making generic binding of openDBX to squeak by
implementing a plugin.
But i'm not yet ready to start with it :)

Sounds good.  Is there anything I can try to learn in the meantime that would make me more useful?

Rob

[matthewf]

I wrote a script that removes traits from a 3.9 or 3.8 image:
http://installer.pbwiki.org/UnloadTraits

In an image with Installer (preferably LPF), do:
Installer install: 'UnloadTraits'

I tested this in 3.10, 3.9, and 3.9.1

What it does:
- Stop all trait activity
- Recompile all classes into the old format
- Remove all traits from the system
- Get rid of pointers to traits or traits-related classes
- Remove/modify lots of methods that understand traits
  functionality
- Unload the traits package
- Install 3 stub methods so that Monticello versions descended
  from 3.9 still work

This is not a modular solution, but it does make a 3.10 or 3.9
system with no traits around anywhere, and the resulting image
is quite stable. I don't intend to make Traits loadable right
now, but if you want to, I can answer questions.

Have fun

--
Matthew Fulmer -- http://mtfulmer.wordpress.com/

[Klaus D. Witzel]

On Sat, 10 May 2008 19:45:57 +0200, Rob Rothwell wrote:

> Hello,
>
> I was just wondering if anyone knew what work would be required to  
> provide a
> threaded ODBC connection in Squeak similar to what is available for
> VisualWorks or Smalltalk/X.  The ability to not freeze the VM while  
> running
> a query would be very useful for me, but I don't even know where to  
> start.

Out of curiosity, what would it be that you intend to let the users do  
while longish queries are tickling the db?

I ask with this background in mind: over several years I headed a team of  
developers who produced 1.5 releases/year of a  
(fat+GUI)client<->(shared+fast)db server app with between 6 and 60 GB data  
etc, 50+ installation sites (institutions with their own db server) and  
100+ users, with 98% of the db data being updated over night  
(incrementally). We brought down the app's db response time from minutes  
to seconds but still never found any reason for using non-blocking  
queries, because nothing much was there that could have been done by the  
user in the meantime.

Do you just want to see non-freezing queries running or have some other  
reason for that need?

/Klaus

> Or, is it a bigger issue than just ODBC, something buried deep within  
> the VM
> and/or the FFI?
>
> Thanks,
>
> Rob

[Philippe Marschall]

2008/5/11, Klaus D. Witzel <klaus.witzel@...>:

> On Sat, 10 May 2008 19:45:57 +0200, Rob Rothwell wrote:
>
>
> > Hello,
> >
> > I was just wondering if anyone knew what work would be required to provide
> a
> > threaded ODBC connection in Squeak similar to what is available for
> > VisualWorks or Smalltalk/X.  The ability to not freeze the VM while
> running
> > a query would be very useful for me, but I don't even know where to start.
> >
>
>  Out of curiosity, what would it be that you intend to let the users do
> while longish queries are tickling the db?

If you use Seaside: reply to other requests.

Cheers
Philippe

>  I ask with this background in mind: over several years I headed a team of
> developers who produced 1.5 releases/year of a
> (fat+GUI)client<->(shared+fast)db server app with between 6
> and 60 GB data etc, 50+ installation sites (institutions with their own db
> server) and 100+ users, with 98% of the db data being updated over night
> (incrementally). We brought down the app's db response time from minutes to
> seconds but still never found any reason for using non-blocking queries,
> because nothing much was there that could have been done by the user in the
> meantime.
>
>  Do you just want to see non-freezing queries running or have some other
> reason for that need?
>
>  /Klaus
>
>
>
> > Or, is it a bigger issue than just ODBC, something buried deep within the
> VM
> > and/or the FFI?
> >
> > Thanks,
> >
> > Rob
> >
>
>
>
>

[Klaus D. Witzel]

On Sun, 11 May 2008 10:22:46 +0200, Philippe Marschall wrote:

> 2008/5/11, Klaus D. Witzel wrote:
>> On Sat, 10 May 2008 19:45:57 +0200, Rob Rothwell wrote:
>>
>> > Hello,
>> >
>> > I was just wondering if anyone knew what work would be required to  
>> provide
>> a
>> > threaded ODBC connection in Squeak similar to what is available for
>> > VisualWorks or Smalltalk/X.  The ability to not freeze the VM while
>> running
>> > a query would be very useful for me, but I don't even know where to  
>> start.
>> >
>>
>>  Out of curiosity, what would it be that you intend to let the users do
>> while longish queries are tickling the db?
>
> If you use Seaside: reply to other requests.

Sure, goes without saying. *but*what*else* ?

> Cheers
> Philippe
>
>>  I ask with this background in mind: over several years I headed a team  
>> of
>> developers who produced 1.5 releases/year of a
>> (fat+GUI)client<->(shared+fast)db server app with between 6
>> and 60 GB data etc, 50+ installation sites (institutions with their own  
>> db
>> server) and 100+ users, with 98% of the db data being updated over night
>> (incrementally). We brought down the app's db response time from  
>> minutes to
>> seconds but still never found any reason for using non-blocking queries,
>> because nothing much was there that could have been done by the user in  
>> the
>> meantime.
>>
>>  Do you just want to see non-freezing queries running or have some other
>> reason for that need?
>>
>>  /Klaus
>>
>>
>>
>> > Or, is it a bigger issue than just ODBC, something buried deep within  
>> the
>> VM
>> > and/or the FFI?
>> >
>> > Thanks,
>> >
>> > Rob
>> >
>>
>>
>>
>>
>
>

[Janko Mivsek]

Klaus D. Witzel wrote:
> Philippe Marschall wrote:

>>>  Out of curiosity, what would it be that you intend to let the users do
>>> while longish queries are tickling the db?
>>
>> If you use Seaside: reply to other requests.
>
> Sure, goes without saying. *but*what*else* ?

For web apps it is really important not to block a whole web app server
while processing a single SQL request. What else? It can be another SQL
request running in parallel, or just building some web pages from a
cached data, etc etc. To repeat, threaded ODBC or something similar is
really needed for Squeak to be used in web app world.

Janko

--
Janko Mivšek
AIDA/Web
Smalltalk Web Application Server
http://www.aidaweb.si

[Philippe Marschall]

2008/5/11, Klaus D. Witzel <klaus.witzel@...>:

> On Sun, 11 May 2008 10:22:46 +0200, Philippe Marschall wrote:
>
>
> > 2008/5/11, Klaus D. Witzel wrote:
> >
> > > On Sat, 10 May 2008 19:45:57 +0200, Rob Rothwell wrote:
> > >
> > > > Hello,
> > > >
> > > > I was just wondering if anyone knew what work would be required to
> provide
> > > a
> > > > threaded ODBC connection in Squeak similar to what is available for
> > > > VisualWorks or Smalltalk/X.  The ability to not freeze the VM while
> > > running
> > > > a query would be very useful for me, but I don't even know where to
> start.
> > > >
> > >
> > >  Out of curiosity, what would it be that you intend to let the users do
> > > while longish queries are tickling the db?
> > >
> >
> > If you use Seaside: reply to other requests.
> >
>
>  Sure, goes without saying. *but*what*else* ?

Doing the usual GUI updates (hover, progress bar, whatsoever) so that
the application does not appear "frozen".

> > Cheers
> > Philippe
> >
> >
> > >  I ask with this background in mind: over several years I headed a team
> of
> > > developers who produced 1.5 releases/year of a
> > > (fat+GUI)client<->(shared+fast)db server app with
> between 6
> > > and 60 GB data etc, 50+ installation sites (institutions with their own
> db
> > > server) and 100+ users, with 98% of the db data being updated over night
> > > (incrementally). We brought down the app's db response time from minutes
> to
> > > seconds but still never found any reason for using non-blocking queries,
> > > because nothing much was there that could have been done by the user in
> the
> > > meantime.
> > >
> > >  Do you just want to see non-freezing queries running or have some other
> > > reason for that need?
> > >
> > >  /Klaus
> > >
> > >
> > >
> > > > Or, is it a bigger issue than just ODBC, something buried deep within
> the
> > > VM
> > > > and/or the FFI?
> > > >
> > > > Thanks,
> > > >
> > > > Rob
> > > >
> > >
> > >
> > >
> > >
> > >
> >
> >
> >
>
>
>
>

[Klaus D. Witzel]

On Sat, 10 May 2008 21:08:53 +0200, Rob Rothwell wrote:

> On 5/10/08, Igor Stasenko wrote:
>>
>> 2008/5/10 Rob Rothwell :
>>
>> I can help with windoze port of openDbx. Lets hope its wouldn't be too
>> hard.
>> Also i can help with making generic binding of openDBX to squeak by
>> implementing a plugin.
>> But i'm not yet ready to start with it :)
>>
>
> Sounds good.  Is there anything I can try to learn in the meantime that
> would make me more useful?

Yes, sure. Igor+my have decided to build a new DB plugin for the  
MS$ platform, so that we perhaps no longer hear complaints about missing  
MS$+DB support (for the major/open DB vendors) in Squeak. You are invited  
to join us, we could make use of some input and also help with non-toy DB  
testing (and whatever you'd like to share or help with).

This is also an invitation to join the project, for Squeakers who asked  
for Oracle support on their MS$ platform.

Send email please.

/Klaus

> Rob

[Klaus D. Witzel]

On Sun, 11 May 2008 11:09:11 +0200, Janko Mivšek wrote:

> Klaus D. Witzel wrote:
>> Philippe Marschall wrote:
>
>>>>  Out of curiosity, what would it be that you intend to let the users  
>>>> do
>>>> while longish queries are tickling the db?
>>>
>>> If you use Seaside: reply to other requests.
>>  Sure, goes without saying. *but*what*else* ?
>
> For web apps it is really important not to block a whole web app server  
> while processing a single SQL request. What else? It can be another SQL  
> request running in parallel, or just building some web pages from a  
> cached data, etc etc. To repeat, threaded ODBC or something similar is  
> really needed for Squeak to be used in web app world.

Yes web, yes seaside, yes generic_term_for_web_framework *but*what*else* ?

> Janko
>