Proposal for a collection API in glib

[Philip Van Hoof]

Hi there,

I would like to propose this API to go into glib/gio:

http://live.gnome.org/IteratorsAPI

A working implementation of it can be found here (just replace Gee.List
with GLib.Seq, as that is the name that we have for it in mind):

http://svn.gnome.org/viewvc/libgee/trunk/gee/

To see users of this API, take a look at for example the Vala project.
On the IteratorsAPI wiki page, at the bottom, there are also a lot of
examples of projects that are right now inventing their own collections.

We are working on adding convenience functions for C to make things as
type safe as possible (the #1 problem with glib's current collection
types):

gchar*  g_iterator_get_as_string (GIterator *iter);
gdouble g_iterator_get_as_double (GIterator *iter);

A normal use-case would be a GObject in a sequence, which would with the
caller-owns API 'g_iterator_get' mean that you need to unref what you
get. We are thinking about making it possible to pass a 'free-function'
to the owner of the items (the collection), and to allow annotating how
to free what you get from 'g_iterator_get' (as it's caller owns).

Right now, with the usage of GHashTable, GPtrArray and GList, language
binding developers loose all meaningful type information.

This means that they have to resort to using manually written glue code.
This is among the reasons that makes fully automated language binding
generation a nearly impossible task at this moment.

In my opinion it's one of the reasons why we never really achieved
topnotch compelling language bindings, even though that was the #1
promise of Gtk+ back when it all started.

It's of course debatable whether or not they topnotch. They are not for
me and having a collection API that actually does make sense outside of
the C world would in my opinion be one more step in the direct direction
of improving this situation.

This of course doesn't "magically" fix existing Cism APIs. But take a
look at the wiki page mentioned above for example on how this can easily
be improved.


Thanks, and please don't troll about how great doubly linked lists are.

It's not the point.


--
Philip Van Hoof, freelance software developer
home: me at pvanhoof dot be
gnome: pvanhoof at gnome dot org
http://pvanhoof.be/blog
http://codeminded.be




_______________________________________________
gtk-devel-list mailing list
gtk-devel-list@...
http://mail.gnome.org/mailman/listinfo/gtk-devel-list

[Yu Feng]

Hi Tim,

On Thu, 2008-07-17 at 17:51 +0200, Philip Van Hoof wrote:

> Hi there,
>
> I would like to propose this API to go into glib/gio:
>
> http://live.gnome.org/IteratorsAPI
>
> A working implementation of it can be found here (just replace Gee.List
> with GLib.Seq, as that is the name that we have for it in mind):
>
> http://svn.gnome.org/viewvc/libgee/trunk/gee/
>
> To see users of this API, take a look at for example the Vala project.
> On the IteratorsAPI wiki page, at the bottom, there are also a lot of
> examples of projects that are right now inventing their own collections.
>
> We are working on adding convenience functions for C to make things as
> type safe as possible (the #1 problem with glib's current collection
> types):
>
> gchar*  g_iterator_get_as_string (GIterator *iter);
> gdouble g_iterator_get_as_double (GIterator *iter);
>

It looks like a GValue. Can it be simplified by using GValue?

> A normal use-case would be a GObject in a sequence, which would with the
> caller-owns API 'g_iterator_get' mean that you need to unref what you
> get. We are thinking about making it possible to pass a 'free-function'
> to the owner of the items (the collection), and to allow annotating how
> to free what you get from 'g_iterator_get' (as it's caller owns).
>
> Right now, with the usage of GHashTable, GPtrArray and GList, language
> binding developers loose all meaningful type information.
>
> This means that they have to resort to using manually written glue code.
> This is among the reasons that makes fully automated language binding
> generation a nearly impossible task at this moment.

I think the problem is that lacking of reference management, GList is
not intended to be used in a language binding; whereas in the reality a
lot of GTK APIs take and return GList, forcing the binding programmers
to write glue code.

> In my opinion it's one of the reasons why we never really achieved
> topnotch compelling language bindings, even though that was the #1
> promise of Gtk+ back when it all started.
>
> It's of course debatable whether or not they topnotch. They are not for
> me and having a collection API that actually does make sense outside of
> the C world would in my opinion be one more step in the direct direction
> of improving this situation.
>
> This of course doesn't "magically" fix existing Cism APIs. But take a
> look at the wiki page mentioned above for example on how this can easily
> be improved.
>
>
> Thanks, and please don't troll about how great doubly linked lists are.
>
> It's not the point.
>
>

_______________________________________________
gtk-devel-list mailing list
gtk-devel-list@...
http://mail.gnome.org/mailman/listinfo/gtk-devel-list

[Philip Van Hoof]

On Thu, 2008-07-17 at 12:46 -0400, Yu Feng wrote:

> >
> > We are working on adding convenience functions for C to make things as
> > type safe as possible (the #1 problem with glib's current collection
> > types):
> >
> > gchar*  g_iterator_get_as_string (GIterator *iter);
> > gdouble g_iterator_get_as_double (GIterator *iter);
> >
>
> It looks like a GValue. Can it be simplified by using GValue?

GValue and other GBoxed solutions where measured by Jürg (Vala project)
for performance.

His analysis showed that boxing using GBoxed and/or GValue would be a
lot slower (too slow). We would want the collection API to perform as
fast as possible, or at least make it possible to have a high performing
collection implementation:

 - Make it EASY (really easy) to make a normal performing one
 - Make it possible to make a high performance one

> I think the problem is that lacking of reference management, GList is
> not intended to be used in a language binding; whereas in the reality a
> lot of GTK APIs take and return GList, forcing the binding programmers
> to write glue code.

Exactly. With this collection API, you'd have a very automatable and
well known way to do reference management for the items that are owned
by the collection.

It's not the only reason, the other reason is that everybody seems to be
reinventing iterator based APIs:

In GLib + Gtk+ you already have three incompatible iterator APIs (or
there will be three as soon as GVariant is going to be used by people);

- GFileEnumerator
- GVariant's iterators
- GtkTreeModel

If you add to that other projects, we get this interesting list:

* Camel (CamelIterator)
  * libanjuta (IAnjutaIterable)
  * libedataserver (EIterator)
  * libgda (GdaDataModelIter)
  * libvala (using libgee)
  * Tinymail (TnyList and TnyIterator)
  * GVariant (g_variant_iter*, etc)
  * GIO (gfileenumerator.h)
  * Gtk+ (GtkTreeModel is an iterable)
  * Clutter and Tidy with ClutterModelIter
  * GStreamer with GstIterator

Clearly, whoever decided that a collections API is not necessary
(because GList, GSList, GHashTable and GPtrArray work just fine in C),
must have made a significant error in his or her judgements.


My conclusion (as usual, it's a harsh one)

I think it's proven from not only the Java, C, C++ and C# world that a
collections API is important, our own GLib based projects have and GLib
itself has started inventing ad-hoc solutions around the lack of a
generic collection API in glib.

I'm not convinced that GList, GSList, GHashTable and GPtrArray are good
components for making a sustainable and future proof API.

Not convinced at all.



Thanks for your reaction, Yu Feng.


Philip

--
Philip Van Hoof, freelance software developer
home: me at pvanhoof dot be
gnome: pvanhoof at gnome dot org
http://pvanhoof.be/blog
http://codeminded.be




_______________________________________________
gtk-devel-list mailing list
gtk-devel-list@...
http://mail.gnome.org/mailman/listinfo/gtk-devel-list

[Havoc Pennington-2]

Hi,

Here are some alternate ideas, just brainstorming:

1) have an iterator concept in gobject-introspection and map from
GList etc. in g-i. So g-i would allow you to invoke a method that
returns a list, and get an iterator back.

If I were doing this in gobject-introspection I'd tend to make the
base API use a stack-allocated lightweight iterator similar to
GtkTreeIter/GtkTextIter, and then bindings that wanted to could write
generic code to wrap that kind of lightweight iterator in a GObject.

Any language binding not using g-i has nothing to stand on if they
whine about manual work - they need to a) port to / help with g-i and
then b) we'll talk.

It would be possible to generically auto-create a GObject-based
iterator like yours, using this g-i feature.

2) Another idea would be an equivalent to registering boxed types:

g_iterator_type_register_static(const char *name, GBoxedCopyFunc
boxed_copy, GBoxedFreeFunc boxed_free, GIteratorNextFunc
iterator_next, GIteratorGetFunc iterator_get);

This would allow language bindings to generically manipulate custom
iterator types like TextIter and TreeIter, without making things a
pain in C and while keeping things lightweight. And without
redoing/breaking TreeModelIter and TextIter and all the other existing
examples you mention.



Why explore alternate ideas? Some downsides to GIterator-as-gobject:

* GObject is pretty heavyweight for something like this, and moreover
right now libglib doesn't depend on libgobject
* the need to unref the iterator is onerous for C programmers using iterators
* the need to subclass a GObject is onerous for C programmers creating iterators
* as Owen mentioned long ago when this was already discussed, we'd end
up duplicating bunches of APIs just to make them use iterators instead
of whatever they use now - this is both bloat and confusing. It would
not be realistic to ever deprecate walking GList etc. - the iterator
replacement is much less convenient, and there's way, way, way too
much code using GList already. You can't deprecate something that
touches this much code.

Havoc
_______________________________________________
gtk-devel-list mailing list
gtk-devel-list@...
http://mail.gnome.org/mailman/listinfo/gtk-devel-list

[Philip Van Hoof]

On Thu, 2008-07-17 at 13:37 -0400, Havoc Pennington wrote:

Hey Havoc,

> Here are some alternate ideas, just brainstorming:
>
> 1) have an iterator concept in gobject-introspection and map from
> GList etc. in g-i. So g-i would allow you to invoke a method that
> returns a list, and get an iterator back.

You could make a GLib.Iterator that uses gobject-introspection, but I
don't think you want to make gobject-introspection the one thing
everybody who wants to expose collections in his API has to use and
learn.

  - Make simple things, the usual use cases, easy
  - Make complex things possible

This comes from "Framework design guidelines" by Krzysztof Cwalina and
Brad Adams. I'm in agreement with that.

It's to be honest what .NET (and Java) is getting right, and we don't.

GObject-introspection on GList, GHashTable, GPtrArray and GSList wont be
simple. Well, for a C developer being an expert in GObject-introspection
and glib it might look simple. Those people should not be our only
target audience when we are tagging application developers, though.

> If I were doing this in gobject-introspection I'd tend to make the
> base API use a stack-allocated lightweight iterator similar to
> GtkTreeIter/GtkTextIter, and then bindings that wanted to could write
> generic code to wrap that kind of lightweight iterator in a GObject.
>
> Any language binding not using g-i has nothing to stand on if they
> whine about manual work - they need to a) port to / help with g-i and
> then b) we'll talk.

Disagree with this. GObject-introspection can be an aid for the many
quirks and Cisms our platform introduces. Future APIs should not focus
on just the C application developers.

It'll also be useful for other tasks that right now require scripts to
parse C code, as gobject-introspection will ship with a bunch of very
interesting tools for that (representing an existing API as an XML that
will be suitable to throw at a XSL compiler).

If GLib library authors wont focus on higher programming languages, then
I fear Gtk+ and its accompanying large set of excellent libraries will
get more and more neglected. But that's just my personal feeling.

It is what excites me most about projects like Vala: they open the eyes
of many C/GLib developers. Same thing about Tim's proposal to have a IDL
above all of GLib and Gtk+'s .h files. The IDL will force developers to
rethink their public APIs drastically (as they'll see how difficult they
have made it for higher programming language users).

I don't know why only C/GLib experts should be the ones writing language
bindings (which is the case right now). It's quite hard to find people
who are into the higher language and yet know really a lot about GLib
too. Usually these people have no reason whatsoever to care about GLib
and its isms anymore.

Usually the language binding developer provides sugar to get rid of the
GLib isms. Perhaps that's because they are ugly in modern programming
environments?

Relatively few people use doubly linked lists in C#, for example.

Vala is an excellent example of how community people who are true black
art masters and experts in GLib/GObject can provide excellent language
bindings.

Its tools just generate the vast majority! And the VAPI files are just
Vala code like any other. Just without the function bodies. For special
stuff you usually have an attribute (like C# has attributes) to finetune
the behaviour of valac to cope with the Cism.

> It would be possible to generically auto-create a GObject-based
> iterator like yours, using this g-i feature.

Yes. This I agree with.

> 2) Another idea would be an equivalent to registering boxed types:

> g_iterator_type_register_static(const char *name, GBoxedCopyFunc
> boxed_copy, GBoxedFreeFunc boxed_free, GIteratorNextFunc
> iterator_next, GIteratorGetFunc iterator_get);

This might be doable... I'll think about this proposal a bit more.

> This would allow language bindings to generically manipulate custom
> iterator types like TextIter and TreeIter, without making things a
> pain in C and while keeping things lightweight. And without
> redoing/breaking TreeModelIter and TextIter and all the other existing
> examples you mention.

*nods*

> Why explore alternate ideas? Some downsides to GIterator-as-gobject:
>
> * GObject is pretty heavyweight for something like this, and moreover
> right now libglib doesn't depend on libgobject

Agree with this. Perhaps a more lightweight GType could be used for the
iterator type. Isn't GStreamer for example providing lightweight GObject
like GTypes?

> * the need to unref the iterator is onerous for C programmers using iterators

We should focus on higher programming languages for future APIs. For
this caller owns makes most sense in my opinion.

Cairo, behdad told me, has an interesting callee-owns setup. I still
think that for iterators caller-owns is the right way, though.

Looking at the samples on that wiki page, I don't think that having to
destroy the iterator is onerous. Just good programming.

> * the need to subclass a GObject is onerous for C programmers creating iterators

It's implementing an interface instead of subclassing a GObject. With
programming languages like Vala this is easy.

And yes, I think projects like Vala, gtk-sharp (Mono), pygtk, gtkmm, etc
are the future for application developers. Not C/Glib in .c files.

Those of us who still develop in C/GLib have the GObject macros:

static void
my_iterator_g_iterator_iface_init (MyIteratorface *iface)
{
        iface->next = ...
        ...
}

G_DEFINE_TYPE_WITH_CODE (MyIterator,
        my_iterator,
        G_TYPE_OBJECT,
        G_IMPLEMENT_INTERFACE (G_TYPE_ITERATOR,
                my_iterator_g_iterator_iface_init))

http://live.gnome.org/Vala/MultiImplementInC


> * as Owen mentioned long ago when this was already discussed, we'd end
> up duplicating bunches of APIs just to make them use iterators instead
> of whatever they use now - this is both bloat and confusing. It would
> not be realistic to ever deprecate walking GList etc. - the iterator
> replacement is much less convenient, and there's way, way, way too
> much code using GList already. You can't deprecate something that
> touches this much code.

I didn't mention deprecating this code ...


Philip

--
Philip Van Hoof, freelance software developer
home: me at pvanhoof dot be
gnome: pvanhoof at gnome dot org
http://pvanhoof.be/blog
http://codeminded.be




_______________________________________________
gtk-devel-list mailing list
gtk-devel-list@...
http://mail.gnome.org/mailman/listinfo/gtk-devel-list

[Philip Van Hoof]

On Thu, 2008-07-17 at 20:06 +0200, Philip Van Hoof wrote:

> Vala is an excellent example of how community people who are true black
> art masters and experts in GLib/GObject can provide excellent language
> bindings.

I meant "who are not true black art masters and ..." of course.


--
Philip Van Hoof, freelance software developer
home: me at pvanhoof dot be
gnome: pvanhoof at gnome dot org
http://pvanhoof.be/blog
http://codeminded.be




_______________________________________________
gtk-devel-list mailing list
gtk-devel-list@...
http://mail.gnome.org/mailman/listinfo/gtk-devel-list

[Jamie McCracken-2]

On Thu, 2008-07-17 at 13:37 -0400, Havoc Pennington wrote:
> Hi,

>
> Why explore alternate ideas? Some downsides to GIterator-as-gobject:
>
> * GObject is pretty heavyweight for something like this, and moreover
> right now libglib doesn't depend on libgobject
> * the need to unref the iterator is onerous for C programmers using iterators
> * the need to subclass a GObject is onerous for C programmers creating iterators
> * as Owen mentioned long ago when this was already discussed, we'd end
> up duplicating bunches of APIs just to make them use iterators instead
> of whatever they use now - this is both bloat and confusing. It would
> not be realistic to ever deprecate walking GList etc. - the iterator
> replacement is much less convenient, and there's way, way, way too
> much code using GList already. You can't deprecate something that
> touches this much code.

As philip's proposal centres around easier bindings this would only
affect public API dealing with licts/collections so all of the above
will probably have negligible impact.

GList would still be around for internal usage or where performance
matters. The onus would still be on devs to make their api more binding
friendly by using iterators so I feel its important for them to have
that choice

jamie

_______________________________________________
gtk-devel-list mailing list
gtk-devel-list@...
http://mail.gnome.org/mailman/listinfo/gtk-devel-list

[Havoc Pennington-2]

Hi,

On Thu, Jul 17, 2008 at 2:06 PM, Philip Van Hoof <spam@...> wrote:
> You could make a GLib.Iterator that uses gobject-introspection, but I
> don't think you want to make gobject-introspection the one thing
> everybody who wants to expose collections in his API has to use and
> learn.

I didn't mean that - I meant if you exposed collections you'd use
GList or whatever, and g-i would know that it was a "GList of Foo",
and g-i would generate an iterator around the list based on that. Then
users of the g-i API for language bindings would see only the
iterator.

>> Any language binding not using g-i has nothing to stand on if they
>> whine about manual work - they need to a) port to / help with g-i and
>> then b) we'll talk.
>
> Disagree with this. GObject-introspection can be an aid for the many
> quirks and Cisms our platform introduces. Future APIs should not focus
> on just the C application developers.

g-i is not for C afaik. It's supposed to replace all the
source-code-scanners and "defs" files and so forth different language
bindings are using. It should be extended until it completely replaces
those things.

g-i allows dropping static stubs - all functions are invoked *through*
g-i, and return values are marshaled with it also.

> It'll also be useful for other tasks that right now require scripts to
> parse C code, as gobject-introspection will ship with a bunch of very
> interesting tools for that (representing an existing API as an XML that
> will be suitable to throw at a XSL compiler).

Ideally all scanning (and merging in lookaside info) will happen in
g-i, which will then have everything a binding needs in the binary
typelib.

> If GLib library authors wont focus on higher programming languages, then
> I fear Gtk+ and its accompanying large set of excellent libraries will
> get more and more neglected. But that's just my personal feeling.

We don't need to break C to support other languages. Most libs will
keep being written in C, and lots of existing useful code is in C, so
we should keep having good C support.

I think the ideal situation for the desktop looks a lot like Firefox
or Emacs or World of Warcraft for that matter, with a C/C++ core and
the top layers (including most visible functionality) in an agile
language. Well OK, if starting 100% from scratch, it might be more
ideal to write everything to a single virtual machine - but we aren't
starting from scratch, we're starting with millions of lines of
existing useful C code.

> I don't know why only C/GLib experts should be the ones writing language
> bindings (which is the case right now). It's quite hard to find people
> who are into the higher language and yet know really a lot about GLib
> too. Usually these people have no reason whatsoever to care about GLib
> and its isms anymore.

g-i should be able to include any code that is generic across all bindings.

>> * as Owen mentioned long ago when this was already discussed, we'd end
>> up duplicating bunches of APIs just to make them use iterators instead
>> of whatever they use now - this is both bloat and confusing. It would
>> not be realistic to ever deprecate walking GList etc. - the iterator
>> replacement is much less convenient, and there's way, way, way too
>> much code using GList already. You can't deprecate something that
>> touches this much code.
>
> I didn't mention deprecating this code ...

The point is that if we don't deprecate GList, as I think we can't,
then we end up with duplication and bloat. (Two versions of all
existing APIs, at minimum.)

Havoc
_______________________________________________
gtk-devel-list mailing list
gtk-devel-list@...
http://mail.gnome.org/mailman/listinfo/gtk-devel-list

[Philip Van Hoof]

On Thu, 2008-07-17 at 14:22 -0400, Jamie McCracken wrote:
> On Thu, 2008-07-17 at 13:37 -0400, Havoc Pennington wrote:

> As philip's proposal centres around easier bindings this would only
> affect public API dealing with licts/collections so all of the above
> will probably have negligible impact.

Exactly (I agree).

> GList would still be around for internal usage or where performance
> matters. The onus would still be on devs to make their api more binding
> friendly by using iterators so I feel its important for them to have
> that choice

GList for performance?! :-)

Not for the things the vast majority of GLib based library developers
are using it for! And if GList must be used in a high performance
situation, the kernel's list.h is better designed anyway: the item's
data is in the same allocation as the next/prev ptrs, instead of having
an allocation for the data and another allocation for the list node.

Looking at how GList is used all over our project's code nearly always
makes me conclude that most people who used a GList probably wanted a
GPtrArray instead. Its realloc overhead is usually not expensive at all.

There are specific uses for doubly linked lists, of course. For UI rich
applications (where you usually want things like sorting and index based
accessing your collections) I don't think GList is very useful.

For index-based access and sorting it's even a performance 'mistake'.

The problem is, I think, that GList's API is more easy than GPTrArray's.

Which validates my POV that:

  - The simple, most common use cases, must be easy
  - The complex things must (just) be possible

If a complex thing becomes even more complex caused by wanting to make a
common use case even a little bit more simple, then don't even think
about it and DO make the complex thing even more complex .. indeed.

Just make a wiki page on Live explaining how to do the complex thing.
The developers who feel brave (and have some competence) will figure it
out anyway.

But don't bother app developers, who spend 90% of their time codifying
common use cases, with complexities.


--
Philip Van Hoof, freelance software developer
home: me at pvanhoof dot be
gnome: pvanhoof at gnome dot org
http://pvanhoof.be/blog
http://codeminded.be




_______________________________________________
gtk-devel-list mailing list
gtk-devel-list@...
http://mail.gnome.org/mailman/listinfo/gtk-devel-list

[Yu Feng]

On Thu, 2008-07-17 at 14:23 -0400, Havoc Pennington wrote:

> Hi,
>
> On Thu, Jul 17, 2008 at 2:06 PM, Philip Van Hoof <spam@...> wrote:
> > You could make a GLib.Iterator that uses gobject-introspection, but I
> > don't think you want to make gobject-introspection the one thing
> > everybody who wants to expose collections in his API has to use and
> > learn.
>
> I didn't mean that - I meant if you exposed collections you'd use
> GList or whatever, and g-i would know that it was a "GList of Foo",
> and g-i would generate an iterator around the list based on that. Then
> users of the g-i API for language bindings would see only the
> iterator.
>
> >> Any language binding not using g-i has nothing to stand on if they
> >> whine about manual work - they need to a) port to / help with g-i and
> >> then b) we'll talk.
> >
> > Disagree with this. GObject-introspection can be an aid for the many
> > quirks and Cisms our platform introduces. Future APIs should not focus
> > on just the C application developers.
>
> g-i is not for C afaik. It's supposed to replace all the
> source-code-scanners and "defs" files and so forth different language
> bindings are using. It should be extended until it completely replaces
> those things.
>
> g-i allows dropping static stubs - all functions are invoked *through*
> g-i, and return values are marshaled with it also.
>
> > It'll also be useful for other tasks that right now require scripts to
> > parse C code, as gobject-introspection will ship with a bunch of very
> > interesting tools for that (representing an existing API as an XML that
> > will be suitable to throw at a XSL compiler).
>
> Ideally all scanning (and merging in lookaside info) will happen in
> g-i, which will then have everything a binding needs in the binary
> typelib.
>
> > If GLib library authors wont focus on higher programming languages, then
> > I fear Gtk+ and its accompanying large set of excellent libraries will
> > get more and more neglected. But that's just my personal feeling.
>
> We don't need to break C to support other languages. Most libs will
> keep being written in C, and lots of existing useful code is in C, so
> we should keep having good C support.
>
> I think the ideal situation for the desktop looks a lot like Firefox
> or Emacs or World of Warcraft for that matter, with a C/C++ core and
> the top layers (including most visible functionality) in an agile
> language. Well OK, if starting 100% from scratch, it might be more
> ideal to write everything to a single virtual machine - but we aren't
> starting from scratch, we're starting with millions of lines of
> existing useful C code.
>
> > I don't know why only C/GLib experts should be the ones writing language
> > bindings (which is the case right now). It's quite hard to find people
> > who are into the higher language and yet know really a lot about GLib
> > too. Usually these people have no reason whatsoever to care about GLib
> > and its isms anymore.
>
> g-i should be able to include any code that is generic across all bindings.
>
> >> * as Owen mentioned long ago when this was already discussed, we'd end
> >> up duplicating bunches of APIs just to make them use iterators instead
> >> of whatever they use now - this is both bloat and confusing. It would
> >> not be realistic to ever deprecate walking GList etc. - the iterator
> >> replacement is much less convenient, and there's way, way, way too
> >> much code using GList already. You can't deprecate something that
> >> touches this much code.
> >
> > I didn't mention deprecating this code ...
>
> The point is that if we don't deprecate GList, as I think we can't,
> then we end up with duplication and bloat. (Two versions of all
> existing APIs, at minimum.)

What about keep using GList in the inside; and always use collection API
when the functions are to be invoked by bindings?

GList's api is really convenient and light for managing weak references.
GHashTable has some ability to manage strong reference but it surely
know little about the GType of the object it holds.

anyway I think GList and GHashTable is enough in doing their job; but
they are a nightmare when binding to other languages. I really hate
those special code in vala in keeping track of the strong references.

BTW: I think it is a good chance to discuss this since people are also
deciding the GTK 3.0 APIs now.

Yu

>
> Havoc
> _______________________________________________
> gtk-devel-list mailing list
> gtk-devel-list@...
> http://mail.gnome.org/mailman/listinfo/gtk-devel-list

_______________________________________________
gtk-devel-list mailing list
gtk-devel-list@...
http://mail.gnome.org/mailman/listinfo/gtk-devel-list

[Mikkel Kamstrup Erlandsen-2]

2008/7/17 Havoc Pennington <hp@...>:

> Hi,
>
> Here are some alternate ideas, just brainstorming:
>
> 1) have an iterator concept in gobject-introspection and map from
> GList etc. in g-i. So g-i would allow you to invoke a method that
> returns a list, and get an iterator back.
>
> If I were doing this in gobject-introspection I'd tend to make the
> base API use a stack-allocated lightweight iterator similar to
> GtkTreeIter/GtkTextIter, and then bindings that wanted to could write
> generic code to wrap that kind of lightweight iterator in a GObject.
>
> Any language binding not using g-i has nothing to stand on if they
> whine about manual work - they need to a) port to / help with g-i and
> then b) we'll talk.
>
> It would be possible to generically auto-create a GObject-based
> iterator like yours, using this g-i feature.

I am not sure I fully understand your proposal here. It would cater
most for the bindings writers and not the C application developers,
right?

Also, if one where to use such a construct in C we would need a new
external tool glib-gen-iterators. Please say it ain't so ;-)

> 2) Another idea would be an equivalent to registering boxed types:
>
> g_iterator_type_register_static(const char *name, GBoxedCopyFunc
> boxed_copy, GBoxedFreeFunc boxed_free, GIteratorNextFunc
> iterator_next, GIteratorGetFunc iterator_get);
>
> This would allow language bindings to generically manipulate custom
> iterator types like TextIter and TreeIter, without making things a
> pain in C and while keeping things lightweight. And without
> redoing/breaking TreeModelIter and TextIter and all the other existing
> examples you mention.

This idea is probably the one I like the most (also over Philip's
original proposal), it has some problems though. The GBoxed
implementation does a lookup based on the GType supplied to
g_boxed_{copy,free}() to get the vtable for the implementation. This
could mean that g_iter_next(GType type, gpointer iter) would have some
overhead which might not be desirable for something we want to be as
snappy as possible.

This might be alleviated a little (or worsened) if we cache the vtable
of the last call to g_iter_next() in a local static variable. Not
sure.

The same problem applies if we want to model a GCollection this way.

> <snip>
> * as Owen mentioned long ago when this was already discussed, we'd end
> up duplicating bunches of APIs just to make them use iterators instead
> of whatever they use now - this is both bloat and confusing. It would
> not be realistic to ever deprecate walking GList etc. - the iterator
> replacement is much less convenient, and there's way, way, way too
> much code using GList already. You can't deprecate something that
> touches this much code.

Does this mean that you/he believes that a collection/iterator api in
glib would be for bindings only, and that people would be better off
using the good olde constructs when doing C code? Or am I reading it
wrong?

Cheers,
Mikkel
_______________________________________________
gtk-devel-list mailing list
gtk-devel-list@...
http://mail.gnome.org/mailman/listinfo/gtk-devel-list

[Mikkel Kamstrup Erlandsen-2]

2008/7/17 Mikkel Kamstrup Erlandsen <mikkel.kamstrup@...>:

> 2008/7/17 Havoc Pennington <hp@...>:
>> Hi,
>>
>> Here are some alternate ideas, just brainstorming:
>>
>> 1) have an iterator concept in gobject-introspection and map from
>> GList etc. in g-i. So g-i would allow you to invoke a method that
>> returns a list, and get an iterator back.
>>
>> If I were doing this in gobject-introspection I'd tend to make the
>> base API use a stack-allocated lightweight iterator similar to
>> GtkTreeIter/GtkTextIter, and then bindings that wanted to could write
>> generic code to wrap that kind of lightweight iterator in a GObject.
>>
>> Any language binding not using g-i has nothing to stand on if they
>> whine about manual work - they need to a) port to / help with g-i and
>> then b) we'll talk.
>>
>> It would be possible to generically auto-create a GObject-based
>> iterator like yours, using this g-i feature.
>
> I am not sure I fully understand your proposal here. It would cater
> most for the bindings writers and not the C application developers,
> right?
>
> Also, if one where to use such a construct in C we would need a new
> external tool glib-gen-iterators. Please say it ain't so ;-)
>
>> 2) Another idea would be an equivalent to registering boxed types:
>>
>> g_iterator_type_register_static(const char *name, GBoxedCopyFunc
>> boxed_copy, GBoxedFreeFunc boxed_free, GIteratorNextFunc
>> iterator_next, GIteratorGetFunc iterator_get);
>>
>> This would allow language bindings to generically manipulate custom
>> iterator types like TextIter and TreeIter, without making things a
>> pain in C and while keeping things lightweight. And without
>> redoing/breaking TreeModelIter and TextIter and all the other existing
>> examples you mention.
>
> This idea is probably the one I like the most (also over Philip's
> original proposal), it has some problems though. The GBoxed
> implementation does a lookup based on the GType supplied to
> g_boxed_{copy,free}() to get the vtable for the implementation. This
> could mean that g_iter_next(GType type, gpointer iter) would have some
> overhead which might not be desirable for something we want to be as
> snappy as possible.
>
> This might be alleviated a little (or worsened) if we cache the vtable
> of the last call to g_iter_next() in a local static variable. Not
> sure.
>
> The same problem applies if we want to model a GCollection this way.

And here is a full proposal using the GBoxed technique (no, not the
drunken uncle technique, sorry):
http://live.gnome.org/MikkelKamstrup/GCollection

If one took of in gobject/gboxed.c it should not be that hard to do.
The work will probably lie in registering the existing collections as
GIterable<!-- -->s.

Cheers,
Mikkel
_______________________________________________
gtk-devel-list mailing list
gtk-devel-list@...
http://mail.gnome.org/mailman/listinfo/gtk-devel-list

[Felipe Contreras]

On Thu, Jul 17, 2008 at 9:06 PM, Philip Van Hoof <spam@...> wrote:
> On Thu, 2008-07-17 at 13:37 -0400, Havoc Pennington wrote:
>> Why explore alternate ideas? Some downsides to GIterator-as-gobject:
>>
>> * GObject is pretty heavyweight for something like this, and moreover
>> right now libglib doesn't depend on libgobject
>
> Agree with this. Perhaps a more lightweight GType could be used for the
> iterator type. Isn't GStreamer for example providing lightweight GObject
> like GTypes?

Indeed:
http://www.koders.com/c/fid658F8E1DB8DB0E76D6B2E27AD4370F1BE87CAC46.aspx?s=gst_mini_object_new#L146

--
Felipe Contreras
_______________________________________________
gtk-devel-list mailing list
gtk-devel-list@...
http://mail.gnome.org/mailman/listinfo/gtk-devel-list

[Mathias Hasselmann-2]

Am Donnerstag, den 17.07.2008, 14:23 -0400 schrieb Havoc Pennington:

> Hi,
>
> On Thu, Jul 17, 2008 at 2:06 PM, Philip Van Hoof <spam@...> wrote:
> > You could make a GLib.Iterator that uses gobject-introspection, but I
> > don't think you want to make gobject-introspection the one thing
> > everybody who wants to expose collections in his API has to use and
> > learn.
>
> I didn't mean that - I meant if you exposed collections you'd use
> GList or whatever, and g-i would know that it was a "GList of Foo",
> and g-i would generate an iterator around the list based on that. Then
> users of the g-i API for language bindings would see only the
> iterator.

Well, I am doing some refactoring right now and during this process it
would be quite helpful if we'd have something better than GList: It
would be really helpful if our GLists would express its element type.

Maybe we can figure out how to do this in a sane way for GTK3?

Ciao,
Mathias
--
Mathias Hasselmann <mathias.hasselmann@...>
Openismus GmbH: http://www.openismus.com/
Personal Site: http://taschenorakel.de/


_______________________________________________
gtk-devel-list mailing list
gtk-devel-list@...
http://mail.gnome.org/mailman/listinfo/gtk-devel-list