Code completion

[Johannes Schmid-4]

Hi!

After the recent improvements in the symbol database I thought about a
way to resolve autocomplete issues like "object->member" or even more
complicated "func()->object->func2()->method27()".

I investigated the code that was in place for scintilla in the 2.2
series. It searches for the declaration of "object" in the current block
to find its type. The approach is rather simple and I guess it won't
work in all cases.

Care needs also to be taken to know which methods are available in the
current context. When you are inside a class member method for example,
you are able to access all methods of the class and its parents as well
as all methods in the current namespace. But you won't be able to access
any method from another class without an object.

I don't know how far the symbol-database can give us information about
the current context but I hope it can give us some information. Even
more important, we need a parser to give us the object we search (could
be returned by a method) and the type of this object.

Any ideas?

Thanks,
Johannes


-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
Don't miss this year's exciting event. There's still time to save $100.
Use priority code J8TL2D2.
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
_______________________________________________
Anjuta-devel mailing list
Anjuta-devel@...
https://lists.sourceforge.net/lists/listinfo/anjuta-devel

[Naba Kumar-2]

Hi Johannes,

On Thu, 2008-05-01 at 01:48 +0200, Johannes Schmid wrote:

> After the recent improvements in the symbol database I thought about a
> way to resolve autocomplete issues like "object->member" or even more
> complicated "func()->object->func2()->method27()".
>
There are two parts in solving this.

1) One is to retrieve the local variable information from current
context. This is the hard part and is still unclear what's the best way
to do it. I have something in mind that involves scanning from each
start-of-block, beginning from closest block and moving further to
higher blocks, until the variable in question is found. This is not a
perfect solution since it doesn't take care of shadow variables within
the same block, but is quite suitable in terms of our iteration in
language-support-cpp-java plugin.

2) The second is to actually autocomplete the current 'expression'.
There are two kinds of expressions (based on how anjuta would
autocomplete):
   a) Expr. that starts with variable. It requires solving part (1).
   b) Expr. that starts with class/type

Part (2) is solved with a c-expression parser generated with yacc/bison.
I have started experimenting with the lexer already and have something
with me to play around. However, to proceed further, it requires support
from IAnjutaSymbolBrowser interface to provide proper children iteration
with full type information. Current API only provides 'type' of the
symbol, not 'type name' (i.e. 'class' vs 'AnjutaClass').

Currently, I am waiting for Massimo to get to the point where we can
start working on improved IAnjutaSymbolBrowser interface. If you think
we can already do that, which I believe is the case since symbol-db
seems to work quite well so far, we can start discussion about it.

Thanks.

Regards,
-Naba



-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
Don't miss this year's exciting event. There's still time to save $100.
Use priority code J8TL2D2.
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
_______________________________________________
Anjuta-devel mailing list
Anjuta-devel@...
https://lists.sourceforge.net/lists/listinfo/anjuta-devel

[Massimo Cora']

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Naba, Johannes,

Naba Kumar wrote:

> Hi Johannes,
>
> On Thu, 2008-05-01 at 01:48 +0200, Johannes Schmid wrote:
>
>> After the recent improvements in the symbol database I thought about a
>> way to resolve autocomplete issues like "object->member" or even more
>> complicated "func()->object->func2()->method27()".
>>
> There are two parts in solving this.
>
> 1) One is to retrieve the local variable information from current
> context. This is the hard part and is still unclear what's the best way
> to do it. I have something in mind that involves scanning from each
> start-of-block, beginning from closest block and moving further to
> higher blocks, until the variable in question is found. This is not a
> perfect solution since it doesn't take care of shadow variables within
> the same block, but is quite suitable in terms of our iteration in
> language-support-cpp-java plugin.

I think we already have a 'quite good' solution on this: I implemented
for symbol-browser [see
http://svn.gnome.org/viewvc/anjuta/trunk/plugins/symbol-browser/an_symbol_view.c?view=markup]
a function that, for a given expression "myvar->getX().toFloat", finds
first the ident 'myvar' [see sv_scan_for_ident ()], then go backwards in
the code looking for its declaration
[sv_extract_type_qualifier_from_expr ()].
Once we have the declaration of a given 'MyClass myvar' we should search
db for some infos on 'MyClass'.

>
> 2) The second is to actually autocomplete the current 'expression'.
> There are two kinds of expressions (based on how anjuta would
> autocomplete):
>    a) Expr. that starts with variable. It requires solving part (1).
>    b) Expr. that starts with class/type
>
> Part (2) is solved with a c-expression parser generated with yacc/bison.
> I have started experimenting with the lexer already and have something
> with me to play around. However, to proceed further, it requires support
> from IAnjutaSymbolBrowser interface to provide proper children iteration
> with full type information. Current API only provides 'type' of the
> symbol, not 'type name' (i.e. 'class' vs 'AnjutaClass').

well, about the type_name you can query a symbol for it's type_name's
value, see isymbol_get_extra_info_string () passing
IANJUTA_SYMBOL_FIELD_TYPE_NAME as IAnjutaSymbolField.
For the children iteration I wrote some functions that may help when
searching for symbols properties. See the 'iteratable queries' on
symbol-db-engine.h. Of course I can add as many queries as you need.
Those were just the ones I thought would have been useful.

let me know how you would implement the c-expression parser as I'm
interested on that.
Btw: a working example seems to be http://codelite.org, where it uses a
grammar parser to do the work. Sources are anyway something cryptic
because the lack of comments.


regards,
Massimo


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (GNU/Linux)

iD8DBQFIG0M+cfVTgMRILk0RAj26AJ9bkcRI4xMsI17jKc2s0asqQU9UWQCaA71A
6/cUD03RjQ452xrlY0A4z2Y=
=35IR
-----END PGP SIGNATURE-----

-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
Don't miss this year's exciting event. There's still time to save $100.
Use priority code J8TL2D2.
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
_______________________________________________
Anjuta-devel mailing list
Anjuta-devel@...
https://lists.sourceforge.net/lists/listinfo/anjuta-devel

[Naba Kumar-2]

Hi Massimo,

On Fri, 2008-05-02 at 18:37 +0200, Massimo Cora' wrote:

> I think we already have a 'quite good' solution on this: I implemented
> for symbol-browser ...

Okay, I will see that.

> well, about the type_name you can query a symbol for it's type_name's
> value, see isymbol_get_extra_info_string () passing
> IANJUTA_SYMBOL_FIELD_TYPE_NAME as IAnjutaSymbolField.

Great. I didn't notice that.

> For the children iteration I wrote some functions that may help when
> searching for symbols properties. See the 'iteratable queries' on
> symbol-db-engine.h. Of course I can add as many queries as you need.
> Those were just the ones I thought would have been useful.
>
Okay, I will let you know if I need anything.

Thanks.

Regards,
-Naba



-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
Don't miss this year's exciting event. There's still time to save $100.
Use priority code J8TL2D2.
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
_______________________________________________
Anjuta-devel mailing list
Anjuta-devel@...
https://lists.sourceforge.net/lists/listinfo/anjuta-devel

[Johannes Schmid-4]

Hi all!

> I think we already have a 'quite good' solution on this: I implemented
> for symbol-browser [see
> http://svn.gnome.org/viewvc/anjuta/trunk/plugins/symbol-browser/an_symbol_view.c?view=markup]
> a function that, for a given expression "myvar->getX().toFloat", finds
> first the ident 'myvar' [see sv_scan_for_ident ()], then go backwards in
> the code looking for its declaration
> [sv_extract_type_qualifier_from_expr ()].
> Once we have the declaration of a given 'MyClass myvar' we should search
> db for some infos on 'MyClass'.

OK, I will have a look at this a properly port this to
language-support-cpp-java (where it belongs). I remember that we once
disabled this code because of some crashes but that can be solved.

> well, about the type_name you can query a symbol for it's type_name's
> value, see isymbol_get_extra_info_string () passing
> IANJUTA_SYMBOL_FIELD_TYPE_NAME as IAnjutaSymbolField.
> For the children iteration I wrote some functions that may help when
> searching for symbols properties. See the 'iteratable queries' on
> symbol-db-engine.h. Of course I can add as many queries as you need.
> Those were just the ones I thought would have been useful.
>

Hmm, that sounds good. I guess we can yet come quite far without the
expression parser but once that one is in place we might be able to
nearly autocomplete everything.

Regards,
Johannes


-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
Don't miss this year's exciting event. There's still time to save $100.
Use priority code J8TL2D2.
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
_______________________________________________
Anjuta-devel mailing list
Anjuta-devel@...
https://lists.sourceforge.net/lists/listinfo/anjuta-devel

[Massimo Cora']

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


Hi Eran,


Eran Ifrah wrote:
>
> I am the author of this sources, which sources do you refer to? the
> grammar (.y, .l) or the code that makes use of them?
> If you need an example / help / anything drop me a word, and I will be
> more than happy to assist.
>

thanks. I had a look at the documentation on
http://codelite.org/docs/CodeLite_Overview.html. I would like to know
where you implemented the "CodeLite processes the text and expression"
step, using the yacc based parser.
I also had a quick look at the sources, but couldn't find the point
where that takes place.

btw : I couldn't even compile codelite because of an error:

LiteEditor/frame.cpp: In member function 'void
Frame::CreateRecentlyOpenedFilesMenu()':
LiteEditor/frame.cpp:2086: error: 'class FileHistory' has no member
named 'SetBaseId'
LiteEditor/frame.cpp: In member function 'void
Frame::CreateRecentlyOpenedWorkspacesMenu()':
LiteEditor/frame.cpp:2114: error: 'class FileHistory' has no member
named 'SetBaseId'
make: *** [Release_gcc_unicode/frame.o] Error 1


> In general codelite took the same approach of where anjute is heading:
> create a look-up table by ctags (actually, i am using a modified ctags,
> since I had to patch it for better template support, namespace and
> other). But the main parsing is done using the grammar parsers (4
> exists, each does different parsing task)
>
> Ofc, the grammar will look a bit scary unless you are familiar with yacc
> & flex.
>

well, yes, I never played with yacc && flex, but at the same time I
would like to know how they works. I think this can be a great occasion.

thanks and regards,
Massimo



-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (GNU/Linux)

iD4DBQFIHb3BcfVTgMRILk0RAlIfAJ9Ha1qmc+aZ1iTZdjG00KP6N/oWlgCYsbPA
Ctu1SGMf5SYdlIaMPKUGnw==
=Gckh
-----END PGP SIGNATURE-----

-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
Don't miss this year's exciting event. There's still time to save $100.
Use priority code J8TL2D2.
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
_______________________________________________
Anjuta-devel mailing list
Anjuta-devel@...
https://lists.sourceforge.net/lists/listinfo/anjuta-devel

[Johannes Schmid-4]

Hi all!

> > In general codelite took the same approach of where anjute is heading:
> > create a look-up table by ctags (actually, i am using a modified ctags,
> > since I had to patch it for better template support, namespace and
> > other). But the main parsing is done using the grammar parsers (4
> > exists, each does different parsing task)
> >

OK, had a look at this parsing code - it's look very good and I think we
should take it as it is. Any problems with integrating C++ code in
language-support-cpp-java? Because rewriting it in C will be painfull
and it does only use STL C++.

The CppParser would end up in
plugins/language-support-cpp-java/cxxparser.

@Eran: Have you tried to merge your ctags changes upstream? We had (and
still have but obsolete) our own copy of ctags in the sources but
decided to use the system version with our new database system. Anyway,
it would be good to have a nice upstream version with proper template
and namespace support. Thanks!

Regards,
Johannes


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Anjuta-devel mailing list
Anjuta-devel@...
https://lists.sourceforge.net/lists/listinfo/anjuta-devel

[Massimo Cora']

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Johannes,

Johannes Schmid wrote:
> OK, had a look at this parsing code - it's look very good and I think we
> should take it as it is. Any problems with integrating C++ code in
> language-support-cpp-java? Because rewriting it in C will be painfull
> and it does only use STL C++.
>


are you talking about the cpp_func_parser.cpp &c files? I suppose
they're auto-generated with yacc&&lex parsers from the .y and .l files.
They can be outputted in C++ but also in C. So I think it would be
better to have them in c, to avoid scintilla-like problems with C++
maintaining. I think also that having a look at the implementation ideas
of CodeLite is good, but probably it's missing something like
inheritance between classes. Data retrieving from ctags database is also
different from our, so there would be changes there too.

btw: see at gdb-6.8 code. It has .y grammars, which generate C code.
It's in that way that variables scope/navigation is possible while
debugging. More: I see
gdb-6.8/gdb$ ls *.y
ada-exp.y  c-exp.y  cp-name-parser.y  f-exp.y  jv-exp.y  m2-exp.y
objc-exp.y  p-exp.y

which are grammars for different languages, so implementing a 'general'
interface for completion would produce completion for more languages at
the same time; of course it'll require more language-support* plugins,
but it would be much like copy&paste.

regards,
Massimo



-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (GNU/Linux)

iD8DBQFIMy0hcfVTgMRILk0RAnnXAJwNv+KLybC2o1kZAbH844x7DDOFewCfb6HK
ArPAnnGhmvSOytARlaD5rLc=
=9gzT
-----END PGP SIGNATURE-----

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Anjuta-devel mailing list
Anjuta-devel@...
https://lists.sourceforge.net/lists/listinfo/anjuta-devel

[Johannes Schmid-4]

Hi Massimo!

> are you talking about the cpp_func_parser.cpp &c files? I suppose
> they're auto-generated with yacc&&lex parsers from the .y and .l files.
> They can be outputted in C++ but also in C. So I think it would be
> better to have them in c, to avoid scintilla-like problems with C++
> maintaining. I think also that having a look at the implementation ideas
> of CodeLite is good, but probably it's missing something like
> inheritance between classes. Data retrieving from ctags database is also
> different from our, so there would be changes there too.

You are right that those are generated by yacc/flex but the .y files
contain embedded C++ code. Anyway, I shouldn't be too much work to port
this code to C (it's not much).

>
> btw: see at gdb-6.8 code. It has .y grammars, which generate C code.
> It's in that way that variables scope/navigation is possible while
> debugging. More: I see
> gdb-6.8/gdb$ ls *.y
> ada-exp.y  c-exp.y  cp-name-parser.y  f-exp.y  jv-exp.y  m2-exp.y
> objc-exp.y  p-exp.y
>
> which are grammars for different languages, so implementing a 'general'
> interface for completion would produce completion for more languages at
> the same time; of course it'll require more language-support* plugins,
> but it would be much like copy&paste.

I took a look at the gdb parsers but those provide much more flexibility
that we don't need. For example they allow to use operators (+,-, etc.)
in statements and allow to call methods with certain arguments. They are
really good for the needs of gdb but for us, I think they are too
complicated. I will use those from codelite instead.

Regards,
Johannes


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Anjuta-devel mailing list
Anjuta-devel@...
https://lists.sourceforge.net/lists/listinfo/anjuta-devel