Re: Information-theoretic functions (David Bateman)

[Muthiah Annamalai-3]

> Date: Mon, 20 Nov 2006 14:07:56 +0100
> From: David Bateman <David.Bateman@...>
> Subject: Re: Information-theoretic functions
> To: Joseph Wakeling <joseph.wakeling@...>
> Cc: octave-forge list <octave-dev@...>,
> octave-sources@...
> Message-ID: <4561A8AC.3090104@...>
> Content-Type: text/plain; charset=UTF-8
>
> Joseph Wakeling wrote:
> > Hello all,
> >
> > For my own private work with Octave I have prepared a set of
> > information-theoretic functions which I thought I would offer to the
> > community.
> >
> > >From browsing the archives I recognise that another user recently
> > contributed a similar set of functions, but the .tar.gz attachment was
> > not available, so I could not compare.  I would very much like to see
> > those if possible.
> >
> > My own functions include one which I think was not in the earlier
> > bundle, to calculate the information gain ratio or uncertainty coefficient.
> >
> > One problem, which I'm not sure how to get round, is in the main
> > information entropy function: it requires vectors at input, but at
> > present only works with row or column vectors, i.e. not with vectors
> > where the active dimension is > 2.
> >  
> You should check with Muthu Annamalai for the overlap with his info
> theory functions in octave-forge, if any, and get these committed to
> octave-forge. Perhaps Muthu can do the commit for you..
>
> D.
>

I think you can look at the CVS online via source-forge's
website. The link to the information theory toolbox is at:
http://octave.cvs.sourceforge.net/octave/octave-forge/main/info-theory/inst/

I think of the functions you have presented, infogain.m is something
that is not in the toolbox. I suggest to you to please look through the
source files already in info-theory toolbox, and see if you can rework
the infogain.m.

Also I would suggest to folks who like to see the info-theory toolbox
grow, to try and rebuild a set of functions starting from the ones in
Dr Kieffer's page http://www.ee.umn.edu/users/kieffer/programs.html . It
is quite disappointing that the code is not available anymore. If we
work toward this objective it would be really nice to have a 'complete'
info-theory toolbox.

All, any suggestions welcome.
Regards,
Muthu Annamalai






_______________________________________________
Octave-sources mailing list
Octave-sources@...
https://www.cae.wisc.edu/mailman/listinfo/octave-sources

[Joseph Wakeling]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Muthiah Annamalai wrote:
> I think you can look at the CVS online via source-forge's
> website. The link to the information theory toolbox is at:
> http://octave.cvs.sourceforge.net/octave/octave-forge/main/info-theory/inst/
>
> I think of the functions you have presented, infogain.m is something
> that is not in the toolbox. I suggest to you to please look through the
> source files already in info-theory toolbox, and see if you can rework
> the infogain.m.

Thanks for the link.  It looks like in fact our functions are
complementary: for example your entropy.m function calculates
information entropy from the set of probabilities, whereas mine takes a
sequence (or two sequences) and calculates, first the probabilities,
then the resulting entropy (or joint entropy).

So, why don't I rewrite my functions to interact with yours? For
example, I can write entropy_seq(x) which takes a sequence, calculates
the probabilities, and then calls your function entropy() to get the answer.

Best wishes,

    -- Joe


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.3 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFFYfIzcjylL0sfzuERAnQXAJ4l0RvVGYRbsHI1jqwqCFdLtPXn4QCfRcO1
iqX4KfRVAGuvfQOC1ZEktTA=
=jJq+
-----END PGP SIGNATURE-----
_______________________________________________
Octave-sources mailing list
Octave-sources@...
https://www.cae.wisc.edu/mailman/listinfo/octave-sources

[Muthiah Annamalai-3]

On Mon, 2006-11-20 at 19:21 +0100, Joseph Wakeling wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Muthiah Annamalai wrote:
> > I think you can look at the CVS online via source-forge's
> > website. The link to the information theory toolbox is at:
> > http://octave.cvs.sourceforge.net/octave/octave-forge/main/info-theory/inst/
> >
> > I think of the functions you have presented, infogain.m is something
> > that is not in the toolbox. I suggest to you to please look through the
> > source files already in info-theory toolbox, and see if you can rework
> > the infogain.m.
>
> Thanks for the link.  It looks like in fact our functions are
> complementary: for example your entropy.m function calculates
> information entropy from the set of probabilities, whereas mine takes a
> sequence (or two sequences) and calculates, first the probabilities,
> then the resulting entropy (or joint entropy).
>
> So, why don't I rewrite my functions to interact with yours? For
> example, I can write entropy_seq(x) which takes a sequence, calculates
> the probabilities, and then calls your function entropy() to get the answer.
>
> Best wishes,
>
>     -- Joe
>
>

Sounds good!
Thanks
Muthu


> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.3 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
>
> iD8DBQFFYfIzcjylL0sfzuERAnQXAJ4l0RvVGYRbsHI1jqwqCFdLtPXn4QCfRcO1
> iqX4KfRVAGuvfQOC1ZEktTA=
> =jJq+
> -----END PGP SIGNATURE-----


_______________________________________________
Octave-sources mailing list
Octave-sources@...
https://www.cae.wisc.edu/mailman/listinfo/octave-sources