wrong metadatas

[nicolas de loof-3]

Hy,

I get strange behaviour wen using repository group :

From a fresh maven installation (empty local repo), with settings set to
mirror central to my archiva repository group URL, the eclipse 2.5 plugin
can't find the required org.eclipse.core:resources:[3.1.0,4.0.0)

The maven-metadata.xml returned by the group URL is emty. requesting
metadatas from individual managed repos in the group are fine. Is this a
known limitation ?

Nico.

[Maria Odea Ching-5]

Hi Nico,

That looks like a new bug :( The metadata should be just the same as the
regular metadata of a managed repo (not belonging to a group) as it uses the
same code for webdav & proxying. Could you file a jira for this?

Thanks,
Deng

On Wed, Jul 9, 2008 at 8:45 PM, nicolas de loof <nicolas@...> wrote:

> Hy,
>
> I get strange behaviour wen using repository group :
>
> From a fresh maven installation (empty local repo), with settings set to
> mirror central to my archiva repository group URL, the eclipse 2.5 plugin
> can't find the required org.eclipse.core:resources:[3.1.0,4.0.0)
>
> The maven-metadata.xml returned by the group URL is emty. requesting
> metadatas from individual managed repos in the group are fine. Is this a
> known limitation ?
>
> Nico.
>

[Dan Tran]

sounds like  a blocking bug? would love to see this fixed in 1.1 :-)

On Wed, Jul 9, 2008 at 7:11 PM, Maria Odea Ching <oching@...> wrote:

> Hi Nico,
>
> That looks like a new bug :( The metadata should be just the same as the
> regular metadata of a managed repo (not belonging to a group) as it uses the
> same code for webdav & proxying. Could you file a jira for this?
>
> Thanks,
> Deng
>
> On Wed, Jul 9, 2008 at 8:45 PM, nicolas de loof <nicolas@...> wrote:
>
>> Hy,
>>
>> I get strange behaviour wen using repository group :
>>
>> From a fresh maven installation (empty local repo), with settings set to
>> mirror central to my archiva repository group URL, the eclipse 2.5 plugin
>> can't find the required org.eclipse.core:resources:[3.1.0,4.0.0)
>>
>> The maven-metadata.xml returned by the group URL is emty. requesting
>> metadatas from individual managed repos in the group are fine. Is this a
>> known limitation ?
>>
>> Nico.
>>
>

[nicolas de loof-3]

MRM-872 created for this, with the concrete case I fall into.

If confirmed, this is a blocker for repository group use

2008/7/10 Dan Tran <dantran@...>:

> sounds like  a blocking bug? would love to see this fixed in 1.1 :-)
>
> On Wed, Jul 9, 2008 at 7:11 PM, Maria Odea Ching <oching@...>
> wrote:
> > Hi Nico,
> >
> > That looks like a new bug :( The metadata should be just the same as the
> > regular metadata of a managed repo (not belonging to a group) as it uses
> the
> > same code for webdav & proxying. Could you file a jira for this?
> >
> > Thanks,
> > Deng
> >
> > On Wed, Jul 9, 2008 at 8:45 PM, nicolas de loof <nicolas@...>
> wrote:
> >
> >> Hy,
> >>
> >> I get strange behaviour wen using repository group :
> >>
> >> From a fresh maven installation (empty local repo), with settings set to
> >> mirror central to my archiva repository group URL, the eclipse 2.5
> plugin
> >> can't find the required org.eclipse.core:resources:[3.1.0,4.0.0)
> >>
> >> The maven-metadata.xml returned by the group URL is emty. requesting
> >> metadatas from individual managed repos in the group are fine. Is this a
> >> known limitation ?
> >>
> >> Nico.
> >>
> >
>

[Maria Odea Ching-2]

Ok, thanks Nico, Dan :)
I'll look into this further.. I guess we could include this in 1.1.1 which
is scheduled to be released right after 1.1 to fix another blocker in 1.1.

Thanks,
Deng

On Thu, Jul 10, 2008 at 2:37 PM, nicolas de loof <nicolas@...> wrote:

> MRM-872 created for this, with the concrete case I fall into.
>
> If confirmed, this is a blocker for repository group use
>
> 2008/7/10 Dan Tran <dantran@...>:
>
> > sounds like  a blocking bug? would love to see this fixed in 1.1 :-)
> >
> > On Wed, Jul 9, 2008 at 7:11 PM, Maria Odea Ching <oching@...>
> > wrote:
> > > Hi Nico,
> > >
> > > That looks like a new bug :( The metadata should be just the same as
> the
> > > regular metadata of a managed repo (not belonging to a group) as it
> uses
> > the
> > > same code for webdav & proxying. Could you file a jira for this?
> > >
> > > Thanks,
> > > Deng
> > >
> > > On Wed, Jul 9, 2008 at 8:45 PM, nicolas de loof <nicolas@...>
> > wrote:
> > >
> > >> Hy,
> > >>
> > >> I get strange behaviour wen using repository group :
> > >>
> > >> From a fresh maven installation (empty local repo), with settings set
> to
> > >> mirror central to my archiva repository group URL, the eclipse 2.5
> > plugin
> > >> can't find the required org.eclipse.core:resources:[3.1.0,4.0.0)
> > >>
> > >> The maven-metadata.xml returned by the group URL is emty. requesting
> > >> metadatas from individual managed repos in the group are fine. Is this
> a
> > >> known limitation ?
> > >>
> > >> Nico.
> > >>
> > >
> >
>

[nicolas de loof-3]

I just started investigating on this. I can't find where the metadatas are
merged when a repository group is set. From my understanding,
ArchivaVirtualDavResource is used with a List<File> of metadatas from each
repo in the group. But I don't find WHERE the metadata files are merged into
a single XML content...



2008/7/10 Maria Odea Ching <oching@...>:

> Ok, thanks Nico, Dan :)
> I'll look into this further.. I guess we could include this in 1.1.1 which
> is scheduled to be released right after 1.1 to fix another blocker in 1.1.
>
> Thanks,
> Deng
>
> On Thu, Jul 10, 2008 at 2:37 PM, nicolas de loof <nicolas@...>
> wrote:
>
> > MRM-872 created for this, with the concrete case I fall into.
> >
> > If confirmed, this is a blocker for repository group use
> >
> > 2008/7/10 Dan Tran <dantran@...>:
> >
> > > sounds like  a blocking bug? would love to see this fixed in 1.1 :-)
> > >
> > > On Wed, Jul 9, 2008 at 7:11 PM, Maria Odea Ching <oching@...>
> > > wrote:
> > > > Hi Nico,
> > > >
> > > > That looks like a new bug :( The metadata should be just the same as
> > the
> > > > regular metadata of a managed repo (not belonging to a group) as it
> > uses
> > > the
> > > > same code for webdav & proxying. Could you file a jira for this?
> > > >
> > > > Thanks,
> > > > Deng
> > > >
> > > > On Wed, Jul 9, 2008 at 8:45 PM, nicolas de loof <nicolas@...>
> > > wrote:
> > > >
> > > >> Hy,
> > > >>
> > > >> I get strange behaviour wen using repository group :
> > > >>
> > > >> From a fresh maven installation (empty local repo), with settings
> set
> > to
> > > >> mirror central to my archiva repository group URL, the eclipse 2.5
> > > plugin
> > > >> can't find the required org.eclipse.core:resources:[3.1.0,4.0.0)
> > > >>
> > > >> The maven-metadata.xml returned by the group URL is emty. requesting
> > > >> metadatas from individual managed repos in the group are fine. Is
> this
> > a
> > > >> known limitation ?
> > > >>
> > > >> Nico.
> > > >>
> > > >
> > >
> >
>

[Maria Odea Ching-5]

The ArchivaVirtualDavResource is for the dav browse (directories), not for
artifact requests..

Hmm, the virtual repos work this way:
- client requests for an artifact
- RepoServlet handles the request and goes to ArchivaDavSessionProvider
- in ArchivaDavSessionProvider, the request is checked if it is a repo group
url or a regular repo url.
   If it is, then loop through the repos under the group. The first
requested artifact found among the repos
   is returned (proxied or already existing). The metadata update/merge
happens when the artifact is
   fetched from the proxies (but this is on a per repository level, there's
no metadata of metadata files
   from each repo being merged at the repo group level)

The same process goes if the request made is for a metadata file. I don't
know why a blank metadata file was returned.. :(

Thanks,
Deng

On Thu, Jul 10, 2008 at 3:39 PM, nicolas de loof <nicolas@...> wrote:

> I just started investigating on this. I can't find where the metadatas are
> merged when a repository group is set. From my understanding,
> ArchivaVirtualDavResource is used with a List<File> of metadatas from each
> repo in the group. But I don't find WHERE the metadata files are merged
> into
> a single XML content...
>
>
>
> 2008/7/10 Maria Odea Ching <oching@...>:
>
> > Ok, thanks Nico, Dan :)
> > I'll look into this further.. I guess we could include this in 1.1.1
> which
> > is scheduled to be released right after 1.1 to fix another blocker in
> 1.1.
> >
> > Thanks,
> > Deng
> >
> > On Thu, Jul 10, 2008 at 2:37 PM, nicolas de loof <nicolas@...>
> > wrote:
> >
> > > MRM-872 created for this, with the concrete case I fall into.
> > >
> > > If confirmed, this is a blocker for repository group use
> > >
> > > 2008/7/10 Dan Tran <dantran@...>:
> > >
> > > > sounds like  a blocking bug? would love to see this fixed in 1.1 :-)
> > > >
> > > > On Wed, Jul 9, 2008 at 7:11 PM, Maria Odea Ching <oching@...>
> > > > wrote:
> > > > > Hi Nico,
> > > > >
> > > > > That looks like a new bug :( The metadata should be just the same
> as
> > > the
> > > > > regular metadata of a managed repo (not belonging to a group) as it
> > > uses
> > > > the
> > > > > same code for webdav & proxying. Could you file a jira for this?
> > > > >
> > > > > Thanks,
> > > > > Deng
> > > > >
> > > > > On Wed, Jul 9, 2008 at 8:45 PM, nicolas de loof <
> nicolas@...>
> > > > wrote:
> > > > >
> > > > >> Hy,
> > > > >>
> > > > >> I get strange behaviour wen using repository group :
> > > > >>
> > > > >> From a fresh maven installation (empty local repo), with settings
> > set
> > > to
> > > > >> mirror central to my archiva repository group URL, the eclipse 2.5
> > > > plugin
> > > > >> can't find the required org.eclipse.core:resources:[3.1.0,4.0.0)
> > > > >>
> > > > >> The maven-metadata.xml returned by the group URL is emty.
> requesting
> > > > >> metadatas from individual managed repos in the group are fine. Is
> > this
> > > a
> > > > >> known limitation ?
> > > > >>
> > > > >> Nico.
> > > > >>
> > > > >
> > > >
> > >
> >
>

[nicolas de loof-3]

This mean the First metadata file found in on repo of the group is returned
!

This is what happen : I get a metadat file with no <version> included (not
an blank file, but an XML construct with just root element) : this is the
content of my first repo in the group, that did not retrieve the expected
artifact.

I made a simple test : place "release" repo before "corporate" in the group,
and I the get the expected metadata.

My conclusion is we have two options :

- support metadata merge from repositories in the group (not just "first
win")
- don't create metadata in managed repo if no artifact was found from
proxies.





2008/7/10 Maria Odea Ching <oching@...>:

> The ArchivaVirtualDavResource is for the dav browse (directories), not for
> artifact requests..
>
> Hmm, the virtual repos work this way:
> - client requests for an artifact
> - RepoServlet handles the request and goes to ArchivaDavSessionProvider
> - in ArchivaDavSessionProvider, the request is checked if it is a repo
> group
> url or a regular repo url.
>   If it is, then loop through the repos under the group. The first
> requested artifact found among the repos
>   is returned (proxied or already existing). The metadata update/merge
> happens when the artifact is
>   fetched from the proxies (but this is on a per repository level, there's
> no metadata of metadata files
>   from each repo being merged at the repo group level)
>
> The same process goes if the request made is for a metadata file. I don't
> know why a blank metadata file was returned.. :(
>
> Thanks,
> Deng
>
> On Thu, Jul 10, 2008 at 3:39 PM, nicolas de loof <nicolas@...>
> wrote:
>
> > I just started investigating on this. I can't find where the metadatas
> are
> > merged when a repository group is set. From my understanding,
> > ArchivaVirtualDavResource is used with a List<File> of metadatas from
> each
> > repo in the group. But I don't find WHERE the metadata files are merged
> > into
> > a single XML content...
> >
> >
> >
> > 2008/7/10 Maria Odea Ching <oching@...>:
> >
> > > Ok, thanks Nico, Dan :)
> > > I'll look into this further.. I guess we could include this in 1.1.1
> > which
> > > is scheduled to be released right after 1.1 to fix another blocker in
> > 1.1.
> > >
> > > Thanks,
> > > Deng
> > >
> > > On Thu, Jul 10, 2008 at 2:37 PM, nicolas de loof <nicolas@...>
> > > wrote:
> > >
> > > > MRM-872 created for this, with the concrete case I fall into.
> > > >
> > > > If confirmed, this is a blocker for repository group use
> > > >
> > > > 2008/7/10 Dan Tran <dantran@...>:
> > > >
> > > > > sounds like  a blocking bug? would love to see this fixed in 1.1
> :-)
> > > > >
> > > > > On Wed, Jul 9, 2008 at 7:11 PM, Maria Odea Ching <
> oching@...>
> > > > > wrote:
> > > > > > Hi Nico,
> > > > > >
> > > > > > That looks like a new bug :( The metadata should be just the same
> > as
> > > > the
> > > > > > regular metadata of a managed repo (not belonging to a group) as
> it
> > > > uses
> > > > > the
> > > > > > same code for webdav & proxying. Could you file a jira for this?
> > > > > >
> > > > > > Thanks,
> > > > > > Deng
> > > > > >
> > > > > > On Wed, Jul 9, 2008 at 8:45 PM, nicolas de loof <
> > nicolas@...>
> > > > > wrote:
> > > > > >
> > > > > >> Hy,
> > > > > >>
> > > > > >> I get strange behaviour wen using repository group :
> > > > > >>
> > > > > >> From a fresh maven installation (empty local repo), with
> settings
> > > set
> > > > to
> > > > > >> mirror central to my archiva repository group URL, the eclipse
> 2.5
> > > > > plugin
> > > > > >> can't find the required org.eclipse.core:resources:[3.1.0,4.0.0)
> > > > > >>
> > > > > >> The maven-metadata.xml returned by the group URL is emty.
> > requesting
> > > > > >> metadatas from individual managed repos in the group are fine.
> Is
> > > this
> > > > a
> > > > > >> known limitation ?
> > > > > >>
> > > > > >> Nico.
> > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
>

[Maria Odea Ching-5]

Oh ok, sorry I thought it was a blank file returned. I haven't tried
replicating it yet, was just looking through the codes.
I guess either of the 2 options would work.. I'm still finishing up some
things in the search, I'll see what I can do for 872 later tonight.
Unless of course you want to work on it? :)

Thanks,
Deng

On Thu, Jul 10, 2008 at 5:13 PM, nicolas de loof <nicolas@...> wrote:

> This mean the First metadata file found in on repo of the group is returned
> !
>
> This is what happen : I get a metadat file with no <version> included (not
> an blank file, but an XML construct with just root element) : this is the
> content of my first repo in the group, that did not retrieve the expected
> artifact.
>
> I made a simple test : place "release" repo before "corporate" in the
> group,
> and I the get the expected metadata.
>
> My conclusion is we have two options :
>
> - support metadata merge from repositories in the group (not just "first
> win")
> - don't create metadata in managed repo if no artifact was found from
> proxies.
>
>
>
>
>
> 2008/7/10 Maria Odea Ching <oching@...>:
>
> > The ArchivaVirtualDavResource is for the dav browse (directories), not
> for
> > artifact requests..
> >
> > Hmm, the virtual repos work this way:
> > - client requests for an artifact
> > - RepoServlet handles the request and goes to ArchivaDavSessionProvider
> > - in ArchivaDavSessionProvider, the request is checked if it is a repo
> > group
> > url or a regular repo url.
> >   If it is, then loop through the repos under the group. The first
> > requested artifact found among the repos
> >   is returned (proxied or already existing). The metadata update/merge
> > happens when the artifact is
> >   fetched from the proxies (but this is on a per repository level,
> there's
> > no metadata of metadata files
> >   from each repo being merged at the repo group level)
> >
> > The same process goes if the request made is for a metadata file. I don't
> > know why a blank metadata file was returned.. :(
> >
> > Thanks,
> > Deng
> >
> > On Thu, Jul 10, 2008 at 3:39 PM, nicolas de loof <nicolas@...>
> > wrote:
> >
> > > I just started investigating on this. I can't find where the metadatas
> > are
> > > merged when a repository group is set. From my understanding,
> > > ArchivaVirtualDavResource is used with a List<File> of metadatas from
> > each
> > > repo in the group. But I don't find WHERE the metadata files are merged
> > > into
> > > a single XML content...
> > >
> > >
> > >
> > > 2008/7/10 Maria Odea Ching <oching@...>:
> > >
> > > > Ok, thanks Nico, Dan :)
> > > > I'll look into this further.. I guess we could include this in 1.1.1
> > > which
> > > > is scheduled to be released right after 1.1 to fix another blocker in
> > > 1.1.
> > > >
> > > > Thanks,
> > > > Deng
> > > >
> > > > On Thu, Jul 10, 2008 at 2:37 PM, nicolas de loof <nicolas@...
> >
> > > > wrote:
> > > >
> > > > > MRM-872 created for this, with the concrete case I fall into.
> > > > >
> > > > > If confirmed, this is a blocker for repository group use
> > > > >
> > > > > 2008/7/10 Dan Tran <dantran@...>:
> > > > >
> > > > > > sounds like  a blocking bug? would love to see this fixed in 1.1
> > :-)
> > > > > >
> > > > > > On Wed, Jul 9, 2008 at 7:11 PM, Maria Odea Ching <
> > oching@...>
> > > > > > wrote:
> > > > > > > Hi Nico,
> > > > > > >
> > > > > > > That looks like a new bug :( The metadata should be just the
> same
> > > as
> > > > > the
> > > > > > > regular metadata of a managed repo (not belonging to a group)
> as
> > it
> > > > > uses
> > > > > > the
> > > > > > > same code for webdav & proxying. Could you file a jira for
> this?
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Deng
> > > > > > >
> > > > > > > On Wed, Jul 9, 2008 at 8:45 PM, nicolas de loof <
> > > nicolas@...>
> > > > > > wrote:
> > > > > > >
> > > > > > >> Hy,
> > > > > > >>
> > > > > > >> I get strange behaviour wen using repository group :
> > > > > > >>
> > > > > > >> From a fresh maven installation (empty local repo), with
> > settings
> > > > set
> > > > > to
> > > > > > >> mirror central to my archiva repository group URL, the eclipse
> > 2.5
> > > > > > plugin
> > > > > > >> can't find the required
> org.eclipse.core:resources:[3.1.0,4.0.0)
> > > > > > >>
> > > > > > >> The maven-metadata.xml returned by the group URL is emty.
> > > requesting
> > > > > > >> metadatas from individual managed repos in the group are fine.
> > Is
> > > > this
> > > > > a
> > > > > > >> known limitation ?
> > > > > > >>
> > > > > > >> Nico.
> > > > > > >>
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

[nicolas de loof-3]

2nd option may be simplier but not really nice, let's consider this use case
:

A user has an issue with a maven plugin. As a nice guy, he creates an issue,
attach a patch, but to get quick fix he deploys a snapshot to its corporate
repo.

To get it using a group repository ("public" proxying public repos +
"corporate"), we need to merge the metadatas from both repositories, so that
the custom snapshot will get listed in metadatas + all available version on
public snapshot repositories.

From my understanding, this will require some "*list available files*" logic
+ "*merge metadatas*" in ArchivaDavResource.createResource(), and not just
"return resource;"

I've commited the "*list available files*" part that was trivial.

I'm not used with Dav (jackRabit) API and metadata file format, so any help
for the "*merge metadatas*"  would be welcome (TODO at line 263). This would
require to create an "in memory" virtual resource for the merged metadata.

Nico.



2008/7/10 Maria Odea Ching <oching@...>:

> Oh ok, sorry I thought it was a blank file returned. I haven't tried
> replicating it yet, was just looking through the codes.
> I guess either of the 2 options would work.. I'm still finishing up some
> things in the search, I'll see what I can do for 872 later tonight.
> Unless of course you want to work on it? :)
>
> Thanks,
> Deng
>
> On Thu, Jul 10, 2008 at 5:13 PM, nicolas de loof <nicolas@...>
> wrote:
>
> > This mean the First metadata file found in on repo of the group is
> returned
> > !
> >
> > This is what happen : I get a metadat file with no <version> included
> (not
> > an blank file, but an XML construct with just root element) : this is the
> > content of my first repo in the group, that did not retrieve the expected
> > artifact.
> >
> > I made a simple test : place "release" repo before "corporate" in the
> > group,
> > and I the get the expected metadata.
> >
> > My conclusion is we have two options :
> >
> > - support metadata merge from repositories in the group (not just "first
> > win")
> > - don't create metadata in managed repo if no artifact was found from
> > proxies.
> >
> >
> >
> >
> >
> > 2008/7/10 Maria Odea Ching <oching@...>:
> >
> > > The ArchivaVirtualDavResource is for the dav browse (directories), not
> > for
> > > artifact requests..
> > >
> > > Hmm, the virtual repos work this way:
> > > - client requests for an artifact
> > > - RepoServlet handles the request and goes to ArchivaDavSessionProvider
> > > - in ArchivaDavSessionProvider, the request is checked if it is a repo
> > > group
> > > url or a regular repo url.
> > >   If it is, then loop through the repos under the group. The first
> > > requested artifact found among the repos
> > >   is returned (proxied or already existing). The metadata update/merge
> > > happens when the artifact is
> > >   fetched from the proxies (but this is on a per repository level,
> > there's
> > > no metadata of metadata files
> > >   from each repo being merged at the repo group level)
> > >
> > > The same process goes if the request made is for a metadata file. I
> don't
> > > know why a blank metadata file was returned.. :(
> > >
> > > Thanks,
> > > Deng
> > >
> > > On Thu, Jul 10, 2008 at 3:39 PM, nicolas de loof <nicolas@...>
> > > wrote:
> > >
> > > > I just started investigating on this. I can't find where the
> metadatas
> > > are
> > > > merged when a repository group is set. From my understanding,
> > > > ArchivaVirtualDavResource is used with a List<File> of metadatas from
> > > each
> > > > repo in the group. But I don't find WHERE the metadata files are
> merged
> > > > into
> > > > a single XML content...
> > > >
> > > >
> > > >
> > > > 2008/7/10 Maria Odea Ching <oching@...>:
> > > >
> > > > > Ok, thanks Nico, Dan :)
> > > > > I'll look into this further.. I guess we could include this in
> 1.1.1
> > > > which
> > > > > is scheduled to be released right after 1.1 to fix another blocker
> in
> > > > 1.1.
> > > > >
> > > > > Thanks,
> > > > > Deng
> > > > >
> > > > > On Thu, Jul 10, 2008 at 2:37 PM, nicolas de loof <
> nicolas@...
> > >
> > > > > wrote:
> > > > >
> > > > > > MRM-872 created for this, with the concrete case I fall into.
> > > > > >
> > > > > > If confirmed, this is a blocker for repository group use
> > > > > >
> > > > > > 2008/7/10 Dan Tran <dantran@...>:
> > > > > >
> > > > > > > sounds like  a blocking bug? would love to see this fixed in
> 1.1
> > > :-)
> > > > > > >
> > > > > > > On Wed, Jul 9, 2008 at 7:11 PM, Maria Odea Ching <
> > > oching@...>
> > > > > > > wrote:
> > > > > > > > Hi Nico,
> > > > > > > >
> > > > > > > > That looks like a new bug :( The metadata should be just the
> > same
> > > > as
> > > > > > the
> > > > > > > > regular metadata of a managed repo (not belonging to a group)
> > as
> > > it
> > > > > > uses
> > > > > > > the
> > > > > > > > same code for webdav & proxying. Could you file a jira for
> > this?
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Deng
> > > > > > > >
> > > > > > > > On Wed, Jul 9, 2008 at 8:45 PM, nicolas de loof <
> > > > nicolas@...>
> > > > > > > wrote:
> > > > > > > >
> > > > > > > >> Hy,
> > > > > > > >>
> > > > > > > >> I get strange behaviour wen using repository group :
> > > > > > > >>
> > > > > > > >> From a fresh maven installation (empty local repo), with
> > > settings
> > > > > set
> > > > > > to
> > > > > > > >> mirror central to my archiva repository group URL, the
> eclipse
> > > 2.5
> > > > > > > plugin
> > > > > > > >> can't find the required
> > org.eclipse.core:resources:[3.1.0,4.0.0)
> > > > > > > >>
> > > > > > > >> The maven-metadata.xml returned by the group URL is emty.
> > > > requesting
> > > > > > > >> metadatas from individual managed repos in the group are
> fine.
> > > Is
> > > > > this
> > > > > > a
> > > > > > > >> known limitation ?
> > > > > > > >>
> > > > > > > >> Nico.
> > > > > > > >>
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

[Maria Odea Ching-5]

Okay, thanks Nico.. I think I can do the merging stuff. Will be back online
later when I get home..

-Deng

On Thu, Jul 10, 2008 at 6:14 PM, nicolas de loof <nicolas@...> wrote:

> 2nd option may be simplier but not really nice, let's consider this use
> case
> :
>
> A user has an issue with a maven plugin. As a nice guy, he creates an
> issue,
> attach a patch, but to get quick fix he deploys a snapshot to its corporate
> repo.
>
> To get it using a group repository ("public" proxying public repos +
> "corporate"), we need to merge the metadatas from both repositories, so
> that
> the custom snapshot will get listed in metadatas + all available version on
> public snapshot repositories.
>
> From my understanding, this will require some "*list available files*"
> logic
> + "*merge metadatas*" in ArchivaDavResource.createResource(), and not just
> "return resource;"
>
> I've commited the "*list available files*" part that was trivial.
>
> I'm not used with Dav (jackRabit) API and metadata file format, so any help
> for the "*merge metadatas*"  would be welcome (TODO at line 263). This
> would
> require to create an "in memory" virtual resource for the merged metadata.
>
> Nico.
>
>
>
> 2008/7/10 Maria Odea Ching <oching@...>:
>
> > Oh ok, sorry I thought it was a blank file returned. I haven't tried
> > replicating it yet, was just looking through the codes.
> > I guess either of the 2 options would work.. I'm still finishing up some
> > things in the search, I'll see what I can do for 872 later tonight.
> > Unless of course you want to work on it? :)
> >
> > Thanks,
> > Deng
> >
> > On Thu, Jul 10, 2008 at 5:13 PM, nicolas de loof <nicolas@...>
> > wrote:
> >
> > > This mean the First metadata file found in on repo of the group is
> > returned
> > > !
> > >
> > > This is what happen : I get a metadat file with no <version> included
> > (not
> > > an blank file, but an XML construct with just root element) : this is
> the
> > > content of my first repo in the group, that did not retrieve the
> expected
> > > artifact.
> > >
> > > I made a simple test : place "release" repo before "corporate" in the
> > > group,
> > > and I the get the expected metadata.
> > >
> > > My conclusion is we have two options :
> > >
> > > - support metadata merge from repositories in the group (not just
> "first
> > > win")
> > > - don't create metadata in managed repo if no artifact was found from
> > > proxies.
> > >
> > >
> > >
> > >
> > >
> > > 2008/7/10 Maria Odea Ching <oching@...>:
> > >
> > > > The ArchivaVirtualDavResource is for the dav browse (directories),
> not
> > > for
> > > > artifact requests..
> > > >
> > > > Hmm, the virtual repos work this way:
> > > > - client requests for an artifact
> > > > - RepoServlet handles the request and goes to
> ArchivaDavSessionProvider
> > > > - in ArchivaDavSessionProvider, the request is checked if it is a
> repo
> > > > group
> > > > url or a regular repo url.
> > > >   If it is, then loop through the repos under the group. The first
> > > > requested artifact found among the repos
> > > >   is returned (proxied or already existing). The metadata
> update/merge
> > > > happens when the artifact is
> > > >   fetched from the proxies (but this is on a per repository level,
> > > there's
> > > > no metadata of metadata files
> > > >   from each repo being merged at the repo group level)
> > > >
> > > > The same process goes if the request made is for a metadata file. I
> > don't
> > > > know why a blank metadata file was returned.. :(
> > > >
> > > > Thanks,
> > > > Deng
> > > >
> > > > On Thu, Jul 10, 2008 at 3:39 PM, nicolas de loof <nicolas@...
> >
> > > > wrote:
> > > >
> > > > > I just started investigating on this. I can't find where the
> > metadatas
> > > > are
> > > > > merged when a repository group is set. From my understanding,
> > > > > ArchivaVirtualDavResource is used with a List<File> of metadatas
> from
> > > > each
> > > > > repo in the group. But I don't find WHERE the metadata files are
> > merged
> > > > > into
> > > > > a single XML content...
> > > > >
> > > > >
> > > > >
> > > > > 2008/7/10 Maria Odea Ching <oching@...>:
> > > > >
> > > > > > Ok, thanks Nico, Dan :)
> > > > > > I'll look into this further.. I guess we could include this in
> > 1.1.1
> > > > > which
> > > > > > is scheduled to be released right after 1.1 to fix another
> blocker
> > in
> > > > > 1.1.
> > > > > >
> > > > > > Thanks,
> > > > > > Deng
> > > > > >
> > > > > > On Thu, Jul 10, 2008 at 2:37 PM, nicolas de loof <
> > nicolas@...
> > > >
> > > > > > wrote:
> > > > > >
> > > > > > > MRM-872 created for this, with the concrete case I fall into.
> > > > > > >
> > > > > > > If confirmed, this is a blocker for repository group use
> > > > > > >
> > > > > > > 2008/7/10 Dan Tran <dantran@...>:
> > > > > > >
> > > > > > > > sounds like  a blocking bug? would love to see this fixed in
> > 1.1
> > > > :-)
> > > > > > > >
> > > > > > > > On Wed, Jul 9, 2008 at 7:11 PM, Maria Odea Ching <
> > > > oching@...>
> > > > > > > > wrote:
> > > > > > > > > Hi Nico,
> > > > > > > > >
> > > > > > > > > That looks like a new bug :( The metadata should be just
> the
> > > same
> > > > > as
> > > > > > > the
> > > > > > > > > regular metadata of a managed repo (not belonging to a
> group)
> > > as
> > > > it
> > > > > > > uses
> > > > > > > > the
> > > > > > > > > same code for webdav & proxying. Could you file a jira for
> > > this?
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > > Deng
> > > > > > > > >
> > > > > > > > > On Wed, Jul 9, 2008 at 8:45 PM, nicolas de loof <
> > > > > nicolas@...>
> > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > >> Hy,
> > > > > > > > >>
> > > > > > > > >> I get strange behaviour wen using repository group :
> > > > > > > > >>
> > > > > > > > >> From a fresh maven installation (empty local repo), with
> > > > settings
> > > > > > set
> > > > > > > to
> > > > > > > > >> mirror central to my archiva repository group URL, the
> > eclipse
> > > > 2.5
> > > > > > > > plugin
> > > > > > > > >> can't find the required
> > > org.eclipse.core:resources:[3.1.0,4.0.0)
> > > > > > > > >>
> > > > > > > > >> The maven-metadata.xml returned by the group URL is emty.
> > > > > requesting
> > > > > > > > >> metadatas from individual managed repos in the group are
> > fine.
> > > > Is
> > > > > > this
> > > > > > > a
> > > > > > > > >> known limitation ?
> > > > > > > > >>
> > > > > > > > >> Nico.
> > > > > > > > >>
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

[nicolas de loof-3]

I've looked at MetaDataTools class.It merges the metadatas from multiple
repositories in a target file. This could be refactored into another method
to return the ArchivaRepositoryMetadata objet, so that it can be used for
repo grouping.

The "*write to file*" process should also be skipped when there is no
concrete metadata (availableVersions or Plugins) to avoid creating "empty"
metadatas in managed repo for requested artifacts that aren't available.

Could this be a security issue with archiva ? requesting a fake artifact
metadata creates a new File. Requesting thousand of them with a random name
will create thousand files. This could be the basis of a "brute force" deny
of service attack, with file system becoming full. Or maybe I'm a paranoid ?

Nicolas.

2008/7/10 Maria Odea Ching <oching@...>:

> Okay, thanks Nico.. I think I can do the merging stuff. Will be back online
> later when I get home..
>
> -Deng
>
> On Thu, Jul 10, 2008 at 6:14 PM, nicolas de loof <nicolas@...>
> wrote:
>
> > 2nd option may be simplier but not really nice, let's consider this use
> > case
> > :
> >
> > A user has an issue with a maven plugin. As a nice guy, he creates an
> > issue,
> > attach a patch, but to get quick fix he deploys a snapshot to its
> corporate
> > repo.
> >
> > To get it using a group repository ("public" proxying public repos +
> > "corporate"), we need to merge the metadatas from both repositories, so
> > that
> > the custom snapshot will get listed in metadatas + all available version
> on
> > public snapshot repositories.
> >
> > From my understanding, this will require some "*list available files*"
> > logic
> > + "*merge metadatas*" in ArchivaDavResource.createResource(), and not
> just
> > "return resource;"
> >
> > I've commited the "*list available files*" part that was trivial.
> >
> > I'm not used with Dav (jackRabit) API and metadata file format, so any
> help
> > for the "*merge metadatas*"  would be welcome (TODO at line 263). This
> > would
> > require to create an "in memory" virtual resource for the merged
> metadata.
> >
> > Nico.
> >
> >
> >
> > 2008/7/10 Maria Odea Ching <oching@...>:
> >
> > > Oh ok, sorry I thought it was a blank file returned. I haven't tried
> > > replicating it yet, was just looking through the codes.
> > > I guess either of the 2 options would work.. I'm still finishing up
> some
> > > things in the search, I'll see what I can do for 872 later tonight.
> > > Unless of course you want to work on it? :)
> > >
> > > Thanks,
> > > Deng
> > >
> > > On Thu, Jul 10, 2008 at 5:13 PM, nicolas de loof <nicolas@...>
> > > wrote:
> > >
> > > > This mean the First metadata file found in on repo of the group is
> > > returned
> > > > !
> > > >
> > > > This is what happen : I get a metadat file with no <version> included
> > > (not
> > > > an blank file, but an XML construct with just root element) : this is
> > the
> > > > content of my first repo in the group, that did not retrieve the
> > expected
> > > > artifact.
> > > >
> > > > I made a simple test : place "release" repo before "corporate" in the
> > > > group,
> > > > and I the get the expected metadata.
> > > >
> > > > My conclusion is we have two options :
> > > >
> > > > - support metadata merge from repositories in the group (not just
> > "first
> > > > win")
> > > > - don't create metadata in managed repo if no artifact was found from
> > > > proxies.
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > 2008/7/10 Maria Odea Ching <oching@...>:
> > > >
> > > > > The ArchivaVirtualDavResource is for the dav browse (directories),
> > not
> > > > for
> > > > > artifact requests..
> > > > >
> > > > > Hmm, the virtual repos work this way:
> > > > > - client requests for an artifact
> > > > > - RepoServlet handles the request and goes to
> > ArchivaDavSessionProvider
> > > > > - in ArchivaDavSessionProvider, the request is checked if it is a
> > repo
> > > > > group
> > > > > url or a regular repo url.
> > > > >   If it is, then loop through the repos under the group. The first
> > > > > requested artifact found among the repos
> > > > >   is returned (proxied or already existing). The metadata
> > update/merge
> > > > > happens when the artifact is
> > > > >   fetched from the proxies (but this is on a per repository level,
> > > > there's
> > > > > no metadata of metadata files
> > > > >   from each repo being merged at the repo group level)
> > > > >
> > > > > The same process goes if the request made is for a metadata file. I
> > > don't
> > > > > know why a blank metadata file was returned.. :(
> > > > >
> > > > > Thanks,
> > > > > Deng
> > > > >
> > > > > On Thu, Jul 10, 2008 at 3:39 PM, nicolas de loof <
> nicolas@...
> > >
> > > > > wrote:
> > > > >
> > > > > > I just started investigating on this. I can't find where the
> > > metadatas
> > > > > are
> > > > > > merged when a repository group is set. From my understanding,
> > > > > > ArchivaVirtualDavResource is used with a List<File> of metadatas
> > from
> > > > > each
> > > > > > repo in the group. But I don't find WHERE the metadata files are
> > > merged
> > > > > > into
> > > > > > a single XML content...
> > > > > >
> > > > > >
> > > > > >
> > > > > > 2008/7/10 Maria Odea Ching <oching@...>:
> > > > > >
> > > > > > > Ok, thanks Nico, Dan :)
> > > > > > > I'll look into this further.. I guess we could include this in
> > > 1.1.1
> > > > > > which
> > > > > > > is scheduled to be released right after 1.1 to fix another
> > blocker
> > > in
> > > > > > 1.1.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Deng
> > > > > > >
> > > > > > > On Thu, Jul 10, 2008 at 2:37 PM, nicolas de loof <
> > > nicolas@...
> > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > MRM-872 created for this, with the concrete case I fall into.
> > > > > > > >
> > > > > > > > If confirmed, this is a blocker for repository group use
> > > > > > > >
> > > > > > > > 2008/7/10 Dan Tran <dantran@...>:
> > > > > > > >
> > > > > > > > > sounds like  a blocking bug? would love to see this fixed
> in
> > > 1.1
> > > > > :-)
> > > > > > > > >
> > > > > > > > > On Wed, Jul 9, 2008 at 7:11 PM, Maria Odea Ching <
> > > > > oching@...>
> > > > > > > > > wrote:
> > > > > > > > > > Hi Nico,
> > > > > > > > > >
> > > > > > > > > > That looks like a new bug :( The metadata should be just
> > the
> > > > same
> > > > > > as
> > > > > > > > the
> > > > > > > > > > regular metadata of a managed repo (not belonging to a
> > group)
> > > > as
> > > > > it
> > > > > > > > uses
> > > > > > > > > the
> > > > > > > > > > same code for webdav & proxying. Could you file a jira
> for
> > > > this?
> > >