[jira] Created: (MNG-3379) Parallel resolution of artifacts

Parallel resolution of artifacts
--------------------------------

                 Key: MNG-3379
                 URL: http://jira.codehaus.org/browse/MNG-3379
             Project: Maven 2
          Issue Type: New Feature
          Components: Artifacts and Repositories
    Affects Versions: 2.0.8
            Reporter: Don Brown
         Attachments: parallel-resolution.diff

Artifacts should be resolved in parallel, grouped by group id's to get around the lack of synchronization in the local repository.  The patch does the following:
* Use a ThreadPoolExecutor to parallelize artifact resolution, but takes care not to resolve multiple artifacts from the same group id simultaneously. (requires Java 5)
* Makes the http wagon the default instead of the poor performing http-client

Disadvantages:
* Requires Java 5, but the backport jars could be substituted pretty easily
* Breaks some plugins due to commons-logging being in the Maven uber jar (required by commons-httpclient), notably the apt plugin (maybe more should use the isolatedRealm setting?)
* Screws up the progress monitor as multiple threads are updating it

Advantages:
* Much faster when combined with the http wagon (WAGON-98).  I was seeing 40% improvement on some test builds.


--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

    [ http://jira.codehaus.org/browse/MNG-3379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_121552 ]

Chris Custine commented on MNG-3379:
------------------------------------

I like the way this is going, but if I am not mistaken it appears that the patch ignores certain settings such as -Dmaven.repo.local and mirror settings in ~/.m2/settings.xml.

> Parallel resolution of artifacts
> --------------------------------
>
>                 Key: MNG-3379
>                 URL: http://jira.codehaus.org/browse/MNG-3379
>             Project: Maven 2
>          Issue Type: New Feature
>          Components: Artifacts and Repositories
>    Affects Versions: 2.0.8
>            Reporter: Don Brown
>         Attachments: parallel-resolution.diff
>
>
> Artifacts should be resolved in parallel, grouped by group id's to get around the lack of synchronization in the local repository.  The patch does the following:
> * Use a ThreadPoolExecutor to parallelize artifact resolution, but takes care not to resolve multiple artifacts from the same group id simultaneously. (requires Java 5)
> * Makes the http wagon the default instead of the poor performing http-client
> Disadvantages:
> * Requires Java 5, but the backport jars could be substituted pretty easily
> * Breaks some plugins due to commons-logging being in the Maven uber jar (required by commons-httpclient), notably the apt plugin (maybe more should use the isolatedRealm setting?)
> * Screws up the progress monitor as multiple threads are updating it
> Advantages:
> * Much faster when combined with the http wagon (WAGON-98).  I was seeing 40% improvement on some test builds.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

    [ http://jira.codehaus.org/browse/MNG-3379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_121595 ]

Don Brown commented on MNG-3379:
--------------------------------

How so?  The patch simply schedules the artifact resolution differently, but the same resolution code should work.  

> Parallel resolution of artifacts
> --------------------------------
>
>                 Key: MNG-3379
>                 URL: http://jira.codehaus.org/browse/MNG-3379
>             Project: Maven 2
>          Issue Type: New Feature
>          Components: Artifacts and Repositories
>    Affects Versions: 2.0.8
>            Reporter: Don Brown
>         Attachments: parallel-resolution.diff
>
>
> Artifacts should be resolved in parallel, grouped by group id's to get around the lack of synchronization in the local repository.  The patch does the following:
> * Use a ThreadPoolExecutor to parallelize artifact resolution, but takes care not to resolve multiple artifacts from the same group id simultaneously. (requires Java 5)
> * Makes the http wagon the default instead of the poor performing http-client
> Disadvantages:
> * Requires Java 5, but the backport jars could be substituted pretty easily
> * Breaks some plugins due to commons-logging being in the Maven uber jar (required by commons-httpclient), notably the apt plugin (maybe more should use the isolatedRealm setting?)
> * Screws up the progress monitor as multiple threads are updating it
> Advantages:
> * Much faster when combined with the http wagon (WAGON-98).  I was seeing 40% improvement on some test builds.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

     [ http://jira.codehaus.org/browse/MNG-3379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Don Brown updated MNG-3379:
---------------------------

    Attachment: parallel-resolution-2.diff

Updated patch that fixes a few concurrency issues, adds support for bigger builds, updates all wagon deps to rc1-SNAPSHOT

> Parallel resolution of artifacts
> --------------------------------
>
>                 Key: MNG-3379
>                 URL: http://jira.codehaus.org/browse/MNG-3379
>             Project: Maven 2
>          Issue Type: New Feature
>          Components: Artifacts and Repositories
>    Affects Versions: 2.0.8
>            Reporter: Don Brown
>         Attachments: parallel-resolution-2.diff, parallel-resolution.diff
>
>
> Artifacts should be resolved in parallel, grouped by group id's to get around the lack of synchronization in the local repository.  The patch does the following:
> * Use a ThreadPoolExecutor to parallelize artifact resolution, but takes care not to resolve multiple artifacts from the same group id simultaneously. (requires Java 5)
> * Makes the http wagon the default instead of the poor performing http-client
> Disadvantages:
> * Requires Java 5, but the backport jars could be substituted pretty easily
> * Breaks some plugins due to commons-logging being in the Maven uber jar (required by commons-httpclient), notably the apt plugin (maybe more should use the isolatedRealm setting?)
> * Screws up the progress monitor as multiple threads are updating it
> Advantages:
> * Much faster when combined with the http wagon (WAGON-98).  I was seeing 40% improvement on some test builds.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

     [ http://jira.codehaus.org/browse/MNG-3379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Don Brown updated MNG-3379:
---------------------------

    Attachment: parallel-resolution-3.diff

Updated patch (version 3) that works with Java 1.4 and eliminates problem with commons-logging (depends on WAGONHTTP-17)

> Parallel resolution of artifacts
> --------------------------------
>
>                 Key: MNG-3379
>                 URL: http://jira.codehaus.org/browse/MNG-3379
>             Project: Maven 2
>          Issue Type: New Feature
>          Components: Artifacts and Repositories
>    Affects Versions: 2.0.8
>            Reporter: Don Brown
>         Attachments: parallel-resolution-2.diff, parallel-resolution-3.diff, parallel-resolution.diff
>
>
> Artifacts should be resolved in parallel, grouped by group id's to get around the lack of synchronization in the local repository.  The patch does the following:
> * Use a ThreadPoolExecutor to parallelize artifact resolution, but takes care not to resolve multiple artifacts from the same group id simultaneously. (requires Java 5)
> * Makes the http wagon the default instead of the poor performing http-client
> Disadvantages:
> * Requires Java 5, but the backport jars could be substituted pretty easily
> * Breaks some plugins due to commons-logging being in the Maven uber jar (required by commons-httpclient), notably the apt plugin (maybe more should use the isolatedRealm setting?)
> * Screws up the progress monitor as multiple threads are updating it
> Advantages:
> * Much faster when combined with the http wagon (WAGON-98).  I was seeing 40% improvement on some test builds.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

    [ http://jira.codehaus.org/browse/MNG-3379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_121656 ]

Chris Custine commented on MNG-3379:
------------------------------------

Hey Don, you are right, it isn't your patch that broke this.  I was testing this patch against 2.0.9-SNAPSHOT from SVN and got your changes intermingled with some recent updates from SVN which broke maven.repo.local setting.

> Parallel resolution of artifacts
> --------------------------------
>
>                 Key: MNG-3379
>                 URL: http://jira.codehaus.org/browse/MNG-3379
>             Project: Maven 2
>          Issue Type: New Feature
>          Components: Artifacts and Repositories
>    Affects Versions: 2.0.8
>            Reporter: Don Brown
>         Attachments: parallel-resolution-2.diff, parallel-resolution-3.diff, parallel-resolution.diff
>
>
> Artifacts should be resolved in parallel, grouped by group id's to get around the lack of synchronization in the local repository.  The patch does the following:
> * Use a ThreadPoolExecutor to parallelize artifact resolution, but takes care not to resolve multiple artifacts from the same group id simultaneously. (requires Java 5)
> * Makes the http wagon the default instead of the poor performing http-client
> Disadvantages:
> * Requires Java 5, but the backport jars could be substituted pretty easily
> * Breaks some plugins due to commons-logging being in the Maven uber jar (required by commons-httpclient), notably the apt plugin (maybe more should use the isolatedRealm setting?)
> * Screws up the progress monitor as multiple threads are updating it
> Advantages:
> * Much faster when combined with the http wagon (WAGON-98).  I was seeing 40% improvement on some test builds.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

    [ http://jira.codehaus.org/browse/MNG-3379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_121747 ]

Brett Porter commented on MNG-3379:
-----------------------------------

Hey Chris - have you created a new issue for the problems you are experiencing?

> Parallel resolution of artifacts
> --------------------------------
>
>                 Key: MNG-3379
>                 URL: http://jira.codehaus.org/browse/MNG-3379
>             Project: Maven 2
>          Issue Type: New Feature
>          Components: Artifacts and Repositories
>    Affects Versions: 2.0.8
>            Reporter: Don Brown
>         Attachments: parallel-resolution-2.diff, parallel-resolution-3.diff, parallel-resolution.diff
>
>
> Artifacts should be resolved in parallel, grouped by group id's to get around the lack of synchronization in the local repository.  The patch does the following:
> * Use a ThreadPoolExecutor to parallelize artifact resolution, but takes care not to resolve multiple artifacts from the same group id simultaneously. (requires Java 5)
> * Makes the http wagon the default instead of the poor performing http-client
> Disadvantages:
> * Requires Java 5, but the backport jars could be substituted pretty easily
> * Breaks some plugins due to commons-logging being in the Maven uber jar (required by commons-httpclient), notably the apt plugin (maybe more should use the isolatedRealm setting?)
> * Screws up the progress monitor as multiple threads are updating it
> Advantages:
> * Much faster when combined with the http wagon (WAGON-98).  I was seeing 40% improvement on some test builds.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

     [ http://jira.codehaus.org/browse/MNG-3379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Work on MNG-3379 started by Brett Porter.

> Parallel resolution of artifacts
> --------------------------------
>
>                 Key: MNG-3379
>                 URL: http://jira.codehaus.org/browse/MNG-3379
>             Project: Maven 2
>          Issue Type: New Feature
>          Components: Artifacts and Repositories
>    Affects Versions: 2.0.8
>            Reporter: Don Brown
>            Assignee: Brett Porter
>         Attachments: parallel-resolution-2.diff, parallel-resolution-3.diff, parallel-resolution.diff
>
>
> Artifacts should be resolved in parallel, grouped by group id's to get around the lack of synchronization in the local repository.  The patch does the following:
> * Use a ThreadPoolExecutor to parallelize artifact resolution, but takes care not to resolve multiple artifacts from the same group id simultaneously. (requires Java 5)
> * Makes the http wagon the default instead of the poor performing http-client
> Disadvantages:
> * Requires Java 5, but the backport jars could be substituted pretty easily
> * Breaks some plugins due to commons-logging being in the Maven uber jar (required by commons-httpclient), notably the apt plugin (maybe more should use the isolatedRealm setting?)
> * Screws up the progress monitor as multiple threads are updating it
> Advantages:
> * Much faster when combined with the http wagon (WAGON-98).  I was seeing 40% improvement on some test builds.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

    [ http://jira.codehaus.org/browse/MNG-3379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_122224 ]

Brett Porter commented on MNG-3379:
-----------------------------------

Don, can you clarify the 3rd patch? It seems that you are using the normal wagon-http, not the standalone one from WAGONHTTP-17 - is this because you just substituted it in your local repo, or did you actually get it working with the standard dependency?

> Parallel resolution of artifacts
> --------------------------------
>
>                 Key: MNG-3379
>                 URL: http://jira.codehaus.org/browse/MNG-3379
>             Project: Maven 2
>          Issue Type: New Feature
>          Components: Artifacts and Repositories
>    Affects Versions: 2.0.8
>            Reporter: Don Brown
>            Assignee: Brett Porter
>         Attachments: parallel-resolution-2.diff, parallel-resolution-3.diff, parallel-resolution.diff
>
>
> Artifacts should be resolved in parallel, grouped by group id's to get around the lack of synchronization in the local repository.  The patch does the following:
> * Use a ThreadPoolExecutor to parallelize artifact resolution, but takes care not to resolve multiple artifacts from the same group id simultaneously. (requires Java 5)
> * Makes the http wagon the default instead of the poor performing http-client
> Disadvantages:
> * Requires Java 5, but the backport jars could be substituted pretty easily
> * Breaks some plugins due to commons-logging being in the Maven uber jar (required by commons-httpclient), notably the apt plugin (maybe more should use the isolatedRealm setting?)
> * Screws up the progress monitor as multiple threads are updating it
> Advantages:
> * Much faster when combined with the http wagon (WAGON-98).  I was seeing 40% improvement on some test builds.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

    [ http://jira.codehaus.org/browse/MNG-3379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_122226 ]

Don Brown commented on MNG-3379:
--------------------------------

I substituted it in my local repo.  In my latest -db release, I changed the version to be more clear.

> Parallel resolution of artifacts
> --------------------------------
>
>                 Key: MNG-3379
>                 URL: http://jira.codehaus.org/browse/MNG-3379
>             Project: Maven 2
>          Issue Type: New Feature
>          Components: Artifacts and Repositories
>    Affects Versions: 2.0.8
>            Reporter: Don Brown
>            Assignee: Brett Porter
>         Attachments: parallel-resolution-2.diff, parallel-resolution-3.diff, parallel-resolution.diff
>
>
> Artifacts should be resolved in parallel, grouped by group id's to get around the lack of synchronization in the local repository.  The patch does the following:
> * Use a ThreadPoolExecutor to parallelize artifact resolution, but takes care not to resolve multiple artifacts from the same group id simultaneously. (requires Java 5)
> * Makes the http wagon the default instead of the poor performing http-client
> Disadvantages:
> * Requires Java 5, but the backport jars could be substituted pretty easily
> * Breaks some plugins due to commons-logging being in the Maven uber jar (required by commons-httpclient), notably the apt plugin (maybe more should use the isolatedRealm setting?)
> * Screws up the progress monitor as multiple threads are updating it
> Advantages:
> * Much faster when combined with the http wagon (WAGON-98).  I was seeing 40% improvement on some test builds.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

    [ http://jira.codehaus.org/browse/MNG-3379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_122227 ]

Brett Porter commented on MNG-3379:
-----------------------------------

do you have a test project that fails if you use standard wagon-http, but works with your version? The ones I've tested worked without making the substitution and I want to check the shade plugin is working as expected... you mentioned the apt plugin?

> Parallel resolution of artifacts
> --------------------------------
>
>                 Key: MNG-3379
>                 URL: http://jira.codehaus.org/browse/MNG-3379
>             Project: Maven 2
>          Issue Type: New Feature
>          Components: Artifacts and Repositories
>    Affects Versions: 2.0.8
>            Reporter: Don Brown
>            Assignee: Brett Porter
>         Attachments: parallel-resolution-2.diff, parallel-resolution-3.diff, parallel-resolution.diff
>
>
> Artifacts should be resolved in parallel, grouped by group id's to get around the lack of synchronization in the local repository.  The patch does the following:
> * Use a ThreadPoolExecutor to parallelize artifact resolution, but takes care not to resolve multiple artifacts from the same group id simultaneously. (requires Java 5)
> * Makes the http wagon the default instead of the poor performing http-client
> Disadvantages:
> * Requires Java 5, but the backport jars could be substituted pretty easily
> * Breaks some plugins due to commons-logging being in the Maven uber jar (required by commons-httpclient), notably the apt plugin (maybe more should use the isolatedRealm setting?)
> * Screws up the progress monitor as multiple threads are updating it
> Advantages:
> * Much faster when combined with the http wagon (WAGON-98).  I was seeing 40% improvement on some test builds.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

    [ http://jira.codehaus.org/browse/MNG-3379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_122228 ]

Don Brown commented on MNG-3379:
--------------------------------

Try struts2-core - https://svn.apache.org/repos/asf/struts/struts2/trunk/core  It uses the APT plugin, which depends on commons-logging.  Another one I've heard of is the xdoclet plugin.

> Parallel resolution of artifacts
> --------------------------------
>
>                 Key: MNG-3379
>                 URL: http://jira.codehaus.org/browse/MNG-3379
>             Project: Maven 2
>          Issue Type: New Feature
>          Components: Artifacts and Repositories
>    Affects Versions: 2.0.8
>            Reporter: Don Brown
>            Assignee: Brett Porter
>         Attachments: parallel-resolution-2.diff, parallel-resolution-3.diff, parallel-resolution.diff
>
>
> Artifacts should be resolved in parallel, grouped by group id's to get around the lack of synchronization in the local repository.  The patch does the following:
> * Use a ThreadPoolExecutor to parallelize artifact resolution, but takes care not to resolve multiple artifacts from the same group id simultaneously. (requires Java 5)
> * Makes the http wagon the default instead of the poor performing http-client
> Disadvantages:
> * Requires Java 5, but the backport jars could be substituted pretty easily
> * Breaks some plugins due to commons-logging being in the Maven uber jar (required by commons-httpclient), notably the apt plugin (maybe more should use the isolatedRealm setting?)
> * Screws up the progress monitor as multiple threads are updating it
> Advantages:
> * Much faster when combined with the http wagon (WAGON-98).  I was seeing 40% improvement on some test builds.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

    [ http://jira.codehaus.org/browse/MNG-3379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_122232 ]

Brett Porter commented on MNG-3379:
-----------------------------------

that worked. I'll be integrating this on another branch since some modifications are still needed and I noticed your own branch has moved on already :)

> Parallel resolution of artifacts
> --------------------------------
>
>                 Key: MNG-3379
>                 URL: http://jira.codehaus.org/browse/MNG-3379
>             Project: Maven 2
>          Issue Type: New Feature
>          Components: Artifacts and Repositories
>    Affects Versions: 2.0.8
>            Reporter: Don Brown
>            Assignee: Brett Porter
>         Attachments: parallel-resolution-2.diff, parallel-resolution-3.diff, parallel-resolution.diff
>
>
> Artifacts should be resolved in parallel, grouped by group id's to get around the lack of synchronization in the local repository.  The patch does the following:
> * Use a ThreadPoolExecutor to parallelize artifact resolution, but takes care not to resolve multiple artifacts from the same group id simultaneously. (requires Java 5)
> * Makes the http wagon the default instead of the poor performing http-client
> Disadvantages:
> * Requires Java 5, but the backport jars could be substituted pretty easily
> * Breaks some plugins due to commons-logging being in the Maven uber jar (required by commons-httpclient), notably the apt plugin (maybe more should use the isolatedRealm setting?)
> * Screws up the progress monitor as multiple threads are updating it
> Advantages:
> * Much faster when combined with the http wagon (WAGON-98).  I was seeing 40% improvement on some test builds.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

    [ http://jira.codehaus.org/browse/MNG-3379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_122233 ]

Brett Porter commented on MNG-3379:
-----------------------------------

branch is at https://svn.apache.org/repos/asf/maven/sandbox/branches/maven/MNG-3379

Here are the changes I'd like to review:
* formatting :)
* are RuntimeExceptions appropriate? Seems more like they should be normal artifact resolution failures and the list of missing artifacts stuck together
* may need a different progress monitor since this shows no progress any more (one that can be done in parallel also is great)
* I'd like to investigate controlling this by settings (pool size)
* needs to be able to be merged to Maven trunk somehow (and check compatibility with embedder)

> Parallel resolution of artifacts
> --------------------------------
>
>                 Key: MNG-3379
>                 URL: http://jira.codehaus.org/browse/MNG-3379
>             Project: Maven 2
>          Issue Type: New Feature
>          Components: Artifacts and Repositories
>    Affects Versions: 2.0.8
>            Reporter: Don Brown
>            Assignee: Brett Porter
>         Attachments: parallel-resolution-2.diff, parallel-resolution-3.diff, parallel-resolution.diff
>
>
> Artifacts should be resolved in parallel, grouped by group id's to get around the lack of synchronization in the local repository.  The patch does the following:
> * Use a ThreadPoolExecutor to parallelize artifact resolution, but takes care not to resolve multiple artifacts from the same group id simultaneously. (requires Java 5)
> * Makes the http wagon the default instead of the poor performing http-client
> Disadvantages:
> * Requires Java 5, but the backport jars could be substituted pretty easily
> * Breaks some plugins due to commons-logging being in the Maven uber jar (required by commons-httpclient), notably the apt plugin (maybe more should use the isolatedRealm setting?)
> * Screws up the progress monitor as multiple threads are updating it
> Advantages:
> * Much faster when combined with the http wagon (WAGON-98).  I was seeing 40% improvement on some test builds.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

    [ http://jira.codehaus.org/browse/MNG-3379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_122239 ]

Don Brown commented on MNG-3379:
--------------------------------

* are RuntimeExceptions appropriate? Seems more like they should be normal artifact resolution failures and the list of missing artifacts stuck together
Agreed.

* may need a different progress monitor since this shows no progress any more (one that can be done in parallel also is great)
Yeah, this is the last biggish chunk of work still to do.  The progress bar just goes crazy :)

* I'd like to investigate controlling this by settings (pool size)
Makes sense, but in my testing, it didn't see to help much after 5, but that could just be due to my particular multi-core box.

> Parallel resolution of artifacts
> --------------------------------
>
>                 Key: MNG-3379
>                 URL: http://jira.codehaus.org/browse/MNG-3379
>             Project: Maven 2
>          Issue Type: New Feature
>          Components: Artifacts and Repositories
>    Affects Versions: 2.0.8
>            Reporter: Don Brown
>            Assignee: Brett Porter
>         Attachments: parallel-resolution-2.diff, parallel-resolution-3.diff, parallel-resolution.diff
>
>
> Artifacts should be resolved in parallel, grouped by group id's to get around the lack of synchronization in the local repository.  The patch does the following:
> * Use a ThreadPoolExecutor to parallelize artifact resolution, but takes care not to resolve multiple artifacts from the same group id simultaneously. (requires Java 5)
> * Makes the http wagon the default instead of the poor performing http-client
> Disadvantages:
> * Requires Java 5, but the backport jars could be substituted pretty easily
> * Breaks some plugins due to commons-logging being in the Maven uber jar (required by commons-httpclient), notably the apt plugin (maybe more should use the isolatedRealm setting?)
> * Screws up the progress monitor as multiple threads are updating it
> Advantages:
> * Much faster when combined with the http wagon (WAGON-98).  I was seeing 40% improvement on some test builds.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira