Real-time effects stacks

[Jorge Rodriguez-7]

There have been two things I've always wanted out of Audacity that I could never get. The first one is a way to cut out a section of music while leaving the rest of it undisturbed. That was implemented recently, "Split Delete." (Should be default behavior in my opinion, deleting a section of sound shouldn't move other stuff around. But anyways!) The other one was real-time effects stacks, or the ability to apply effects non-destructively, realtime, during play. This transforms Audacity from a powerful sound editing tool into a multitrack recording studio.

So, I hear that 1.3.5 is coming out and then is the time to start putting new stuff in Audacity and it's important to me that this to be one of them, so I'm pretty sure I'm willing to spare a couple of hours of my week to make sure that this feature is part of the next Audacity release.

Questions. At a high level, what would be required to get this feature working? Audacity already supports applying effects, they just need to be applied during playback instead of immediately. I think you would need a UI for managing effects, and the order that they are processed in, and a mechanism for controlling the effects. The current dialogs that do this can probably be reused. And then of course, the code for applying the effects in realtime instead of destructively. Can someone elaborate on exactly what needs to happen? What I specifically want to know is even a vague estimate of how many man hours is this going to be? I've done things tangentally related to this, but not really the same thing, so who is going to help me when I invariably run into a wall?

PS. CVS?

--
Jorge Rodriguez
Matrieya Studios, LLC
Email: jrodriguez@...
Phone: [phone]
-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
Don't miss this year's exciting event. There's still time to save $100.
Use priority code J8TL2D2.
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
_______________________________________________
Audacity-devel mailing list
Audacity-devel@...
https://lists.sourceforge.net/lists/listinfo/audacity-devel

[James Crook]

Jorge Rodriguez wrote:

> There have been two things I've always wanted out of Audacity that I
> could never get. The first one is a way to cut out a section of music
> while leaving the rest of it undisturbed. That was implemented
> recently, "Split Delete." (Should be default behavior in my opinion,
> deleting a section of sound shouldn't move other stuff around. But
> anyways!) The other one was real-time effects stacks, or the ability
> to apply effects non-destructively, realtime, during play. This
> transforms Audacity from a powerful sound editing tool into a
> multitrack recording studio.
>
> So, I hear that 1.3.5 is coming out and then is the time to start
> putting new stuff in Audacity and it's important to me that this to be
> one of them, so I'm pretty sure I'm willing to spare a couple of hours
> of my week to make sure that this feature is part of the next Audacity
> release.
>
> Questions. At a high level, what would be required to get this feature
> working? Audacity already supports applying effects, they just need to
> be applied during playback instead of immediately. I think you would
> need a UI for managing effects, and the order that they are processed
> in, and a mechanism for controlling the effects. The current dialogs
> that do this can probably be reused. And then of course, the code for
> applying the effects in realtime instead of destructively. Can someone
> elaborate on exactly what needs to happen? What I specifically want to
> know is even a vague estimate of how many man hours is this going to
> be? I've done things tangentally related to this, but not really the
> same thing, so who is going to help me when I invariably run into a wall?
>
> PS. CVS?
>

CVS = http://en.wikipedia.org/wiki/Concurrent_Versions_System
GSoC = http://en.wikipedia.org/wiki/Google_summer_of_code

How many hours?  Lots and lots.  It is simply not realistic to attempt
this on two hours a week, with no prior experience of modifying
Audacity.  I'd strongly suggest you try out smaller modifications to
Audacity first, just to get the hang of compiling, testing, finding your
way around the code.  You'll get to the end goal faster.

We do have a plan for real time, which is to get there in stages, and
there are ways you can help us get there faster.  A first step is one of
the more risky GSoC projects we are doing this year.  Have a look at:
Michael Chinen's project on this page:
http://www.audacityteam.org/wiki/index.php?title=GSoC_2008_Projects

One way you could help is by understanding what is being planned and
why, why we are doing it this way, and when Michael starts coding trying
out his betas, helping to find problems early.  If you are understanding
his code, then you may be able to suggest fixes and improvements that
help progress the project - so he'll end up with a more complete
implementation than otherwise by the end of the project - but you'll
only be able to do that at all if you've already got fairly confident
and familiar with the Audacity code - and that takes time.  You should
try compiling Audacity from source and making small changes now to be
ready in time.

For the 'big picture', also have a look at Postfish Integration on
http://www.audacityteam.org/wiki/index.php?title=More_GSoC_Ideas

That's not going to happen during this GSoC, but it gives you an idea of
why doing real time is not a small simple change and it makes it clearer
why render-on-demand is a first step.  We'd certainly appreciate help
with getting there, but you'll need to be up to speed on developing with
Audacity for it to be helpful to us.  To get up to speed, pick some
smaller pieces/details of Audacity to work on, where you and we get
something we can use sooner.  It doesn't need to be in the audio-path
for it to help you gain experience with the code.  If you need any help
compiling Audacity from source just ask here.  We expect there to be
some problems at first, and we're well used to giving tips to help sort
out compile issues.


Good to hear from you,


--James.

-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
Don't miss this year's exciting event. There's still time to save $100.
Use priority code J8TL2D2.
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
_______________________________________________
Audacity-devel mailing list
Audacity-devel@...
https://lists.sourceforge.net/lists/listinfo/audacity-devel

[Michael Chinen]

Jorge,

Thanks for the email.  It has got me thinking about the long-term
structure of my project.

I have mixed feelings about non-destructive effect stacks in audio
editors.  Non-destructive real-time effects stacking is a very useful
and fundamental tool in a digital audio workstation like protools or
ardour.  That being said, I have always thought of audacity as a
sample audio editor as opposed a DAW.  The purpose of a audio editor
leans more towards fast as possible destructive transformation of a
small number of input tracks, wheras a DAW is more concerned with
non-destructive playback so that the user can take his time to find
the best mix between many input tracks.  The main reason that audio
editors tend to lack this feature is that they have to be very fast on
rendering the final audio file, and DAWs are not expected to be fast
(bouncing to disk on protools takes forever.)  Non-destructive
real-time effects rendering comes with the cost of having to recompute
the effect when writing the file to disk.  One alternative is to keep
the "destructive"  copy around, but the implementation for this is not
trivial - it begins to resemble the on-demand projects we have been
talking about.

If this is the case then it seems like we should prioritize the
functionality for desctructive, on-demand stacks, and then move
towards desctuctive real-time effect playback stacks.  After that
point the non-destructive implementation can be considered.  Would
other people like to chime in their thoughts on this?

Michael



2008/5/4, James Crook <crookj@...>:

> Jorge Rodriguez wrote:
>  > There have been two things I've always wanted out of Audacity that I
>  > could never get. The first one is a way to cut out a section of music
>  > while leaving the rest of it undisturbed. That was implemented
>  > recently, "Split Delete." (Should be default behavior in my opinion,
>  > deleting a section of sound shouldn't move other stuff around. But
>  > anyways!) The other one was real-time effects stacks, or the ability
>  > to apply effects non-destructively, realtime, during play. This
>  > transforms Audacity from a powerful sound editing tool into a
>  > multitrack recording studio.
>  >
>  > So, I hear that 1.3.5 is coming out and then is the time to start
>  > putting new stuff in Audacity and it's important to me that this to be
>  > one of them, so I'm pretty sure I'm willing to spare a couple of hours
>  > of my week to make sure that this feature is part of the next Audacity
>  > release.
>  >
>  > Questions. At a high level, what would be required to get this feature
>  > working? Audacity already supports applying effects, they just need to
>  > be applied during playback instead of immediately. I think you would
>  > need a UI for managing effects, and the order that they are processed
>  > in, and a mechanism for controlling the effects. The current dialogs
>  > that do this can probably be reused. And then of course, the code for
>  > applying the effects in realtime instead of destructively. Can someone
>  > elaborate on exactly what needs to happen? What I specifically want to
>  > know is even a vague estimate of how many man hours is this going to
>  > be? I've done things tangentally related to this, but not really the
>  > same thing, so who is going to help me when I invariably run into a wall?
>  >
>  > PS. CVS?
>  >
>
>
> CVS = http://en.wikipedia.org/wiki/Concurrent_Versions_System
>  GSoC = http://en.wikipedia.org/wiki/Google_summer_of_code
>
>  How many hours?  Lots and lots.  It is simply not realistic to attempt
>  this on two hours a week, with no prior experience of modifying
>  Audacity.  I'd strongly suggest you try out smaller modifications to
>  Audacity first, just to get the hang of compiling, testing, finding your
>  way around the code.  You'll get to the end goal faster.
>
>  We do have a plan for real time, which is to get there in stages, and
>  there are ways you can help us get there faster.  A first step is one of
>  the more risky GSoC projects we are doing this year.  Have a look at:
>  Michael Chinen's project on this page:
>  http://www.audacityteam.org/wiki/index.php?title=GSoC_2008_Projects
>
>  One way you could help is by understanding what is being planned and
>  why, why we are doing it this way, and when Michael starts coding trying
>  out his betas, helping to find problems early.  If you are understanding
>  his code, then you may be able to suggest fixes and improvements that
>  help progress the project - so he'll end up with a more complete
>  implementation than otherwise by the end of the project - but you'll
>  only be able to do that at all if you've already got fairly confident
>  and familiar with the Audacity code - and that takes time.  You should
>  try compiling Audacity from source and making small changes now to be
>  ready in time.
>
>  For the 'big picture', also have a look at Postfish Integration on
>  http://www.audacityteam.org/wiki/index.php?title=More_GSoC_Ideas
>
>  That's not going to happen during this GSoC, but it gives you an idea of
>  why doing real time is not a small simple change and it makes it clearer
>  why render-on-demand is a first step.  We'd certainly appreciate help
>  with getting there, but you'll need to be up to speed on developing with
>  Audacity for it to be helpful to us.  To get up to speed, pick some
>  smaller pieces/details of Audacity to work on, where you and we get
>  something we can use sooner.  It doesn't need to be in the audio-path
>  for it to help you gain experience with the code.  If you need any help
>  compiling Audacity from source just ask here.  We expect there to be
>  some problems at first, and we're well used to giving tips to help sort
>  out compile issues.
>
>
>  Good to hear from you,
>
>
>  --James.
>
>  -------------------------------------------------------------------------
>  This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
>  Don't miss this year's exciting event. There's still time to save $100.
>  Use priority code J8TL2D2.
>  http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
>  _______________________________________________
>  Audacity-devel mailing list
>  Audacity-devel@...
>  https://lists.sourceforge.net/lists/listinfo/audacity-devel
>

-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
Don't miss this year's exciting event. There's still time to save $100.
Use priority code J8TL2D2.
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
_______________________________________________
Audacity-devel mailing list
Audacity-devel@...
https://lists.sourceforge.net/lists/listinfo/audacity-devel

[Richard Ash (audacity-help)]

On Sun, 2008-05-04 at 12:13 -0400, Michael Chinen wrote:

> I have mixed feelings about non-destructive effect stacks in audio
> editors.  Non-destructive real-time effects stacking is a very useful
> and fundamental tool in a digital audio workstation like protools or
> ardour.  That being said, I have always thought of audacity as a
> sample audio editor as opposed a DAW.  The purpose of a audio editor
> leans more towards fast as possible destructive transformation of a
> small number of input tracks, wheras a DAW is more concerned with
> non-destructive playback so that the user can take his time to find
> the best mix between many input tracks.  The main reason that audio
> editors tend to lack this feature is that they have to be very fast on
> rendering the final audio file, and DAWs are not expected to be fast
> (bouncing to disk on protools takes forever.)  Non-destructive
> real-time effects rendering comes with the cost of having to recompute
> the effect when writing the file to disk.  One alternative is to keep
> the "destructive"  copy around, but the implementation for this is not
> trivial - it begins to resemble the on-demand projects we have been
> talking about.

Interesting perspective. To me the more important thing has always been
that you shouldn't need a huge, carefully tuned machine to run audacity,
which implies keeping the CPU and memory needed for playback down as far
as possible. This in turn means rendering effects in non-real time
first.

> If this is the case then it seems like we should prioritize the
> functionality for desctructive, on-demand stacks, and then move
> towards desctuctive real-time effect playback stacks.  After that
> point the non-destructive implementation can be considered.  Would
> other people like to chime in their thoughts on this?

One first step I suggested to another SoC student was to re-factor the
existing undo stack so it could be accessed on a per-track basis. This
would let you undo changes to track A without touching subsequent
changes to track B. This is I think relatively straightforward in
principal, although not trivial.

The next stage would be to ensure that all the effect parameters were
stored when an item was pushed to the undo stack, so that we gained a
per-track redo function, that would repeat the last effect on this
track, on this track. This means redo without storing any audio
information, just using the track state at the start and the processing
parameters.

That's not really useful, but facilitiates the next step, which is an
(offline) way to go back up the stack undoing effects until a point is
reached, then re-do (calculate from scratch on the new audio data)
subsequent operations to get back to where we were, but with one (or
more) effects missing out of the middle.

At about this point is becomes imperitive that the undo stacks can be
saved in the project, not deleted when you exit audacity (although this
needs to be an option, so people's projects don't grow forever).

The main strength of this is that it keeps the primary project state as
the output, not the input, so audacity exports quickly (only the final
mix to do). It also makes use of the existing undo structures, and makes
it trivial to clear away really old history to save disk space
(something that's often quite hard in fully non-destructive editors,
because all of the original files are still needed, even if they only
contribute a few seconds of heavily processed audio).

Hope this makes sense as some possible routes to getting the jobs done,
rather than just cloning how other people do the job.

Richard


-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
Don't miss this year's exciting event. There's still time to save $100.
Use priority code J8TL2D2.
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
_______________________________________________
Audacity-devel mailing list
Audacity-devel@...
https://lists.sourceforge.net/lists/listinfo/audacity-devel

[Jorge Rodriguez-7]

Thanks for all the responses.

> It is simply not realistic to attempt this on two hours a week

When I said "a couple hours" I didn't mean two, I realize this is more than a two hour a week job.

> I have mixed feelings about non-destructive effect stacks in audio
> editors.  Non-destructive real-time effects stacking is a very useful
> and fundamental tool in a digital audio workstation like protools or
> ardour.  That being said, I have always thought of audacity as a
> sample audio editor as opposed a DAW.

The main problem with Ardour is their downloads page:

http://ardour.org/binary_downloads

Where's the Windows binary? My opinion is that Ardour, while it is a good recording tool, is made by a bunch of Linux elitists who aren't willing to "pander" to Windows people. To quote Paul:

"you seem to assume that the crowd we would get from there being a windows version is a desirable thing. many other *nix open source projects have been overwhelmed when they have ported to windows - a huge, sudden influx of users with zero background in software development, and no infrastructure to offer them support. we don't want to end up in that situation."

I don't blame them honestly, X-Chat for example canceled their Windows version because of how much of a bother it was. .However, the result is that everybody I know uses and loves Audacity, and nobody knows that Ardour exists.

Anyhow, Audacity can be more of an "audio processing" tool as opposed to a DAW, but still process data non-destructively. I would think that not all users want to process their data completely destructively all the time. Maybe they would like to keep their "dry" audio data around, and maybe they would like Audacity to handle this and not have to backup and import the raw audio data themselves? I think that's reasonable.

> One first step I suggested to another SoC student was to re-factor the
> existing undo stack so it could be accessed on a per-track basis.

This would be helpful, but I don't know if it would be very intuitive to have to select which track you want to do the undo on. It doesn't make much sense, and it's not what you would expect the program to do when you hit ctl-z. Aside from that, while it would solve Mr. Chinen's problem where Audacity is more of an audio editor than a DAW, it seems like a roundabout way to accomplish this, using the undo stack to do what a completely separate stack should do. It smells of a hack to me.

But, it did give me an interesting idea, which is very similar and I think has the best of both worlds. Each track would have an effects stack, but instead of being applied to the real-time buffer or to the file itself destructively, what if a copy of the audio data was made, and the effects rendered to it immediately and destructively, and Audacity would keep both versions around? The "dry" version of the audio data would be kept around for if the user turns effects off or if Audacity needs to re-render the effects stack, and the "wet" version of the audio data is kept around playback and exporting?

I like this solution because it involves no real-time effects rendering, it allows the exporting to happen exactly as fast as it happens right now, and it does not significantly increase the requirements on hardware, other than the extra hard drive space needed to store two versions of each track with effects. What do you guys think?

Finally, I should let you know that I'm not a Summer of Code student, I'm a professional software developer who is also a recording musician, and thus I'm not eligible to do SoC. Also, I was expressing my WTF on your continued usage of CVS when SVN has been out for four years. In any case, I'll pull Audacity out of CVS soon (ick) and start mucking around in the source code.

PS: Where's the bug tracker? I want to look up why muted tracks aren't saved to the project file or recognized during export.

--
Jorge Rodriguez
Matrieya Studios, LLC
Email: jrodriguez@...
Phone: [phone]
-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
Don't miss this year's exciting event. There's still time to save $100.
Use priority code J8TL2D2.
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
_______________________________________________
Audacity-devel mailing list
Audacity-devel@...
https://lists.sourceforge.net/lists/listinfo/audacity-devel

[Michael Chinen]

2008/5/5, Jorge Rodriguez <jrodriguez@...>:

> But, it did give me an interesting idea, which is very similar and I think
> has the best of both worlds. Each track would have an effects stack, but
> instead of being applied to the real-time buffer or to the file itself
> destructively, what if a copy of the audio data was made, and the effects
> rendered to it immediately and destructively, and Audacity would keep both
> versions around? The "dry" version of the audio data would be kept around
> for if the user turns effects off or if Audacity needs to re-render the
> effects stack, and the "wet" version of the audio data is kept around
> playback and exporting?
>
> I like this solution because it involves no real-time effects rendering, it
> allows the exporting to happen exactly as fast as it happens right now, and
> it does not significantly increase the requirements on hardware, other than
> the extra hard drive space needed to store two versions of each track with
> effects. What do you guys think?

This is essentially what I meant by destructive on-demand effect
stacks.  I should exlain the on-demand part - it was proposed to get
the best of both worlds, some other things might be considered.  By
On-Demand, I mean that when the user clicks on the 50% mark and hits
play, the effect does not process the sound from start to end, but
instead jumps to the 50% mark and starts processing there immediately
so the user can hear the effect instantly.   Currently effects are
processed from the start to end in linear progression.

As a starting point I am implementing this for peak summary data,
which is used to view the waveform.  This is not an effect, but the
behavior and structure is similar and we are thinking to integrate it
with this system.

Note the naive implementation of this will not work for all audacity
effects.  One example is reverse or repeat.  These effects use
information from a different part of the waveform to compute an
arbitrary chunk of sound, and the on-demand stack just won't make
sense for those.  In most DAWs, these effects are simply not allowed
as being capable of having stack behavior.  However, I don't think
Audacity currently makes a distinction between those, so its something
to consider.

>
> Finally, I should let you know that I'm not a Summer of Code student, I'm a
> professional software developer who is also a recording musician, and thus
> I'm not eligible to do SoC. Also, I was expressing my WTF on your continued
> usage of CVS when SVN has been out for four years. In any case, I'll pull
> Audacity out of CVS soon (ick) and start mucking around in the source code.
>
> PS: Where's the bug tracker? I want to look up why muted tracks aren't saved
> to the project file or recognized during export.
>
>
> --
> Jorge Rodriguez
> Matrieya Studios, LLC
> Email: jrodriguez@...
>  Phone: [phone]
> -------------------------------------------------------------------------
>  This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
>  Don't miss this year's exciting event. There's still time to save $100.
>  Use priority code J8TL2D2.
> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
> _______________________________________________
>  Audacity-devel mailing list
>  Audacity-devel@...
> https://lists.sourceforge.net/lists/listinfo/audacity-devel
>
>

-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
Don't miss this year's exciting event. There's still time to save $100.
Use priority code J8TL2D2.
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
_______________________________________________
Audacity-devel mailing list
Audacity-devel@...
https://lists.sourceforge.net/lists/listinfo/audacity-devel

[Jorge Rodriguez-7]

Oh sorry, I guess I didn't get the on-demand part. I think I understand better now, however I'm not sure we're talking about the same thing so I'm a bit confused.

I think we are trying to solve different problems, and so we're talking on different wavelengths here. What I think you're talking about is, in the current version of Audacity, when the user selects an effect it is applied immediately and destructively, but you want it to be only remembered and applied "on-demand" the next time the user clicks play, only for the section of the song that the user is playing, in order to save up-front processing time. That's okay, but even if it does it in the context of an effects stack it's still not what I really want to implement, which is non-destructive effects. The on-demand behavior is not really all that desirable to me, I don't mind that the program takes a moment to render the effect that I just selected, so long as I can go and change it to listen to it again without having to worry about the fact that all my data was just overwritten. On-demand processing does sound like it would be a good idea, and it would reduce the amount of processing required (or at least spread it out more) but I'd like to focus on non-destructive effects stacks implemented in the way I outlined in my previous email. Here's how I imagine it might work, from a work-flow perspective:

1) The user adds an effect or two to his stack.
2) Audacity renders these effects to a separate .au audio file, keeping the original pristine.
3) The user hits play, and Audacity plays the version of the audio with the effects applied.
4) The user changes a parameter on the first effect and adds a third effect.
5) Audacity re-renders the track from the original pristine audio track.
6) The user hits play and hears his tweaks.

From a UI perspective, the rendering can happen at any time, be it on-demand, or at the push of a button in the track's control panel, or when the effects window screen is closed, or whenever. I'm trying to meet a design requirement that I gleaned from your first email, that the goal of this design is not to increase the processor requirements when adding these effects. What I'm looking for is input on whether or not this is a good design for how to implement this feature.

--
Jorge Rodriguez
Matrieya Studios, LLC
Email: jrodriguez@...
Phone: [phone]
-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
Don't miss this year's exciting event. There's still time to save $100.
Use priority code J8TL2D2.
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
_______________________________________________
Audacity-devel mailing list
Audacity-devel@...
https://lists.sourceforge.net/lists/listinfo/audacity-devel

[L.R.N]

Jorge Rodriguez wrote:
> 1) The user adds an effect or two to his stack.
> 2) Audacity renders these effects to a separate .au audio file,
> keeping the original pristine.
> 3) The user hits play, and Audacity plays the version of the audio
> with the effects applied.
> 4) The user changes a parameter on the first effect and adds a third
> effect.
> 5) Audacity re-renders the track from the original pristine audio track.
> 6) The user hits play and hears his tweaks.
My thoughts:
1) Implement real-time effects stacks. One stack per track, each effect
has one additional parameter (besides it's normal parameters) - the area
where it is applied, so you can apply different effects to different
parts of the track.
2) When user makes a change in the stack, Audacity begins background
audio-recomputation process immediately (this may be tweaked with an
option), simultaneously for each changed stack.
3) When user selects some part of the project (or places the cursor
somewhere) and clicks "play" the relevant part (selected part, or part
after the cursor) is marked as "top priority" and Audacity stars
processing it right away.
4) While Audacity is preparing this part, user must wait (with or
without modal dialog asking him to wait, maybe we could just add small
"work in progress" indicator on one of toolbars, without popping-up a
dialog, like browsers do when they are loading a page).
5) Once some (exact amount could be tweaked in options) part of relevant
audio is ready (pre-buffered), Audacity starts playback.
6) If Audacity estimates that it can't render audio fast enough to keep
up with playback, it notices user about it (some indicator somewhere on
some toolbar flashing red, or something like that). Playback will pause
once in a while to wait until next part of audio is ready.
7) User can always see the progress of rendering (parts of the tracks
being colored in different color, or maybe additional bar that shows
which parts of a track are processed). If user changed the stack, track
(or part of it) is invalidated and revets to "unprocessed" state (and
user can see that).
8) Anything that was rendered is buffered (on HDD). If user listened
through the whole project and didn't changed the stacks, this buffer
will contain current project's output (as if it was pre-computed
linearly, just like it is done in current version). Separate for each
track. If user later changes the stack for one track, only this track is
recomputed (and maybe only a part of this track, if effect wasn't
applied to the whole track).

Original is kept intact.
As for "reverse" and some other effects...Audacity (i mean - the
developers ;) ) has to be clever. For example, when making a reverse, it
takes audio from different part of the track. This only makes the
"reverse" effect more difficult to implement, but not impossible. For
echo effect Audaicy should start processing before the relevant part
(how long before - depends on echo parameters). Etc. It's just the
matter of implementing effects in proper way.


-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
Don't miss this year's exciting event. There's still time to save $100.
Use priority code J8TL2D2.
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
_______________________________________________
Audacity-devel mailing list
Audacity-devel@...
https://lists.sourceforge.net/lists/listinfo/audacity-devel

[James Crook]

Jorge Rodriguez wrote:
> Where's the Windows binary? My opinion is that Ardour, while it is a
> good recording tool, is made by a bunch of Linux elitists who aren't
> willing to "pander" to Windows people.
I don't see it as elitism, more a sign of an already overstretched team
- as we are too, but that's another story.  Ardour have in fact built
experimental Windows versions.  There is an experimental version of Jack
that runs on Windows.  My ideal is that in time we would start using
Jack too, finding ways to make it reliable on Windows.  If we get it
right we and Ardour can then start plundering each others code.  
Certainly Paul isn't elitist about that - and is in favour of that
happening, but he sees it as being probably too far in the future to be
relevant at this stage.  Real time for Audacity would bring that day
closer.  In the longer term our two projects need to be 'sharing
technology' more.

> But, it did give me an interesting idea, which is very similar and I
> think has the best of both worlds. Each track would have an effects
> stack, but instead of being applied to the real-time buffer or to the
> file itself destructively, what if a copy of the audio data was made,
> and the effects rendered to it immediately and destructively, and
> Audacity would keep both versions around? The "dry" version of the
> audio data would be kept around for if the user turns effects off or
> if Audacity needs to re-render the effects stack, and the "wet"
> version of the audio data is kept around playback and exporting?
The main distinction between this and the actual plan is having a
per-track effects stack rather than an overall one.  Our plan is to
achieve the SAME user experience by tagging the single overall stack.  
That the overall effects stack is our undo-stack is 'just' an
implementation detail that helps us get there in stages faster.

> I like this solution because it involves no real-time effects
> rendering, it allows the exporting to happen exactly as fast as it
> happens right now, and it does not significantly increase the
> requirements on hardware, other than the extra hard drive space needed
> to store two versions of each track with effects. What do you guys think?
We already have that information around for 'undo' so it wouldn't
increase the hard drive space over what we use now.  We're storing not
only the dry and wet end-result, but also all intermediate results too.  
By design, audio that is moved around is very cheap because we point to
blockfiles rather than repeat them.  "What do you guys think?", yeah
sure, that's where we want to get to.

> Finally, I should let you know that I'm not a Summer of Code student,
> I'm a professional software developer who is also a recording
> musician, and thus I'm not eligible to do SoC. Also, I was expressing
> my WTF on your continued usage of CVS when SVN has been out for four
> years. In any case, I'll pull Audacity out of CVS soon (ick) and start
> mucking around in the source code.
I read the 'CVS?' as 'what's that?' - meaning that you probably also
wouldn't know what I meant by 'GSoC' in the next sentence either ! , so
I might as well get both definitions out the way quickly and easily with
links rather than spell it out.  We considered moving from CVS to SVN
for GSoC, but decided against it.  That's mostly because we may be
moving off sourceforge altogether after GSoC.  Moving twice is avoidable
work.


--James.

-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
Don't miss this year's exciting event. There's still time to save $100.
Use priority code J8TL2D2.
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
_______________________________________________
Audacity-devel mailing list
Audacity-devel@...
https://lists.sourceforge.net/lists/listinfo/audacity-devel

[Michael Chinen]

Nice thoughts, LRN.  I think you pretty much summed up the consensus.

However, for reverse effects, as well as effects that greatly alter
the length or continuity of the waveform, I have a different position.
 I agree that it is possible to reverse in stack mode, and the problem
is cool to think about, certainly.

However, I don't think it makes sense to have those type of effects on
as a stack structure for UI purposes.  This is why DAWs have effects
that can be chained and stacked, and effects that can't - not because
it is harder to implement, but because the user expectation of these
stacked effects are "guitar pedal"-like effects that do something with
the waveform directly beneath wherever the cursor is.  Effects that
make significant changes to continuity/structure can break this
expectation.



2008/5/5, LRN <lrn1986@...>:

>  As for "reverse" and some other effects...Audacity (i mean - the
>  developers ;) ) has to be clever. For example, when making a reverse, it
>  takes audio from different part of the track. This only makes the
>  "reverse" effect more difficult to implement, but not impossible. For
>  echo effect Audaicy should start processing before the relevant part
>  (how long before - depends on echo parameters). Etc. It's just the
>  matter of implementing effects in proper way.
>
>
>
>  -------------------------------------------------------------------------
>  This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
>  Don't miss this year's exciting event. There's still time to save $100.
>  Use priority code J8TL2D2.
>  http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
>  _______________________________________________
>  Audacity-devel mailing list
>  Audacity-devel@...
>  https://lists.sourceforge.net/lists/listinfo/audacity-devel
>

-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
Don't miss this year's exciting event. There's still time to save $100.
Use priority code J8TL2D2.
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
_______________________________________________
Audacity-devel mailing list
Audacity-devel@...
https://lists.sourceforge.net/lists/listinfo/audacity-devel

[Jorge Rodriguez-7]

I also agree with LRN's eight bullet points. I honestly can't really find any way to improve them, and if nobody else disagrees, we could use them as a basis for moving forward. However, that looks like a whole lot of work, and I think we need to break the task into chunks that can be tackled one at a time. I'm going to take some time going over the code and doing some research on how I can get this started.

--
Jorge Rodriguez
Matrieya Studios, LLC
Email: jrodriguez@...
Phone: [phone]
-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
Don't miss this year's exciting event. There's still time to save $100.
Use priority code J8TL2D2.
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
_______________________________________________
Audacity-devel mailing list
Audacity-devel@...
https://lists.sourceforge.net/lists/listinfo/audacity-devel

[Lars Luthman-3]

On Mon, 2008-05-05 at 18:52 -0400, Michael Chinen wrote:

> Nice thoughts, LRN.  I think you pretty much summed up the consensus.
>
> However, for reverse effects, as well as effects that greatly alter
> the length or continuity of the waveform, I have a different position.
>  I agree that it is possible to reverse in stack mode, and the problem
> is cool to think about, certainly.
>
> However, I don't think it makes sense to have those type of effects on
> as a stack structure for UI purposes.  This is why DAWs have effects
> that can be chained and stacked, and effects that can't - not because
> it is harder to implement, but because the user expectation of these
> stacked effects are "guitar pedal"-like effects that do something with
> the waveform directly beneath wherever the cursor is.  Effects that
> make significant changes to continuity/structure can break this
> expectation.

You can have two separate stacks, for realtime and non-realtime effects,
and view the non-realtime effects as an explicit undo stack. It could
also include things like copy and paste commands, manual sample point
tweaks etc. I've been thinking about writing a sampler like this (and
got started on it, but other things got in the way).


--ll


-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
Don't miss this year's exciting event. There's still time to save $100.
Use priority code J8TL2D2.
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
_______________________________________________
Audacity-devel mailing list
Audacity-devel@...
https://lists.sourceforge.net/lists/listinfo/audacity-devel

[Richard Ash (audacity-help)]

On Mon, 2008-05-05 at 22:14 -0400, Jorge Rodriguez wrote:
> I also agree with LRN's eight bullet points. I honestly can't really
> find any way to improve them, and if nobody else disagrees, we could
> use them as a basis for moving forward. However, that looks like a
> whole lot of work, and I think we need to break the task into chunks
> that can be tackled one at a time.

That was the point of what I said about deriving from the existing undo
architecture (at an internal level - how it is interfaced is a separate
issue, and certainly CTRL+Z should always undo the last item, but it
doesn't mean there aren't other operations you can do which turn out to
use the undo stack underneath).

There is already a lot of very complex, very well written code
underlying the audacity project system which enables it not only to
perform edits quickly but maintain a loss-less, reference-counting undo
system that is disk-efficient. This structure is capable of supporting
all the operations that have been talked about without major changes.
The render changes will alter how some code access the project
structure, but fundamentally the whole system remains a large,
tree-based, copy-on-write, disk backed data buffer in which all the
audio data is kept. The missing feature is out-of-order stack access,
which I believe to be a matter of implementation rather than design - it
could be added relatively easily now a demand exists. Ditto an option
not to discard history when a project is closed - this is useful for
most users, but not in any way inherent in the design.

What I'm trying to avoid is replicating the way other software works
just because that's the way other software works, not because it's
actually a good solution. For most audio tasks the most likely changes
to want to undo or alter are those made most recently. This means that
any version control system (which is essentially what we are aiming at
here - version management within an audacity project) should store the
most recent changes efficiently at the expense of older changes.

The approach taken by other software of "master recordings are the
ultimate, everything else is secondary" is a hang-over from analogue,
tape-based systems where generation loss was a major concern. Audacity
is not some kind of digital tape recorder. It's a non-linear, random
access edit system from day one, and the design reflects that.

So to make effect layering possible, we leverage what is already there,
not try and splice on a design from somewhere else over the top. The
interface may hide the design completely - for the sake of migrating
users we may want to emulate a linear, pseudo-tape environment in the
user interface, but that shouldn't dominate how we design the data
structures to deliver it.

We also shouldn't add obstacles that don't need to exist. Why should a
user have to understand which effects are deemed "linear" enough to be
added to stacks and which aren't? Where do you draw the line between 2%
speed correction on a track from tape, and a double-speed guitar solo?
Technically none of the audacity effects except invert and amplify can
be applied to a sample in isolation - every one requires a buffer of
samples either side to calculate it effectively. This is why effects are
applied in a linear manner at the moment. Without the constraint of
rendering on the fly, the linear / non-linear / latency battle is
meaningless, so we shouldn't try to re-introduce it just because other
people's architectural models can't cope without it.

A large amount of software design is plagued by a continual design to
clone other software not develop something new. This is at it's worst
when the people doing the design have had long careers in the field -
they design what they know how to use, not what is actually logical, or
easy to use if you are starting from scratch. I don't want audacity to
become another DAW, mainly because I find the psuedo-tapedeck approach
inflexible and unhelpful for what I do. Most of the time, that isn't
creating linear tonal transformations of musical performances - it's
highly non-linear re-working of mixed recordings. Frequently my workflow
looks more like a modern video edit session than a traditional audio
session, yet I'm doing essentially the same thing - transforming locally
time-linear media. Why does a "professional" audio package look so
different to the sound section of a "professional" video edit package? I
see it mostly as an accident of the people who design these systems and
what they were brought up on - open reel magnetic tape in one case,
storyboards fed into tape-to-tape edit controllers in the other.
Thankfully, I've never had to use either in anger, and probably never
will do. I'd prefer my software tools not to ape them either.

Sorry about the rant, but I've spent weeks reading proposals from people
with no idea how audacity works internally who want to make more drastic
changes in 3 months than the entire project has achieved in 3 years.
Along the way a fair number have given the impression that they not only
don't know how audacity works now, but don't care, and have no intention
of trying to understand or fit in, we should throw it out and follow
their master plan. One of the problems with open source is that code
usually precedes design, which can prevent coherent thinking from
emerging.

Richard


-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
Don't miss this year's exciting event. There's still time to save $100.
Use priority code J8TL2D2.
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
_______________________________________________
Audacity-devel mailing list
Audacity-devel@...
https://lists.sourceforge.net/lists/listinfo/audacity-devel

[Jorge Rodriguez-7]

I understand, you want to use the stack functionality that supports the undo feature to implement effects stacks, but not actually present that to the user as an "undo stack for effects" but rather as just a regular effects stack which stores every iteration of effect from bottom to top. That makes a bit more sense than what I thought you were wanting to do, but I still disagree that it's the proper way to go.

I have issue with a lot of this, for example the talk of getting rid of the older parts of the stack. What? No! Moreover, storing every layer of the stack on the hard disk is in my opinion rather pointless when it's so cheap to reprocess, and the occasion of the user actually going in and listening to the effects stack with only the top ones muted is rare.

Let me give an example. Here's my track, with the effects stack on top:

Reverb
Compression
Echo
BassBoost
Noise removal
Click removal
Audio

So, you're saying that after sticking the Reverb on there, the user will only ever edit the Reverb and Compression, and sometimes the Echo, and so Audacity should forget about the original track and only keep BassBoost and up? Seriously? What if I want to go back and tweak the noise removal? What if I forgot to remove a click? The whole point of non-destructive editing is that it never forgets about the original unless I want it to. If I want to, I should always be able to go in and put an LFO before the BassBoost. On the other hand, I don't think that I'll ever want to undo the click removal, right? So I should be able to render that destructively if I want. And, if I'm the type of person who is reckless and carefree and I'm satisfied with the stack and I don't care about keeping the original track, I should be able to render part or all of stack destructively.

The other thing I'm worried about is, why would you keep a different audio file for every single point in the stack? How often are users going to be listening to just the four bottom items but not the top three? In my opinion, almost never. Moreover, since using a stack to store every step mean that you can't see what it sounds like with BassBoost Echo and Reverb but no Compression without re-rendering, you're in the same place as if you had never used a stack to cache all this audio.

Why not just store the original track, and then store the rendered version, and be done with it? It's much easier to subclass Sequence and do some special logic in WaveTrack than gutting the undo stack code to support such a rare use case. Or... am I understanding you wrong? Maybe it will help if I add a requirements to LRN's list:

9) The ability to render part or all of the effects stack destructively onto the original.

You know, essentially the only difference between your propos