How to introduce delay for some requests?

[Toni Lamar]

I would like to introduce delay for some of the requests - for example: some of the public pages that have a feedback form.

If the request is from some bot (especially spamer bot) than I would like to make it appear as it takes just much more time time, compared to the "normal" processing speed of forms from requests coming from browsers.

A CAPTCHA is just a too ugly and a too big problem for the users, especially when it comes to feedback forms, so the delay is the only remaining solution.

Is this possible with Click?

Thnx.

[lhazlewood]

What do you want to take "too long"?  The loading of the feedback form page?  Or the submission of the page? 

Can't you use Thread.sleep to do this?  I'm not sure what the exactly requirements are or where the delay should occur, or that it is a good idea to begin with ;)

On Thu, Jul 24, 2008 at 12:38 PM, Toni Lamar <toni.lamar@...> wrote:

I would like to introduce delay for some of the requests - for example: some
of the public pages that have a feedback form.

If the request is from some bot (especially spamer bot) than I would like to
make it appear as it takes just much more time time, compared to the
"normal" processing speed of forms from requests coming from browsers.

A CAPTCHA is just a too ugly and a too big problem for the users, especially
when it comes to feedback forms, so the delay is the only remaining
solution.

Is this possible with Click?

Thnx.

--
View this message in context: http://www.nabble.com/How-to-introduce-delay-for-some-requests--tp18635818p18635818.html
Sent from the click-user mailing list archive at Nabble.com.


-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Click-user mailing list
Click-user@...
https://lists.sourceforge.net/lists/listinfo/click-user

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Click-user mailing list
Click-user@...
https://lists.sourceforge.net/lists/listinfo/click-user

[Toni Lamar]

lhazlewood wrote:

What do you want to take "too long"?  The loading of the feedback form
page?  Or the submission of the page?

I don't know. I suppose the last part.
I saw this strategy used by some PHP + Apache sites to reduce the number of spam bots.
Since it's not possible to know 100% if a request comes from a bot, resetting the connection is not an option. Delays seem to discourrage mass spamming, but detecting the bot should deliver as few false positives as possible.

lhazlewood wrote:

Can't you use Thread.sleep to do this?  

I believed that using Tread.sleep in Servlets (Click servlet in this case) is not reccomanded because
unknown side effects - it's not always only one thread - sometimes are pooled, sometimes are more, if commet is used than on thread is delivering to more requests, etc.

lhazlewood wrote:

I'm not sure what the exactly
requirements are or where the delay should occur, or that it is a good idea
to begin with ;)

As usual, there are no "exact requirements" :):
Just to have less spam to interfere with public forms(mostly feedback like) from customers :).

Thnx.

[sabob]

I've noticed a few sites does away with capcha's and instead ask
context related questions such as "What is the second word of the
third paragraph".

Toni Lamar wrote:
> I believed that using Tread.sleep in Servlets (Click servlet in this case)
> is not reccomanded because
> unknown side effects - it's not always only one thread - sometimes are
> pooled, sometimes are more, if commet is used than on thread is delivering
> to more requests, etc.

Why not use a filter to slow things down? You should be safe as long
as you don't sleep inside a synchronized block because then the entire
application will go to bed :)

You can also built checks which places a threshold on the amount
requests coming from specific IP addresses.

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Click-user mailing list
Click-user@...
https://lists.sourceforge.net/lists/listinfo/click-user

[lhazlewood]

On Thu, Jul 24, 2008 at 1:12 PM, Toni Lamar <toni.lamar@...> wrote:

lhazlewood wrote:

>
> Can't you use Thread.sleep to do this?
>

I believed that using Tread.sleep in Servlets (Click servlet in this case)
is not reccomanded because
unknown side effects - it's not always only one thread - sometimes are
pooled, sometimes are more, if commet is used than on thread is delivering
to more requests, etc.

You don't have to worry about it:  a request has a 1:1 correspondence with a thread.  So does a JTA transaction (by specification).  Yes, threads are almost always pooled, but this doesn't matter - the thread is only out of the pool when it is processing a request, in which case it won't be affected by any other request. 

If the thread processing that request does Thread.sleep, it just means it takes longer for that request to complete and subsequently for that thread to go back in the pool.  No other requests or threads are affected by it.

Cheers,

Les

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Click-user mailing list
Click-user@...
https://lists.sourceforge.net/lists/listinfo/click-user

[Toni Lamar]

sabob wrote:

I've noticed a few sites does away with capcha's and instead ask
context related questions such as "What is the second word of the
third paragraph".

This is not acceptable for the customers :(.
The forms already ask too many questions, and the patience of the users is low (to the "leaving the page" level - judging by the stats with hidden javascript), so evey suplemental questions is not allowed :(.

sabob wrote:

> I believed that using Tread.sleep in Servlets (Click servlet in this case)
> is not reccomanded because
> unknown side effects - it's not always only one thread - sometimes are
> pooled, sometimes are more, if commet is used than on thread is delivering
> to more requests, etc.

Why not use a filter to slow things down?

Well, I haven't found a "smart" filter that does this (for PHP and Rails there are already ready to use solutions). Besides, to decide the probability if a request comes from a bot or not, some more data is required -> in Click is ready bound and easier to check, so I thought maybe there's a simple a way to do it
in Click, e.g. some trick in "onRender()" ?

sabob wrote:

You can also built checks which places a threshold on the amount
requests coming from specific IP addresses.

This makes sense only combined with a blacklist, since there are many proxies e.g. form companies, or even some providers with too few IP addresses. In these cases the "treshold" is quickly "consumed", and the users get very frustrated.

Since Click is addresed to commercial projects, it would be nice if there were some "read to use" solutions (or at least best practices) for such problems.

Thnx.

[Ozakca, Muzaffer]

Would a spam filter - like Akismet - work for you? Or maybe you can store the last time the comment form is posted in the session object and not let another one posted until some defined time passes. I'm sure it's not foolproof though.

Muzaffer

> -----Original Message-----
> From: click-user-bounces@... [mailto:click-user-
> bounces@...] On Behalf Of Toni Lamar
> Sent: Thursday, July 24, 2008 2:41 PM
> To: click-user@...
> Subject: Re: [Click-user] How to introduce delay for some requests?
>
>
>
> sabob wrote:
> >
> > I've noticed a few sites does away with capcha's and instead ask
> > context related questions such as "What is the second word of the
> > third paragraph".
> >
> This is not acceptable for the customers :(.
> The forms already ask too many questions, and the patience of the users is
> low (to the "leaving the page" level - judging by the stats with hidden
> javascript), so evey suplemental questions is not allowed :(.
>
>
>
> sabob wrote:
> >
> >> I believed that using Tread.sleep in Servlets (Click servlet in this
> >> case)
> >> is not reccomanded because
> >> unknown side effects - it's not always only one thread - sometimes are
> >> pooled, sometimes are more, if commet is used than on thread is
> >> delivering
> >> to more requests, etc.
> >
> > Why not use a filter to slow things down?
> >
> Well, I haven't found a "smart" filter that does this (for PHP and Rails
> there are already ready to use solutions). Besides, to decide the
> probability if a request comes from a bot or not, some more data is required
> -> in Click is ready bound and easier to check, so I thought maybe there's a
> simple a way to do it
> in Click, e.g. some trick in "onRender()" ?
>
>
> sabob wrote:
> >
> > You can also built checks which places a threshold on the amount
> > requests coming from specific IP addresses.
> >
> This makes sense only combined with a blacklist, since there are many
> proxies e.g. form companies, or even some providers with too few IP
> addresses. In these cases the "treshold" is quickly "consumed", and the
> users get very frustrated.
>
> Since Click is addresed to commercial projects, it would be nice if there
> were some "read to use" solutions (or at least best practices) for such
> problems.
>
> Thnx.
>
>
> --
> View this message in context: http://www.nabble.com/How-to-introduce-delay-
> for-some-requests--tp18635818p18638007.html
> Sent from the click-user mailing list archive at Nabble.com.
>
>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great
> prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> Click-user mailing list
> Click-user@...
> https://lists.sourceforge.net/lists/listinfo/click-user

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Click-user mailing list
Click-user@...
https://lists.sourceforge.net/lists/listinfo/click-user

[lhazlewood]

Per my last email, unless you can find a better solution or one is suggested, there is nothing wrong with using Thread.sleep to achieve this.  Give it a try and see if it makes your customers happy enough ;)

On Thu, Jul 24, 2008 at 2:40 PM, Toni Lamar <toni.lamar@...> wrote:

sabob wrote:
>
> I've noticed a few sites does away with capcha's and instead ask
> context related questions such as "What is the second word of the
> third paragraph".
>

This is not acceptable for the customers :(.
The forms already ask too many questions, and the patience of the users is
low (to the "leaving the page" level - judging by the stats with hidden
javascript), so evey suplemental questions is not allowed :(.

sabob wrote:
>
>> I believed that using Tread.sleep in Servlets (Click servlet in this
>> case)
>> is not reccomanded because
>> unknown side effects - it's not always only one thread - sometimes are
>> pooled, sometimes are more, if commet is used than on thread is
>> delivering
>> to more requests, etc.
>
> Why not use a filter to slow things down?
>

Well, I haven't found a "smart" filter that does this (for PHP and Rails
there are already ready to use solutions). Besides, to decide the
probability if a request comes from a bot or not, some more data is required
-> in Click is ready bound and easier to check, so I thought maybe there's a
simple a way to do it
in Click, e.g. some trick in "onRender()" ?

sabob wrote:
>
> You can also built checks which places a threshold on the amount
> requests coming from specific IP addresses.
>

This makes sense only combined with a blacklist, since there are many
proxies e.g. form companies, or even some providers with too few IP
addresses. In these cases the "treshold" is quickly "consumed", and the
users get very frustrated.

Since Click is addresed to commercial projects, it would be nice if there
were some "read to use" solutions (or at least best practices) for such
problems.

Thnx.


--
View this message in context: http://www.nabble.com/How-to-introduce-delay-for-some-requests--tp18635818p18638007.html

Sent from the click-user mailing list archive at Nabble.com.


-------------------------------------------------------------------------

This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Click-user mailing list
Click-user@...
https://lists.sourceforge.net/lists/listinfo/click-user

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Click-user mailing list
Click-user@...
https://lists.sourceforge.net/lists/listinfo/click-user

[Toni Lamar]

Ozakca, Muzaffer wrote:

Would a spam filter - like Akismet - work for you?

Of course not.
First the cost of that service is 10 times more than the cost of the "final service" sold the the customer :).
Also, with all "third party" solutions(Like aksimet) there's a big big problem:
from the FAQ:
--------
How does it work?
When a new comment, trackback, or pingback comes to your blog it is submitted to the Akismet web service ...
--------
No customer would ever agree to send it's complete data it collects from it's users to some unknonw 3rd party, just to filter the spam :).

Ozakca, Muzaffer wrote:

Or maybe you can store the last time the comment form is posted in the session object and not let another one posted until some defined time passes. I'm sure it's not foolproof though.

Most bots reopen a new connection (thus a new session) for each submit (at least this is what all the logs show): in fact I haven't seen one that does not :).

Any ideas for other strategies?

Thnx.

[sabob]

Toni Lamar wrote:
> This is not acceptable for the customers :(.
> The forms already ask too many questions, and the patience of the users is
> low (to the "leaving the page" level - judging by the stats with hidden
> javascript), so evey suplemental questions is not allowed :(.


Makes sense.


> Well, I haven't found a "smart" filter that does this (for PHP and Rails
> there are already ready to use solutions). Besides, to decide the
> probability if a request comes from a bot or not, some more data is required
> -> in Click is ready bound and easier to check, so I thought maybe there's a
> simple a way to do it
> in Click, e.g. some trick in "onRender()" ?

You could create a Page which sleeps a bit in its onInit method for
example. Then extend from this Page when you need to.

>
> Since Click is addresed to commercial projects, it would be nice if there
> were some "read to use" solutions (or at least best practices) for such
> problems.

Agreed that would be nice.

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Click-user mailing list
Click-user@...
https://lists.sourceforge.net/lists/listinfo/click-user

[Ozakca, Muzaffer]

> -----Original Message-----
> From: click-user-bounces@... [mailto:click-user-
>
> Ozakca, Muzaffer wrote:
> >
> > Would a spam filter - like Akismet - work for you?
> >
> Of course not.


Why "of course not"? It was meant to be an example of such spam filter software (note the "like" in there).
I'm in no way affiliated with any company that produces spam filters.
I was trying to help :)

> First the cost of that service is 10 times more than the cost of the "final
> service" sold the the customer :).
> Also, with all "third party" solutions(Like aksimet) there's a big big
> problem:
> from the FAQ:
> --------
> How does it work?
> When a new comment, trackback, or pingback comes to your blog it is
> submitted to the Akismet web service ...
> --------
> No customer would ever agree to send it's complete data it collects from
> it's users to some unknonw 3rd party, just to filter the spam :).
>
>
>
> Ozakca, Muzaffer wrote:
> >
> > Or maybe you can store the last time the comment form is posted in the
> > session object and not let another one posted until some defined time
> > passes. I'm sure it's not foolproof though.
> >
> Most bots reopen a new connection (thus a new session) for each submit (at
> least this is what all the logs show): in fact I haven't seen one that does
> not :).
>
> Any ideas for other strategies?
>
> Thnx.
> --
> View this message in context: http://www.nabble.com/How-to-introduce-delay-
> for-some-requests--tp18635818p18638565.html
> Sent from the click-user mailing list archive at Nabble.com.
>
>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great
> prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> Click-user mailing list
> Click-user@...
> https://lists.sourceforge.net/lists/listinfo/click-user

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Click-user mailing list
Click-user@...
https://lists.sourceforge.net/lists/listinfo/click-user

[sabob]

Ozakca, Muzaffer wrote:

> Why "of course not"? It was meant to be an example of such spam filter software (note the "like" in there).
> I'm in no way affiliated with any company that produces spam filters.
> I was trying to help :)


I did a cursory search and cannot find much info on open source
implementation. Surely they must exist!

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Click-user mailing list
Click-user@...
https://lists.sourceforge.net/lists/listinfo/click-user

[Toni Lamar]

Ozakca, Muzaffer wrote:

> Ozakca, Muzaffer wrote:
> >
> > Would a spam filter - like Akismet - work for you?
> >
> Of course not.

Why "of course not"? It was meant to be an example of such spam filter software (note the "like" in there).
I'm in no way affiliated with any company that produces spam filters.

Sorry for responding like in a computer program :).
I ment, because of #1 and mostly #2, that webservice it is not an alternative for us in this case, but from a general technical point of view this solution is interesting because a big shared blacklist is much more efficient than many small distributed ones.

But to calm the fears of customers, it would be even more useful if e.g. the most of processing would take place at the customers (using a downloaded software, not a webservice), and on the server (like the Akismet), only the hashed blacklists would be placed (something like virus signatures from antivirus software), so that no visible customer/user data to be sent to third parties.

Ozakca, Muzaffer wrote:

I was trying to help :)

Your help is very appreciated :).
My way of expressing in a natural language(not a computer one :) ) is is just not very helpful to make others undertand what I wanted to express :).

Thnx.

[Toni Lamar]

sabob wrote:

I did a cursory search and cannot find much info on open source
implementation.

I also did some searches for an open source alternative, but really haven't found one :(.

sabob wrote:

Surely they must exist!

A while ago I also thought that for every commercial solution there must be some good open source alternatives. However, when trying to use the open source solutions in commercial projects (and getting to work with the dirty details :) ), it showed me that in just too many cases that was not really an alternative :(.

Thnx.

[sabob]

Two interesting articles mention some techniques of blocking spam bots:

http://green-beast.com/blog/?p=220
http://www.webaim.org/blog/spam_free_accessible_forms/

I especially like the "honeycomb spam trap" from the first article.
Seems easy to create a custom field which if filled in does not
validate...


Toni Lamar wrote:

>
> sabob wrote:
>> I did a cursory search and cannot find much info on open source
>> implementation.
>>
> I also did some searches for an open source alternative, but really haven't
> found one :(.
>
>
> sabob wrote:
>> Surely they must exist!
>>
> A while ago I also thought that for every commercial solution there must be
> some good open source alternatives. However, when trying to use the open
> source solutions in commercial projects (and getting to work with the dirty
> details :) ), it showed me that in just too many cases that was not really
> an alternative :(.
>
> Thnx.

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Click-user mailing list
Click-user@...
https://lists.sourceforge.net/lists/listinfo/click-user

[Toni Lamar]

sabob wrote:

Two interesting articles mention some techniques of blocking spam bots:

http://green-beast.com/blog/?p=220
http://www.webaim.org/blog/spam_free_accessible_forms/

Thank you for the links :).

sabob wrote:

I especially like the "honeycomb spam trap" from the first article.
Seems easy to create a custom field which if filled in does not
validate...

The technique is really nice, but sadly the assumption:
----------
Since ‘bots will fill everything and anything to circumvent any required field hiccup, they’ll fill this tempting input, too.
----------
is false.

In the second article the same thing about how bots work:
----------
Most spambots will find your form, determine what the form element names are, and find the URL where the form is posted to. The software will then post those form elements with modified, spam-filled values back to the form submission URL.
----------

From my limited experience, this is not how most bots work (on share sites there are many SPAM engines that can be downloaded and tested to see how they work in order to prevent them ).

From my limited experience, most bots these days use a human for start the spamming, and are similar to Selenium Recorder: http://selenium-ide.openqa.org/
to get the initial "params". (it is visible in application logs, that most initial entries are filled with dummy data but not spam - I guess this is for the spammer to simpler identify what to replace with a variable).

This "script" (a selenium like) is than manually changed (for "variables"), or automatically (if the values are recoblizable - e.g. for names "NNNNN NNNN"), and fed to the bot that is using it as template to replace the variables with values from it's dictionaries.

The logical falacy in many blog posts is to consider that just because it's a "bot" it is fully automatic and it's only "one" tool, so it's stupid - well, it is not :(, it consists of many "tools", and the huge amounth of spam just shows that :(.


Thnx.