Feature Request 1572837: mutex API nesting level info

[Harald Kipp]

Can someone please look to

[ 1572837 ] mutex API nesting level info
http://sourceforge.net/tracker/index.php?func=detail&aid=1572837&group_id=34079&atid=410690

I'm not very familiar with this module and not sure, whether this is
meaningful. How can mutual exclusion become nested?

Thanks,

Harald

_______________________________________________
http://lists.egnite.de/mailman/listinfo/en-nut-discussion

[Ole Reinhardt-2]

Hi,

> [ 1572837 ] mutex API nesting level info
> http://sourceforge.net/tracker/index.php?func=detail&aid=1572837&group_id=34079&atid=410690

My understanding of this feature request is, that you get back an
information of how much threads are waiting on this mutex. I don't feel
this information is very usefull.

Ole

--
 _____________________________________________________________
|                                                             |
| Embedded-IT          Hard- und Softwarelösungen             |
|                                                             |
| Ole Reinhardt        Tel. / Fax:        [phone] |
| Luisenstraße 29      Mobil:             [phone] |
| 57076 Siegen         eMail:    ole.reinhardt@... |
| Germany              Web:         http://www.embedded-it.de |
|                      UstID / VAT:       DE198944716         |
|_____________________________________________________________|

_______________________________________________
http://lists.egnite.de/mailman/listinfo/en-nut-discussion

[Nathan Moore-5]

I believe that what you would actually get back is how many times the
current thread has called NutMutexLock on a particular lock.
The nesting of these mutexes is so that if a function like fputc were to
lock the file's mutex around it's operation, and then fputc was also called
by fprintf which needed to keep all of it's output together it could also
lock the file's mutex without worrying about blocking fputc being blocked or
releasing the lock prematurely.
Returning the count might offer some debugging help (ie "NutMutexUnlock just
returned != 0 at what should have been top level"), but I don't see how
useful it would be in production use.

Nathan
_______________________________________________
http://lists.egnite.de/mailman/listinfo/en-nut-discussion

[Matthias Ringwald]

I've posted in the tracker to ask what it is used/needed for.  
Otherwise I would agree with Nathan that it's probably for debugging  
purposes.

Hi

adding the return statement would not be hard. But why do you want  
this? I assume it could be used to write some asserts.
Would directly accessing mutex->count be very inconvenient in your case?

matthias


On 09.07.2008, at 16:31, Nathan Moore wrote:

> I believe that what you would actually get back is how many times the
> current thread has called NutMutexLock on a particular lock.
> The nesting of these mutexes is so that if a function like fputc  
> were to
> lock the file's mutex around it's operation, and then fputc was also  
> called
> by fprintf which needed to keep all of it's output together it could  
> also
> lock the file's mutex without worrying about blocking fputc being  
> blocked or
> releasing the lock prematurely.
> Returning the count might offer some debugging help (ie  
> "NutMutexUnlock just
> returned != 0 at what should have been top level"), but I don't see  
> how
> useful it would be in production use.
>
> Nathan
> _______________________________________________
> http://lists.egnite.de/mailman/listinfo/en-nut-discussion

_______________________________________________
http://lists.egnite.de/mailman/listinfo/en-nut-discussion

[Przemyslaw Rudy]

Hi,
Sorry for the delay.

The story with this mutex is pretty simple:
There is a function that calls the mutex. This function can be called
recursively. But only for the first nested call an additional action
must be made. Thus the simplest way for this is to check the mutex value
just after taking it.

This is not for the debug. There is obviously no problem taking the
mutex value directly, but accessing internal data structures is a hack
in my opinion. On the other hand there is no big pain giving this value
back with public API.

Thanks
Przemek


Matthias Ringwald pisze:

> I've posted in the tracker to ask what it is used/needed for.  
> Otherwise I would agree with Nathan that it's probably for debugging  
> purposes.
>
> Hi
>
> adding the return statement would not be hard. But why do you want  
> this? I assume it could be used to write some asserts.
> Would directly accessing mutex->count be very inconvenient in your case?
>
> matthias
>
>
> On 09.07.2008, at 16:31, Nathan Moore wrote:
>
>> I believe that what you would actually get back is how many times the
>> current thread has called NutMutexLock on a particular lock.
>> The nesting of these mutexes is so that if a function like fputc  
>> were to
>> lock the file's mutex around it's operation, and then fputc was also  
>> called
>> by fprintf which needed to keep all of it's output together it could  
>> also
>> lock the file's mutex without worrying about blocking fputc being  
>> blocked or
>> releasing the lock prematurely.
>> Returning the count might offer some debugging help (ie  
>> "NutMutexUnlock just
>> returned != 0 at what should have been top level"), but I don't see  
>> how
>> useful it would be in production use.
>>
>> Nathan
>> _______________________________________________
>> http://lists.egnite.de/mailman/listinfo/en-nut-discussion
>
> _______________________________________________
> http://lists.egnite.de/mailman/listinfo/en-nut-discussion
>
>

_______________________________________________
http://lists.egnite.de/mailman/listinfo/en-nut-discussion

[Matthias Ringwald]

Hi Przmek


On 13.07.2008, at 23:01, Przemek Rudy wrote:
> This is not for the debug. There is obviously no problem taking the
> mutex value directly, but accessing internal data structures is a hack
> in my opinion. On the other hand there is no big pain giving this  
> value
> back with public API.

I completely agree with this statement but I'm still wondering if  
checking
the recursive mutex is the right way. The POSIX.1 standard does not  
provide
this option, so my gut feeling is that maybe there should be an even  
more
clean solution than returning the locking level.

In my understanding, a mutex is used to protect access to a shared  
object.
The recursive mutex allows a function/thread to successively lock
(assert ownership of) this mutex without checking whether it already  
owns the mutex.

If you already protect the function with a mutex, you're save to use a  
simple
int counter as your own locking count inside the function. Then you  
neither
have to access internals of the mutex nor have to use "exotic" :) APIs.


Does this make sense in your case?

Cheers
  Matthias

(Sorry for beeing picky but I'm C.S. PhD student... :)

> Hi,
> Sorry for the delay.
>
> The story with this mutex is pretty simple:
> There is a function that calls the mutex. This function can be called
> recursively. But only for the first nested call an additional action
> must be made. Thus the simplest way for this is to check the mutex  
> value
> just after taking it.

_______________________________________________
http://lists.egnite.de/mailman/listinfo/en-nut-discussion

[Alain Mouette]

Why not use a Semaphore? I understand that it is basicaly a counter,
just as needed. I have already used mutexes implemented as semaphores
with a max count of 1...

Alain

Matthias Ringwald escreveu:

> Hi Przmek
>
>
> On 13.07.2008, at 23:01, Przemek Rudy wrote:
>> This is not for the debug. There is obviously no problem taking the
>> mutex value directly, but accessing internal data structures is a hack
>> in my opinion. On the other hand there is no big pain giving this  
>> value
>> back with public API.
>
> I completely agree with this statement but I'm still wondering if  
> checking
> the recursive mutex is the right way. The POSIX.1 standard does not  
> provide
> this option, so my gut feeling is that maybe there should be an even  
> more
> clean solution than returning the locking level.
>
> In my understanding, a mutex is used to protect access to a shared  
> object.
> The recursive mutex allows a function/thread to successively lock
> (assert ownership of) this mutex without checking whether it already  
> owns the mutex.
>
> If you already protect the function with a mutex, you're save to use a  
> simple
> int counter as your own locking count inside the function. Then you  
> neither
> have to access internals of the mutex nor have to use "exotic" :) APIs.
>
>
> Does this make sense in your case?
>
> Cheers
>   Matthias
>
> (Sorry for beeing picky but I'm C.S. PhD student... :)
>
>> Hi,
>> Sorry for the delay.
>>
>> The story with this mutex is pretty simple:
>> There is a function that calls the mutex. This function can be called
>> recursively. But only for the first nested call an additional action
>> must be made. Thus the simplest way for this is to check the mutex  
>> value
>> just after taking it.
>
> _______________________________________________
> http://lists.egnite.de/mailman/listinfo/en-nut-discussion
>
>

_______________________________________________
http://lists.egnite.de/mailman/listinfo/en-nut-discussion

[Nathan Moore-5]

>
> If you already protect the function with a mutex, you're save to use a
> simple
> int counter as your own locking count inside the function. Then you
> neither
> have to access internals of the mutex nor have to use "exotic" :) APIs.
>
> I think the idea of the recursive mutexes is that you don't need to worry
about if a mutex will be locked 'again' by a lower level function, so
keeping a counter makes the user need to be more aware of when a lower level
would do this, and edit the lower level code to also count.
Personally I haven't needed this value returned, but I can see how it may be
very useful to find errors.  Just using the count as a boolean at the
application level would be enough for simple testing, though:
    unsigned rc;
    ...
    rc = NutMutexLock(&lock);
    ...
    assert (NutMutexUnlock(&lock)==(rc-1) );
(or similar) could be useful anywhere, if there weren't the NutMutexTryLock
that returns a value already.

Not changing away from posix-like only matters if you are trying to achieve
posix compatability, which itn't likely to a large degree within NutOs.
Also, it's just a value that can be ignored if you don't use it.

Nathan
_______________________________________________
http://lists.egnite.de/mailman/listinfo/en-nut-discussion

[Matthias Ringwald]

On 14.07.2008, at 16:52, Alain M. wrote:

> Why not use a Semaphore? I understand that it is basicaly a counter,
> just as needed. I have already used mutexes implemented as semaphores
> with a max count of 1...
>
> Alain

Yepp. But, again, in Nut/OS and Posix, you cannot read-out a  
semaphore. Even more, a semaphore initialized to one cannot be aquired  
a second time, even if its by the same thread. that's why the  
recursive mutex is good for, it allows repeated locking by the  same  
thread.

matthias
_______________________________________________
http://lists.egnite.de/mailman/listinfo/en-nut-discussion

[Przemyslaw Rudy]

Matthias Ringwald pisze:
> ...
> semaphore. Even more, a semaphore initialized to one cannot be aquired  
> a second time, even if its by the same thread. that's why the  
> recursive mutex is good for, it allows repeated locking by the  same  
> thread.
Yes, that is the reason...

So I see just up to two questions that need to be answered:
1. If to follow posix or not:
[yes] - do not change anything.
[no]:
2. if the returned mutex counter is useful:
[yes] - then change mutex code
[no]  - do not change anything.
_______________________________________________
http://lists.egnite.de/mailman/listinfo/en-nut-discussion

[Nathan Moore-5]

> Why not use a Semaphore? I understand that it is basicaly a counter,
> just as needed.

That counter is counting a different thing.  Semaphores allow some settable
number of callers to enter at any given time but is not thread aware.
The recursive mutex is thread aware.  It only lets one thread lock the mutex
at any given time, but counts the number of times that that mutex does lock
and unlock.  Only when this counter is 0 can another thread get the lock.
You can't do this with semaphores.

Nathan
_______________________________________________
http://lists.egnite.de/mailman/listinfo/en-nut-discussion

[Alain Mouette]

Oi Andreas, essa resposta faz sentido para você? Ele afirma com muita
propriedade que samafores não são "thread aware", existe isso???

(isso é na lista do NutOS, por isso é importante)
Alain

----------------
Why not use a Semaphore? I understand that it is basicaly a counter,
just as needed. I have already used mutexes implemented as semaphores
with a max count of 1...

Alain

Nathan Moore escreveu:

>> Why not use a Semaphore? I understand that it is basicaly a counter,
>> just as needed.
>
> That counter is counting a different thing.  Semaphores allow some settable
> number of callers to enter at any given time but is not thread aware.
> The recursive mutex is thread aware.  It only lets one thread lock the mutex
> at any given time, but counts the number of times that that mutex does lock
> and unlock.  Only when this counter is 0 can another thread get the lock.
> You can't do this with semaphores.
>
> Nathan

_______________________________________________
http://lists.egnite.de/mailman/listinfo/en-nut-discussion

[Alain Mouette]

Hi,

I was checking with a friend who has a better understanding of those
matters:

Neither os us have eve known of a mutex that can be locled more than
once on the same thread just because the thread lock on the first atempt
if it is not available.

Also never heard of a Semaphore that is not thread-aware.

I was wrong about using Semaphores as counters, this is a non-standard use.

The "normal" way of controling re-entrancy is to have a mutex + a
counter, no standard simpler way. In Nut, NutEnterCritical() can be
lighter, but it may depend on the application.

My vote: use only what is standard and well known to every one. (This
has already been Nut's way for long...)

Alain

Przemek Rudy escreveu:

> Matthias Ringwald pisze:
>> ...
>> semaphore. Even more, a semaphore initialized to one cannot be aquired  
>> a second time, even if its by the same thread. that's why the  
>> recursive mutex is good for, it allows repeated locking by the  same  
>> thread.
> Yes, that is the reason...
>
> So I see just up to two questions that need to be answered:
> 1. If to follow posix or not:
> [yes] - do not change anything.
> [no]:
> 2. if the returned mutex counter is useful:
> [yes] - then change mutex code
> [no]  - do not change anything.
> _______________________________________________
> http://lists.egnite.de/mailman/listinfo/en-nut-discussion
>
>

_______________________________________________
http://lists.egnite.de/mailman/listinfo/en-nut-discussion

[Nathan Moore-5]

>
> Also never heard of a Semaphore that is not thread-aware.
>
You're right.  The way I worded that wasn't very good.  All I meant was that
they aren't aware of which thread(s) have them, only which threads are
waiting on them.

Nathan
_______________________________________________
http://lists.egnite.de/mailman/listinfo/en-nut-discussion

[duane ellis-6]

Przemek Rudy wrote:
> Hi,
> Sorry for the delay.
>
> The story with this mutex is pretty simple:
> There is a function that calls the mutex. This function can be called
> recursively.
>  

Posix PTHREADS describes this as a: PTHREAD_MUTEX_RECURSIVE.

POSIX says:
    A recursive mutex can be locked repeatedly by the owner. The mutex
doesn't become unlocked until the owner has called
*pthread_mutex_unlock*() for each successful lock request that it has
outstanding on the mutex.

An other approach is a "PTHREAD_MUTEX_ERRORCHECK" type:

    An errorcheck mutex checks for deadlock conditions that occur when a
thread re-locks an already held mutex. If a thread attempts to relock a
mutex that it already holds, the lock request fails with the *EDEADLK*
error

== My tidbit ==
Obviously both of the above require an "current owner" in the mutex
structure. (I do not remember the internals of Nut's mutexes)

But - To convert an "error check" into a recursion type manually - do this:

LOCK:
    LOCK (or attempt to lock) the mutex.
          Either this SUCCEEDS, or ERRORS with "you already own the lock"
    obviously you just got the lock or you already own the lock and had
the error.
    Bump a recursion count
    (It is protected by the lock you must made)

WORK:
    Do your work.

UNLOCK:
    You obviously own the lock, the "count" is protected by the lock.
    Decrement the recursion count.
    If the count is now zero - RELEASE the lock.
    Otherwise - do not release the lock.

-Duane.





_______________________________________________
http://lists.egnite.de/mailman/listinfo/en-nut-discussion

[Matthias Ringwald]

On 14.07.2008, at 21:43, Alain M. wrote:
>
> The "normal" way of controling re-entrancy is to have a mutex + a
> counter, no standard simpler way. In Nut, NutEnterCritical() can be
> lighter, but it may depend on the application.

Hi. I hope you all know, that NutEnterCritical is ONLY for critical  
sections and...
If you do any i/o, e.b. printf,.. there could be thread switch and the  
interrupts will be re-enabled.
So only use it for a very short section and don't think it helps with  
other THREADS.

If you do no I/O and don't call any NutEventWait... there will be no  
other thread by concept, hence
no need for NutEnterCritical().

Matthias
_______________________________________________
http://lists.egnite.de/mailman/listinfo/en-nut-discussion

[Bernard Fouché]

Hi,

FYI, here is the point of view of eCos designers about recursive mutexes
(taken from http://ecos.sourceware.org/docs-latest/ref/kernel-mutexes.html):
---


    Recursive Mutexes

The implementation of mutexes within the eCos kernel does not support
recursive locks. If a thread has locked a mutex and then attempts to
lock the mutex again, typically as a result of some recursive call in a
complicated call graph, then either an assertion failure will be
reported or the thread will deadlock. This behaviour is deliberate. When
a thread has just locked a mutex associated with some data structure, it
can assume that that data structure is in a consistent state. Before
unlocking the mutex again it must ensure that the data structure is
again in a consistent state. Recursive mutexes allow a thread to make
arbitrary changes to a data structure, then in a recursive call lock the
mutex again while the data structure is still inconsistent. The net
result is that code can no longer make any assumptions about data
structure consistency, which defeats the purpose of using mutexes.
----

IMHO if a thread locks a mutex, then the thread can remember that fact
in its own set of variable and avoid a new lock attempt.

The problem then becomes 'how to ease 'per thread globals' management'
for the programmer.

It would be handy to be able to 'attach' a data structure to a thread to
ease management of 'globals' for a particular thread.

For instance having a 'void *' pointer inside NUTTHREADINFO
<http://www.ethernut.de/api/struct___n_u_t_t_h_r_e_a_d_i_n_f_o.html> for
application use (a 'HANDLE'). That was quickly discussed a long time ago
("Per thread global variables" circa 18/10/2005 and 19/10/2005) but I
don't think that this idea made its way yet into NutOs code.

With this feature, and using a 'struct' to hold whatever data a thread
needs to handle at the thread level, a thread could easily remember if
it has already locked some mutex and need not do it yet another time. To
implement this we would need a new API call like
'NutThreadSetApplicationDataPointer()' (a better name is welcomed).

When the thread is started, then the developper would malloc() a
structure holding what he/she needs to manage mutexes and whatever other
required info, initialize it, and then call
'NutThreadSetApplicationDataPointer(<malloc'ed struct pointer>)'.

NutThreadDestroy(), seeing that this pointer is not null, would
automagically free it before completely releasing  the thread. The
application data pointer would be accessed in application code thru
'NUTTHREADINFO *runningThread' or a particular #define'd constant to
ease code reading.


    Bernard

Alain M. wrote:

> Hi,
>
> I was checking with a friend who has a better understanding of those
> matters:
>
> Neither os us have eve known of a mutex that can be locled more than
> once on the same thread just because the thread lock on the first atempt
> if it is not available.
>
> Also never heard of a Semaphore that is not thread-aware.
>
> I was wrong about using Semaphores as counters, this is a non-standard use.
>
> The "normal" way of controling re-entrancy is to have a mutex + a
> counter, no standard simpler way. In Nut, NutEnterCritical() can be
> lighter, but it may depend on the application.
>
> My vote: use only what is standard and well known to every one. (This
> has already been Nut's way for long...)
>
> Alain
>
> Przemek Rudy escreveu:
>  
>> Matthias Ringwald pisze:
>>    
>>> ...
>>> semaphore. Even more, a semaphore initialized to one cannot be aquired  
>>> a second time, even if its by the same thread. that's why the  
>>> recursive mutex is good for, it allows repeated locking by the  same  
>>> thread.
>>>      
>> Yes, that is the reason...
>>
>> So I see just up to two questions that need to be answered:
>> 1. If to follow posix or not:
>> [yes] - do not change anything.
>> [no]:
>> 2. if the returned mutex counter is useful:
>> [yes] - then change mutex code
>> [no]  - do not change anything.
>> _______________________________________________
>> http://lists.egnite.de/mailman/listinfo/en-nut-discussion
>>
>>
>>    
> _______________________________________________
> http://lists.egnite.de/mailman/listinfo/en-nut-discussion
>
>  

_______________________________________________
http://lists.egnite.de/mailman/listinfo/en-nut-discussion

[Nathan Moore-5]

>
> IMHO if a thread locks a mutex, then the thread can remember that fact
> in its own set of variable and avoid a new lock attempt.
>
> The problem then becomes 'how to ease 'per thread globals' management'
> for the programmer.
>
> It would be handy to be able to 'attach' a data structure to a thread to
> ease management of 'globals' for a particular thread.


Well a thread may lock an arbitrary number of recursive mutexes at any given
moment,
and adding more and more data to the thread info struct and processing this
info could get
heavy. The current implementation uses O(1) memory (the wait queue memory is
just the
thread's data that's allocated anyway) and the only operation that takes
more than O(1) time
is NutMutexLock when someone else already has the lock.  Then the current
thread is added
to a queue (this is just NutEventWait) which takes O(number of waiting
threads).
Also, the current implementation (as well as the proposed return value
changes) is
completely optional and non-intrusive on other parts of NutOs.

The whole idea is that within any function a programmer only needs to worry
about if she
has matched up the local locks and unlocks and never ever have to think
about what
called functions might lock and unlock, and just assume that those functions
have also
matched up the locking and unlocking of all recursive mutexes that it might
have used.

Nathan
_______________________________________________
http://lists.egnite.de/mailman/listinfo/en-nut-discussion