Mail Archives: djgpp-workers/2002/10/22/20:15:34
I am trying to make sure that this exchange contains a list of
things needed to be done, so I have unsnipped a portion. Please
keep it a massive whole, so I can eventually use it for review. I
still have to go for the .h files, nor have I looked for
malloc.txh. Another useful thing would be an example of a
debugging package using all this. I have limited time, my wife
seems to be amazingly able to find things other than programming
for me to do :-)
Eli Zaretskii wrote:
> CBFalconer wrote:
> > Eli Zaretskii wrote:
> > > On Mon, 21 Oct 2002, CBFalconer wrote:
> > >
> > > > > The following external interfaces are missing:
> > > > > mallinfo
> > > > > malloc_debug
> > > > > malloc_verify
> > > > > mallocmap
> > > > > __libc_malloc_hook
> > > > > __libc_malloc_fail_hook
> > > > > __libc_free_hook
> > > > > __libc_free_null_hook
> > > > > __libc_realloc_hook
> > > > > __malloc_get_freelist
> > > > > __malloc_get_slop
> > > > > __malloc_get_bytes_in_use
> > > > > __malloc_get_chunks_in_use
> > > > > __malloc_get_sbrked
> > > > >
> > > > > These will need to be added, or alternate debugging routines written and
> > > > > documented. It appears retrofitting CBF's code with the above routines
> > > > > would be the right thing to do (but is fairly substantial work).
> > > >
> > > > I made provisions for getting at the internal structure, and used
> > > > an example of it in the dumps from the testing code. If you will
> > > > give me the details of what those interfaces have to do I will see
> > > > (no promises) how they can be generated.
> > >
> > > IIRC, all those interfaces, or at least those which are intended to be
> > > for public use, are documented in the library reference (see malloc.txh
> > > in CVS).
> >
> > I still have not gotten CVS installed here, much less learned to
> > use it properly, especially from a remote source.
> >
> > > > The point to me is that
> > > > any such interfaces are done through header files, and leave the
> > > > malloc code itself free to do what it needs.
> > >
> > > I'm not sure I understand. The debugging hooks are called by malloc at
> > > strategic places, provided that they are non-NULL (each hook is a pointer
> > > to a function, by default set to NULL). malloc is free to do what it
> > > needs, but it must call the hooks while it's doing that.
> >
> > Why do it within malloc? Isn't it easier to define say
> >
> > void * xmalloc(size_t sz)
> > {
> > void *p;
> >
> > xmentryhook(sz);
> > p = malloc(sz);
> > xmexithook(p);
> > return p;
> > }
> >
> > in the users space.
>
> Because the hooks don't just tell you when malloc is called. They
> tell you something about the inner workings of malloc, so a program
> which hooked into malloc can gather information about memory
> allocation.
>
> For example, __malloc_get_freelist returns the free list of memory
> blocks, an otherwise internal structure. An application can then
> examine the free list for whatever it needs, like count the size of
> the free memory pool.
The _sysquery call in nmalloc does just that - it returns
sufficient information to completely scan the free list. Having
made the call to get the info, the user is free to scan that free
list at any time. In fact he can usually (but not always) scan
all allocated blocks also. He can also follow the hidden linkages
from any allocated block, including its actual size, physical
neighbors, etc. All this involves *NO* run-time overhead. See
the code in tnmalloc.c
>
> Another example is __libc_malloc_fail_hook, which, if points to
> something non-NULL, is called by malloc when it is about to fail the
> call. An application that sets up such a hook can later print how
> many allocations failed. __libc_malloc_hook is called whenever a
> block is returned to an application, and the info about that block is
> passed as arguments to the hook; this can be used to gather statistics
> about memory allocations. Etc., etc.
>
> malldbg.c uses these hooks to implement a simple malloc debugging
> and memory reporting package.
How is that different from an external intercept (see the sample
xmalloc call above)? It is perfectly free to collect statistics
on the count of calls and the count of NULL returns.
>
> > Incorporating such pointers in the module will mean they
> > have to be NULL initialized and tested
>
> True. I don't think a test against NULL is a significant overhead in
> normal usage.
>
> > and non-standard routines will have to be present to set and reset
> > them.
>
> Well, messing with internals of a library implementation is inherently
> non-standard. Those hooks are meant to be used by memory allocation
> debuggers and various malloc extensions, so standardization of the
> internal names is not the issue. For example, malldbg.c implements
> several fairly-standard functions using these facilities.
>
> > It also means that
> > such routines as "free(NULL)" cannot simply do an early return,
> > but must use the single return point.
>
> I don't see why. `free' simply does this:
>
> if (ptr == 0)
> {
> if (__libc_free_hook)
> __libc_free_hook();
> return;
> }
>
> So you again pay the price of a single comparison.
>
> > Or is name changing the operational problem?
>
> It's mainly a back compatibility problem.
>
> > At any rate, if you can ease my getting the descriptions of the
> > interfaces, I can think about things much better, rather than
> > making wild guesses.
>
> I'll happily help you understand teh current implementation. Just
> ask questions here if the docs is not enough.
>
> > Just looking at the names above realloc_fail_hook seems to be
> > missing.
>
> realloc fails if malloc fails, and the latter has a fail hook.
Not in nmalloc. realloc does not call malloc, because it goes to
lengths to avoid copying the data fields. However it does use the
same subroutines that malloc uses.
>
> But I don't have anything against adding a reall-c-specific fail
> hook, if that's what people want.
>
> > And what about calls that
> > are rejected because the memory has been corrupted or free/realloc
> > is passed an invalid pointer (some of which I detect and do a
> > signal).
>
> If these are detected, we probably want additional hooks for these
> specific failures.
On advice here I made them use SIGABRT, which can sometimes be
caught. For example, an attempt to free something not mallocd, or
already in the free list, is not fatal if converted into a NOP
rather than destruction of the arena, so I allowed a return from
the signal to continue. On the other hand if the error was
something like a fouled arena, a return from the signal will call
exit(EXIT_FAILURE). Before signalling each of these errors emits
an appropriate message to stderr. There there is no problem at
all in adding a hook. The question would be before or after the
signal?.
If we install all these hooks I would probably modify _sysquery to
also return a pointer to an array of hook pointers, and the user
can then put whatever they want in there. Which pointer hooks
what would be a matter of convention. It would be very vulnerable
to poor programming. We could then supply precooked routines
ready for inclusion in the user module to use those pointers to
set/clear hooks. At least that would be my interface
recommendation.
There are only three routines in the package. Thus hooks are
limited, for each, to entry, exit, failure, success, and abort.
Free has no success or failure mechanism, but does have an abort.
--
Chuck F (cbfalconer AT yahoo DOT com) (cbfalconer AT worldnet DOT att DOT net)
Available for consulting/temporary embedded and systems.
<http://cbfalconer.home.att.net> USE worldnet address!
- Raw text -