Mail Archives: djgpp-workers/2002/05/16/08:04:51
Eli Zaretskii wrote:
> On Wed, 15 May 2002, CBFalconer wrote:
>
> > Which brings up, to me, "why does emacs replace malloc etc."?
>
> Because Emacs, being a Lisp interpreter, constantly allocates and
> deallocates memory in both very small chunks (for Lisp objects like cons
> cells), medium-size chunks (strings and arrays), and very large chunks
> (buffer text storage). A typical malloc implementation, including the
> one in DJGPP's libc, doesn't handle this mix very well, especially in
> programs that run for many days and weeks. (The normal usage of Emacs
> is to start a session once, then leave it running at all times.) The
> result is memory fragmentation that inevitably leads to Emacs allocating
> more and more memory from the system, until you reboot.
>
> > Possibly it has problems with free when many blocks have been
> > allocated, or with multiple reallocs?
>
> Actually, the main problem is to be able to join many small free'd chunks
> into larger chunks. If you fail to do that, you'll hit the problem of
> being unable to allocate memory for reading a file because the free pool
> is fragmented. Then you'll need to sbrk more from the OS.
nmalloc handles this.
>
> This pattern happens a lot in Emacs because you typicall load a file, do
> something with it for some time, then load another file. The ``do
> something'' part allocates many small Lisp objects (since many features in
> Emacs, even important parts of the display engine, are implemented in Lisp).
> Then, all of a sudden, you need many KBytes for the next file.
>
> > In both cases the problems have been handled in nmalloc
>
> The Emacs relocating allocator does things that are unthinkable for any
> general-purpose malloc implementation. For example, under certain
> conditions, it relocates allocated buffers from under your feet. That's
> right: you call malloc or realloc, and when these return, some of the
> C pointers you've got from previous malloc calls are no longer valid,
> they point to chunks of memory that doesn't hold what it held before, and
> text of some buffers has been relocated to other addresses(!). (This, of
> course, requires some cooperation between malloc and the application, and
> some specific knowledge about other library functions that can call
> malloc.)
This nmalloc does not handle. However the effect can occur with
any malloc, by simply keeping a copy of a pointer and then
reallocing that pointer. nmalloc goes to some lengths to avoid
changing the pointer under realloc, but it obviously must be
allowed to occur.
p = malloc(SIZE);
p1 = p;
if ((t = realloc(p, NEWSIZE))) p = t;
else reallocfailure();
/* p1 is not necessarily valid anymore */
>
> I doubt if any sane implementation of malloc can do such crazy
> things. And yet Emacs needs this to prevent indefinite growth of
> its memory footprint.
If you restrict the usage of pointers, as you can with a LISP,
then such operations become quite reasonable. You effectively
insert an indirection level, via handles. But that should not
require replacing the system malloc, which would open the gates
for misusage by calling almost any standard library function that
uses malloc.
--
Chuck F (cbfalconer AT yahoo DOT com) (cbfalconer AT worldnet DOT att DOT net)
Available for consulting/temporary embedded and systems.
<http://cbfalconer.home.att.net> USE worldnet address!
- Raw text -