Mail Archives: djgpp-workers/2002/05/16/01:31:37
On Wed, 15 May 2002, CBFalconer wrote:
> Which brings up, to me, "why does emacs replace malloc etc."?
Because Emacs, being a Lisp interpreter, constantly allocates and
deallocates memory in both very small chunks (for Lisp objects like cons
cells), medium-size chunks (strings and arrays), and very large chunks
(buffer text storage). A typical malloc implementation, including the
one in DJGPP's libc, doesn't handle this mix very well, especially in
programs that run for many days and weeks. (The normal usage of Emacs
is to start a session once, then leave it running at all times.) The
result is memory fragmentation that inevitably leads to Emacs allocating
more and more memory from the system, until you reboot.
> Possibly it has problems with free when many blocks have been
> allocated, or with multiple reallocs?
Actually, the main problem is to be able to join many small free'd chunks
into larger chunks. If you fail to do that, you'll hit the problem of
being unable to allocate memory for reading a file because the free pool
is fragmented. Then you'll need to sbrk more from the OS.
This pattern happens a lot in Emacs because you typicall load a file, do
something with it for some time, then load another file. The ``do
something'' part allocates many small Lisp objects (since many features in
Emacs, even important parts of the display engine, are implemented in Lisp).
Then, all of a sudden, you need many KBytes for the next file.
> In both cases the problems have been handled in nmalloc
The Emacs relocating allocator does things that are unthinkable for any
general-purpose malloc implementation. For example, under certain
conditions, it relocates allocated buffers from under your feet. That's
right: you call malloc or realloc, and when these return, some of the
C pointers you've got from previous malloc calls are no longer valid,
they point to chunks of memory that doesn't hold what it held before, and
text of some buffers has been relocated to other addresses(!). (This, of
course, requires some cooperation between malloc and the application, and
some specific knowledge about other library functions that can call
malloc.)
I doubt if any sane implementation of malloc can do such crazy things.
And yet Emacs needs this to prevent indefinite growth of its memory
footprint.
> which is just standing around
> waiting for critical testing in the DJGPP environment.
I agree that it's bad to leave contributions unreviewed. Could someone
please find time to do that? (I'm sorry to ask for something I cannot do
myself, but lately I have difficulty finding time for any substantial
work on DJGPP; I even can hardly cope with putting out ports of new
releases of GNU packages.)
> If the emacs module is independantly replaceable, Laurynas might
> want to try an immediate bodily replacement and see if the
> problems go away (or if new ones arise, which I doubt).
It's very easy to build Emacs with the library malloc: just add this
line to src/config.h:
#define SYSTEM_MALLOC 1
(You might also need to add _CRT0_FLAG_FILL_SBRK_MEMORY to
_crt0_startup_flags, but I won't recommend doing that unless you see
strange problems or aborts.) To do this with nmalloc, one needs to
replace malloc and friends in libc.a with the alternative implementation,
or hack src/Makefile to add libnmalloc.a (or whatever it is called) to
the link command line.
The ultimate test is to be able to have a live session, actively used for
editing and other stuff which Emacs does, run for a significant amount of
time, and watch its memory footprint with some tool. The footprint
should level out at some point, instead of constantly growing.
- Raw text -