Mail Archives: djgpp/1993/07/13/20:59:24
A few days back I posted a message asking for solutions to the problem
of determining how much electronic RAM was available on the heap. While
it did not generate as much mail as the getcwd question :-), there were
several interesting responses, and requests to post the best answers.
So here goes (and I apologize in advance for the length):
------------------------------------------------------------------------
1. Jeremy Mathers <pynq AT midway DOT uchicago DOT edu> wrote:
I don't think there is any RAM free function - since DJGPP explicitly
makes no logical distinction between real and virtual memory (some would
say that this is a strength of the package).
However, this question comes up frequently, and one solution is to put
your swapfile on a small disk - say, a small RAM disk. Now, your
program will fail very soon after it starts using virtual mem.
My response:
-----------
This is an ingenious solution, but suffers from the obvious disadvantage
of requiring the user to reconfigure his/her system.
------------------------------------------------------------------------
2. Pieter Kunst (P.J.) <kunst AT prl DOT philips DOT nl> wrote:
I had more or less the same requirement for my memory hungry programs.
What I did is rather simple: I added my own wrapper functions around
'malloc()' and 'free()', and added an extra function which calculates
the amount of memory currently allocated. It works fine. The total amount
of available physical memory is displayed when you use 'debug32' to run
the program.
Another reason for this approach is that you can add extra checks in
'free()' to prevent "releasing" memory which was not obtained by a
previous call to 'malloc()', and at exit of your program you can check
for "memory leaks" (blocks of memory which are not freed by the program).
"Memory leaks" in a program can be a serious source for unexpected swapping.
My response:
-----------
If the "extra function" computes only *allocated* memory, then it does not
really address my problem. I want to compute *available* memory, which is
almost the inverse.
------------------------------------------------------------------------
3. Croquette Vincent <vincent AT physique DOT ens DOT fr> wrote:
I have had the same pb for the same motivations, apparently there is no
(simple) way to determine how much free RAM, you have but my experience
with 2D FFT is that provide you have basically enough RAM in the machine
to fit your 2D array + 1~2 Mb you will swapp only once (or not) and the
FFT is going to be fast, we are currently dealing with array 64*32768.
However, you have to be carefuill when allocating a 2D array with DJGPP:
the allocation procedure work by chunk of power of 2 and if you allocate
a 512 element line, it needs 4 extra bytes for the record and thus
reserve a 1024 chunk ! Thus allocating a 512*512 array line by line will
in fact freeze 2 times the expected size, with big array this turns out
to be dramatic. One way to overcome this pb is to allocate the big chunk
512*512 in one operation and then map an array of pointers to begining
of lines. If my explanantion is difficult to understand, I can repeat in
french or I can send an example if needed.
My response:
-----------
Please do *not* repeat in French! I am allocating memory exactly as you
describe (see next message). I am curious how you determined that the
malloc overhead is 4 bytes.
------------------------------------------------------------------------
4. George R. Welch <grw AT tamu DOT edu> wrote:
I have fought with this problem a lot, and I believe it is boils down to
malloc using some buddy-system-like algorythm that only allocates in
powers of 2. It is an *extreme* nuicance, and can be really
*devastating* if you do something like: malloc(16 megs + small number),
because your system will have to come up with 32 megs to satisfy the
malloc.
Fortunately, sbrk() does not suffer this problem, so I wrote my own
malloc that just calls sbrk() and returns an appropriate pointer. The
disadvantage is that I never dealt with free(), because I don't need it.
My response:
-----------
The observation that malloc(n) often seems to allocate far more than n
bytes is well worth noting. Be assured that I am indeed allocating the
dynamic 2-D array in one big chunk (well, really one small chunk
followed by one big chunk). The problem is that the array is first
accessed row-wise and then column-wise, and the latter causes an
excessive amount of paging to disk (*far* more than just one swap) when
it does not all fit in RAM.
I will attempt to upload my dynamic 2-D array code to omnigate.clarkson.edu
in /pub/msdos/djgpp/pub/alloc2d.zip later today. I believe it may be
generally useful.
I have also experimented with sbrk(), which is faster and has less
overhead than malloc() (ref: C Users Journal, Feb 91, p.97). Most texts
recommend against using this low-level system call, for good reason:
it establishes a linear (LIFO) heap rather than a random heap.
BTW, sbrk(0) returns a pointer to the current break address, which is
a convenient way to determine just how much memory has been allocated:
#include <stdio.h>
#include <malloc.h>
main()
{
char *c;
unsigned long m1, m2;
m1 = (unsigned long)sbrk(0);
c = malloc(1024);
m2 = (unsigned long)sbrk(0);
printf("%d bytes were just allocated\n", m2 - m1);
}
Turbo C outputs "1032 bytes were just allocated" - the 8 extra bytes are
obviously malloc bookkeeping overhead. However, with DJGPP the output
is "4096 bytes were just allocated"! I don't think it's simply a power
of 2, since 4096 sounds suspiciously like a page size - have to take a
look at the code for malloc sometime....
------------------------------------------------------------------------
5. jmiller AT chess DOT eisc DOT utoledo DOT edu wrote:
Your problem is very familiar! I have had students write programs for the
same application and things get very slow when page faulting begins. One
crude solution might be to do a system call to mem or memory if you are
using 4dos and redirect the output to a file which your program can open
and read.
The best solution would be to modify go32 so that that calls could be made to
the XMS driver, EMS driver or the bios for extended memory (depending on how
go32 is getting memory) to implement a C function that would return the
amount of free memory. Things are likely to be much simpler when go32 version
2 becomes available since interrupts will be much better supported. This would
allow gcc to compile code that would perform the calls to the various drivers
directly and would be relatively trivial.
My response:
-----------
More ingenious solutions. I also prefer the latter, which avoids making
a system call to a possibly nonexistent program (MEM.EXE only became
available with DOS 5.0). In fact, I have written a C-callable library of
EMS/XMS routines for Turbo C, and tried porting it to GCC. Unfortunately,
I used Turbo-specific features (such as pseudo register variables), and
I know very little about the GNU assembler. Would anyone care to help
translate my routines to GCC/GAS? My translation produces the error:
Unsupported DOS request at eip=......
Perhaps the necessary interrupts are not even supported (sigh).
------------------------------------------------------------------------
Again, I apologize for the length of this message. Thanks to all who
tried to help, or at least commiserate!
- JW
Dr. John M. Weiss, Associate Professor
Department of Mathematics and Computer Science
South Dakota School of Mines and Technology
501 East St. Joseph Street
Rapid City, SD 57701-3995
605-394-6145 jweiss AT silver DOT sdsmt DOT edu
- Raw text -