NEXTSTEP Debugging Q&A

Creation Date: August 14, 1998
Keywords: NeXTSTEP

Disclaimer

This document pertains to the NeXTSTEP operating system, which is no longer a supported product of Apple Computer.  This information is provided only as a convenience to our customers who have not yet upgraded their systems, and may not apply to OPENSTEP, WebObjects, or any other product of Apple Enterprise Software.  Some questions in this Q&A document may not apply to version 3.3 or to any given specific version of NeXTSTEP.

Q:  In my application, I invoke other commands and programs by using system() or other relatives like popen(), execl() .  Sometimes when debugging my application, I run into this error message:

	sh: privileges disabled because of outstanding IPC access to task

What does it mean and what can I do about it?

A:  This has to do with an unfortunate interaction between gdb and setuid program execution.    When gdb is debugging a process, it owns the exception ports of that process.  When that process forks a child process, gdb would own the exception ports of that child process as well.   Because of security issues, the kernel disallows gdb from owning the exceptions ports of a child process that is setuid.  When you attempt this, the kernel generates the privileges error message and the system() call fails.

There will be no conflict outside the debugger and you can run gdb as root as a workaround for debugging.

Q:  The gdb debugger refuses to display the value of my variable ªx.º  Following is the source code and the gdb output:

	- convert:sender {
	    	char    *fName;
 
	    	foo = NO;
    		[self enableConvert];
	    	fName = [myText stringValue];
		<--- break here

	(gdb) print fName
	No symbol "fName" in current context.
	
A:  When code is compiled with the -O flag, the compiler can take liberties to optimize away certain source code variables.  This is accomplished by careful juggling of registers and order of expression evaluation.  For example, stack variables which never have their address taken and are used only across a very few instructions can ªdisappearº without a trace.   Try compiling your code without optimization and see if this eliminates some of these surprises.  The standard ªmake <appname>º rules do invoke optimization as shown by the -O flag:

	myhost> cc -g -O -Wall  -c QCombo.m -o obj/QCombo.o

The standard ªmake debugº rules do not:

	myhost> cc -g -Wall -DDEBUG  -c QCombo.m -o debug_obj/QCombo.o

The ªDebugº button in ProjectBuilder does not assume ªmake debug.º  You have to add ªdebugº to the Args text field to pass it as an argument to the make process.
	
The man page for cc contains more detail over how one can control compiler optimizations; a small section is repeated here:

-O	Optimize.  Optimizing compilation takes somewhat more time, and a lot more memory for a large function.

	Without -O, the compiler's goal is to reduce the cost of compilation and to make debugging produce the expected results.  With -O, the compiler tries to reduce code size and execution time.  Some of the -f options described below turn specific kinds of optimization on or off.

-g	Produce debugging information in GDB format.
	
	Unlike most other C compilers, GNU CC allows you to use -g with -O.  The shortcuts taken by optimized code may occasionally produce surprising results:  Some variables you declared may not exist at all; flow of control may briefly move where you didn't expect it; some statements may not be executed because they compute constant results or their values were already at hand; some statements may execute in different places because they were moved out of loops.  Nevertheless, this makes it possible to debug optimized output if necessary.

The fastest way to pursue one missing variable would be to recompile the troublesome source file from the shell, without the -O flag.  Then relink, and debug the new executable.  
	
Q:  When I run gdb (Release 3) or kgdb (Release 2) on my loadable kernel server, the slave system doesn't stop at the breakpoints I've set.   What am I doing wrong?  

Q:  The symbols in my kernel server seem to have addresses that are relative to the origin of the kernel server, not the origin of the kernel.   What am I doing wrong?   Are these related?

A:  When you compile a kernel server with kl_ld, it produces a relocatable object module.     This  module has not been linked against the kernel, and the addresses of its symbols start at 0.  When you invoke kl_util -a, it edits the server's links against the currently running kernel,  and produces a loadable module, with addresses that reflect its actual location in the kernel.   

Normally, the loadable module is simply loaded into the kernel, and not copied into the file system.    However, if you  invoke kl_ld with the -d option,  then when you later execute kl_util -a, the loader will leave behind a copy of the loadable object file (-d option in bold face below):

	mysystem% kl_ld -n myserver -l Load_commands -u \
		Unload_Commands -i instance_variables \
		-d  myserver_loadable \
		-o myserver_reloc \
		myserver_object_file.o other_object.o
	mysystem% ls *reloc *loadable
		myserver_reloc 
	mysystem%

	mysystem% kl_util -a myserver_reloc
	mysystem% ls *reloc *loadable
		myserver_reloc myserver_loadable
	mysystem%

(Note that you must specify the _loadable suffix as part of <name>.   The system doesn't do that for you.)

It is the _loadable module against which you must execute gdb(kgdb), not the _reloc object.

If you run gdb(kgdb) against the relocatable module, or against the wrong loadable module, the addresses seen are incorrect, and breakpoints do not work.

If you have any question as to whether the module your using is the right one, you can look at its addresses with the nm command, and compare them to the addresses of the same symbols in the kernel you're debugging.   They should be the same.

Q:  When linking my program with MallocDebug,  I get the error message

	failed on vm_allocate
	
in the console when I tried to allocate 43 MB.  However, if I relink the exact same program without the MallocDebug library,  it runs fine.  Why?

Q:  When linking in MallocDebug, I noticed that my program  VSIZE (using ps -ux)  which was 2MB without MallocDebug, would start out way highÐwell over 20MB.  Why?

A:  The problem is that MallocDebug requires all of the memory to be allocated contiguously, which means that it must guess how much the program might need and allocate all of that at once at the beginning  The current guestimate is 40 MB.  This is the reason why when running your program with MallocDebug, you have noticed that the VSIZE went up tremendously.

So if you use MallocDebug,  you can only ask for a total of 40 MB over the life of your program. After you cross that barrier,  it fails.  It doesn't matter if you have freed any or all of the memory you have previously asked for.  The indicative error message is : "failed on vm_allocate" when you try to use malloc(),  calloc(), or realloc().  

There is no workaround other than running your program without using MallocDebug. 
Note also that MallocDebug cannot currently  be used together with ProcessMonitor.