Convocation

Overriding a system call in AIX

At some point, I used to write code for AIX kernel. (I have written modules for the Solaris and HPUX kernels too, but these day's I am purely a user-space developer). Still, I sometimes get questions/comments related to AIX on email. Here is a recent one

I went through your blogs..quite nice..
I need some help with the aix kernel.I m currently implement a system call.
What I need to do is basically intercept a system call on aix 5.3.
How can I do this.?.
On linux I have the system call table , which i can replace the current system calls with my own using the NR offset into the sys_call_table.
How do i do the same in aix kernel?

AIX is a different beast altogether, unlike a lot of other Unixes, because its BSD heritage is not as strong and a bit under the hood. So a lot of things work very differently in AIX, and over-riding a system call is one of them.


For the impatient, here is a short answer.
Write a function with the same signature as the system call and load it as a kernel extension. Any application started after this new module has been loaded will see your symbol as a system call, instead of the original one, due to the way AIX loader works.

For most of the time - you are interested in intercepting a system call, rather than overriding it. Let's say you want to intercept getpid(). Now you could write your own version of getpid with an exactly same signature, do whatever you wanted to do, and then transfer control to the original getpid(). IBM has an example here on how could you add a new syscall to AIX.

Follow it, but name your syscall getpid() (or whatever you want to override).

Herein lies the catch. There is no way you can call the original getpid(), because just having overridden that symbol, you can no longer access the original. Any attempts to call getpid() from with getpid() will result in ...

Well, what else, but recursion.

So to work around this problem, over-riding a system call means writing two kernel modules. The first one would merely re-export the original system call with a different name. The second, will actually override the syscall by redefining it, and then call the original one as exported by the first module. To make it cleaner, you can automate the loading of the first module from within the second module, so to the user, it is only one module that they load.

Usually you would do this fairly early on in the boot process, because your intercept will not apply to applications started before the interceptor module was loaded.

I will update this post sometime in the future with the long answer.

Null Pointer on AIX

Have a look at this code.

#include

int main() {
int *a = 0;
int b = *a;

printf("hello world\n");
return 0;
}

What will the output be. Will it ever get to say hello ? In most worlds other than IBM's, it won't.
But AIX has its own idiosyncracies. So the qualified answer is, "it depends"

Here is what a very old text I found had to say (This was from AIX 3.2 times)

   9.1.  Derefencing NULL Pointers



The word at memory location zero contains a  zero.

That  means  that  code which has been incorrectly

written to use a NULL pointer to represent a  null

string  will work, but will not be portable to all

UNIX operating systems, and may not work on future

AIX systems.

So that explains it. On AIX, at least till version 5.3, you can
dereference a null pointer, and get away with it :)


Here is the link to the original document , for more such trivia. Would post more of these later, need to get back to work.

http://www.rootunix.org/AIX/325bsdpo.txt

Fine Print: This is a repost. I have now decided to use thruput as my only blog, giving up on http://360.yahoo.com/mqzaidi/blog. You would see some old content resurrected from that blog here.