I've seen this behavior on many types of systems (including Unix), so it probably is not specific to RHEL4. Have you tried recycling whatever application was using the shared memory? Have you also tried making certain that whatever uses originally kicked off the process, are no longer logged in? If it still doesn't go away, your only solution may be a reboot.
Is this behavior causing you any real problems, or is it just annoying? Perhaps your application or database has some kind of built-in utility to help you. For example, with Oracle, you can use the sysresv command to get rid of shared memory segments also. Finally, I'm probably telling you something you already know, but please be very careful when using ipcrm, as you can crash your system if you get rid of something you're not supposed to.
This was first published in May 2007