Starting gnome-session in a VNC session is impossible if it's already running on the console.

I can run gnome-panel on the VNC session, but that has lots of problems -- for example, the desktop switcher refuses to appear in the VNC session, and a panel that I have on the bottom-right corner of the (1024x768) screen in the VNC session makes the same panel on the (much bigger) console screen hop over to the same position which is now in the middle of the screen.

I've even gave up on any useful corner panel (reverted to the ancient Sawfish pager thingy instead) -- and there are still an annoying problem: every time the VNC session is closed, the panel is trying to resurrect itself on the console, which leads to an error dialog that there's a pannel there allready -- and requires clicking OK about 120 times for it to go away.

Is there any sane way to use Gnome on both my console and a separate VNC session?

(I don't believe that I'm the only one left that is crazy enough to have two sessions running on a single machine...)