Our bodies MUST travel to the death realm because all our equipment goes with us to the death realm. Indeed, we can sell things to the Death Guardian and obtain money for that which comes with us to the death realm. (He would not need to buy/sell anything if we just went there spiritually.) When we exit the death realm, all our equiment is intact.
The only thing that goes against this philosophy is the existence of "sentient" NPCs that drop items (rogues, gladiators, etc.?) Their items do not travel to the death realm, and nobody will ever find a rogue or gladiator there. Should we assume that rogues and gladiators are "regular" folk who won't get to the death realm, and us PCs who started as peasants are the "heroes"? I'm not so sure that's a good idea.
I don't really have a good solution, though.