I'll just put this simply: I've played scouts before. I've never had problems. In fact, I've never had a character die while scouting, not once, but I have made a lot of encounters far easier with decent scouting. I've seen one scout character die to a trap, but that was level 2 in World's Largest Dungeon and that trap would have killed anyone else in the party except the Crusader had she not taken that hit, so that's hardly an argument against scouting. So, maybe others can't do it, but I can, and have, and this whole argument has boiled down to a bunch of people saying "I can't do it, therefor it's impossible." So instead of going back and forth on specific scenarios from a DM who clearly has no experience with this, I'll just post how to scout, because that's actually useful. Here's the main things you need to keep in mind, and I'll phrase them for all kinds of games (high optimization and low).
1) Be invisible to as many things as possible.
A) The most basic part of this is Hide and Move Silently, because everything has Spot and Listen, even if they have no ranks. As a rule, you're only truly safe if your Hide and Move Silently, after modifiers, are 11 higher than the enemy's Spot and Listen. That way, if you take 10 while sneaking around the enemy cannot possibly find you. This is trivially easy... a Halfling straight out of the box gets +5 Hide, and +3 Move Silently, while a Whispergnome gets +9 Hide and +5 Move Silently. Through in max ranks in both skills and a 14 base thrown into dex and you're looking at +11/+9 or +15/+11 even at level 1, when the vast majority of enemies don't even have spot ranks. Note also that distracted people have a -5 to their spot and listen, and there's an additional -1 to spot and listen for every 10 feet between the observer and what they're spotting. With these kinds of penalties, you don't even need items to avoid any chance of detection. As you go up in levels you'll want to invest a bit of money into keeping Hide and Move Silently up, an amount determined by how much it comes up. The Collar of Umbral Metamorphosis (22k continuous, 10k for 10 minutes per day) is great for this, providing the Dark template cheap (which gives +8 hide, +6 move silently, Darkvision and Superior Low Light Vision, and (Ex) hide in plain sight along with a few other nice abilities). Shadowsilk Leather Armor is just 1,750gp and provides +2 to hide and move silently while also being extremely light weight (an issue for low strength stealth types), and it's not even magical so crafting it for 1/3 price is a possibility in games where that works. There's also basic mundane gear like Silent Shoes and Darkweave Clothing that can increase both skills. And if you're a Factotum, you even get to add your Int to it. In the end, you should have little to no trouble keeping your Hide and Move Silently up so high that nothing can possibly detect you via Spot or Listen.
B) Scent, Tremorsense, Blindsense, and Blindsight are all relatively common abilities that autodetect you, and you just can't have that. Luckily, these are easily countered by taking the Darkstalker feat, which requires them to follow the Spot-Hide/Listen-Move Silently mechanic. Since you needed those to be up anyway, this handles the problem nicely, but the feat is basically required if you want to be a stealthy character. Without it, you're hosed... which is part of why Wizards and such actually aren't very good at this sort of thing. No spell protects you from these things (at least not in a useful way).
C) Lifesense. This feat from Libris Mortis is only available to undead and causes all living creatures to glow brightly in your vision (a medium or smaller creature gives off 60 feet of light, while larger creatures double the light given off for every size category they are above medium). Note that even invisible light sources give off light, so this sense completely trumps invisibility and no amount of hiding will save you (the enemy won't see you, but they'll know exactly where you are anyway. They'll even see you coming from around corners). On the bright side (heh) Necropolitans are found in this same book. Basically, if Libris Mortis is in play, you absolutely want to be a Necropolitan (which is very useful for stealthers anyway for a variety of reasons). If it's not in play, Lifesense isn't an issue anyway so there's nothing to worry about. And if you do become a Necropolitan, take this feat!
D)
Touchsight. This auto-win detection can be annoying, but with a minute per level duration, it shouldn't even be up unless the party has already given away the fact that it's there (which may happen). Nothing besides being incorporeal can evade it (and being permanently incorporeal has its own issues), but it has some weaknesses. First of all, it requires Line of Effect to see things. If you have the ability to see through walls (Mindsight, Earthdreamer) you can watch someone who might have Touchsight up without fear, and remember the following from the Line of Effect rules: "An otherwise solid barrier with a hole of at least 1 square foot through it does not block a spell’s line of effect. Such an opening means that the 5-foot length of wall containing the hole is no longer considered a barrier for purposes of a spell’s line of effect. " In other words, if you're peering at the target through a keyhole or grating or any other surface that doesn't have a full 1 square foot hole in it, Touchsight can't see you at all. Also, note that it has a 60 foot radius, though this can be enhanced. We'll get more into proper scouting distance later. Finally, it's a rare power available only to Psions and Wilders, so it shouldn't come up much at all.
E) Mindsight. This is a feat from Lords of Madness that autodetects anything with a mind within the range of your telepathy (for most creatures, this will be 100 feet or less). First of all, check with your DM to see what this power actually does. By RAW, it autodetects everything with no chance of blocking it... if this is so, TAKE THIS FEAT. However, the same book that published it mentions that Illithids hate undead, and one of the reasons is that their mental detection abilities don't work on undead. This suggests that RAI was for Mindsight not to work on people who are immune to Mind Affecting, in which case you'll want to be Necropolitan even more (or at least get some other method of getting that immunity). But if it's played by RAW, this is a good reason to stay 100 feet away from enemies that might have the feat. Note that this is another rare ability... it requires the enemy to be telepathic and have taken the feat. This is one good reason why having a few Knowledge skills is wise, as it lets you know about such abilities.
F) Darkvision. This is a really common sense that still works off spot/hide, so you can hide from it just fine. It's mentioned here because the Ring of Darkhidden makes you completely invisible to it, and most underground/night monsters that use it rely on it as their sole vision mode when it's dark. With that ring, which is quite cheap, you're now completely invisible to them the whole time. If you play in the underdark or some similar area, your whole party should have such rings. An important note is the range here... Darkvision is usually a 60' range, sometimes 120', very rarely anything past this. You want to scout so that your party doesn't come within 120' of areas you haven't scouted, so that they're not picked up by this visual mode.
G) Magical detection. There's a bunch of abilities (Arcane Sight, Detect Magic, Detect Undead, etc) that use magic to spot unusual things. Theoretically, they should still require spot, but just in case note that almost all magical senses are blocked by a thin layer of lead. Yes, you can actually hide behind a lead sheet or lead-lined cloak (though note that weight could make this impractical). The obvious permanent one people could use is Arcane Sight, but the range on that ability is 120' and it makes their eyes glow blue, so if you see someone with blue glowing eyes, stay out of the 120' range (and note that they're likely a caster). Like Touchsight, these senses aren't that common, but you should consider their existence.
2) Have better detection than your enemies.
A) Spot and Listen are handy ways to find things... they're automatic, after all. But as we saw earlier, using Spot and Listen are at a serious disadvantage against Hide and Move Silently. Distraction carries a -5 penalty, and distance gives a -1 per 10 feet penalty. Considering most scouts want to stay pretty far from the enemy, Spot and Listen end up being pretty poor ways of locating people. Also, Invisibility screws Spot pretty good and Silence screws Listen. As such, while it's a good idea to have a decent Spot score, it may not be worthwhile to waste too many points and resources maxing both these skills out (though that will depend on where else you spend your skill points and what exactly your character intends to do). As before, having night vision abilities (Low Light, Darkvision) are nice, but don't rely on them.
B) Scent, Blindsight, Blindsense, Tremorsense. These are obviously useful abilities to have, as they autodetect enemies who don't have Darkstalker (few will). Blindsight is easily gotten via the Blindfold of True Darkness, but doing that means you can't see anything past 30' (but non visual senses like Mindsight work fine). If you're using Lifesense, this is obviously a terrible idea, but in general getting a Blindfold that you can put on temporarily at times might be worth having. Scent is easiest gotten via Hunter's Stance. But again, the range is quite short, so neither of these abilities are all that great. It's pretty hard to get any of these senses with any decent range. Heck, if anything, it's almost better to have an attack animal with the party that has these (like bats or dogs or whatever) and leave them back there in case the party is about to get jumped by an invisible thing you somehow missed, while you go ahead and do the scouting thing. These aren't high priority senses.
C) Lifesense. As above, this sense is AMAZING for spotting things. It works on anything that's alive (take that, Mimic) and lets you know about them long before you even get close. If it's available, be a Necropolitan and take it, no question.
D) Mindsight. As above, this is another amazing sense. If it's available and you can do so, take it. It's easy for Beguilers, Arcane Tricksters, and Factotums to get via a Mindbender dip. Rogues, Scouts, and Ninjas are left out in the cold on this one (part of why they're really not that good at scouting). Make sure to check with your DM so you know what it can pick up... by RAW, this thing works through walls (unlike divination, it goes through lead too).
E) Magic. For the most part, magical detection isn't very good. Abilities like Mindblank trump it wholesale, while a sheet of lead or bit of rock stops it cold, and there are even abilities that make magical detection give the wrong answer. Plus, the durations are generally low. If you can get something like Permanent Arcane Sight or Persistent Detect Undead or whatever then go for it, but as a rule you want abilities that warn you if something's there when you didn't realize it, not abilities you have to cast once you already suspect an enemy.
3) Be at the right range. Note how most of these abilities have specific ranges where they function, from 30' to 120' generally. Make sure you're not walking into an autodetect ability that you can't counter, and definitely keep your party from walking into such a thing. Also, your distance from your party is important. You want to be far enough away that they don't get heard or spotted (remember, there's that -1 penalty per 10 feet, so if you can detect things while they're 200 feet away from the party then the party is fine), but not so far away that you're outside of your party's engagement range (which will depend on party composition and level... Beguilers have no trouble dropping Glitterdusts from 130' away and archers usually can fire at 200' away, but full attack melee types are often unable to effect anything usefully unless they're already adjacent, so plan accordingly). After all, if you screw up, you don't want to be by yourself fighting something. You also need to be able to communicate with the party (another good reason why having Telepathy is handy, though there are other methods), in case you get hit while they can't see you. I generally find that being 95' away from the front of the party is just about right in most situations. As a scout, if you're splitting the party, you're doing it wrong.
4) Have ablative defenses. Since you're in front, you're likely to take one big hit before the party kills whatever attacked you, so you want one off defenses that guarantee survival. A ring of +1 natural armor is all fine and well for the tank, but you need stuff that guarantees survival right now from that one hit you're going to take from a trap or from a creature that somehow managed to spot you. This is a reason why a dip into Swordsage is extremely useful for any scout... Moment of Perfect Mind (and similar maneuvers), Counter Charge, and Shadow Jaunt are amazing as once per encounter defenses. They guarantee that one save or lose won't kill you (including traps), that one melee opponent won't reach you, and that you can escape (even from grapples) when you need to. Shadow Jaunt with Cloak of Deception is already a great stealth combo anyway, so this works perfectly. In the end, if you do it right, a monster jumping you is actually falling for bait and guaranteed to be dead as the entire party destroys it, and you're the best person to trigger a trap anyway (though try to avoid said traps, the published ones are incredibly deadly at low levels).
5) Don't spend everything on scouting. You should still be able to do other stuff. Most scouts are actually assassins or diplomats or combat archers or something that's very helpful even when not scouting. There's no requirement to spend huge resources on being a scout... you should still be very effective for the party in general.
And in the end of all this, what's the point of scouting? This is a game where actions are everything, and a single standard action can screw your entire party. If a fight starts with a hidden Beguiler casting Glitterdust on the party in the surprise round, you could be looking at a TPK. If it instead starts with your party landing the Glitterdust and revealing the Beguiler as well as his little Rogue friends, this is going to be a trivially easy fight. Plus, in a game full of monsters with various immunities and vulnerabilities, strengths and weaknesses, knowing your enemy in advance and knowing how to fight him is critical. Far better to avoid wasting attacks that do nothing against a creature that was immune, or to use AoEs and hit the hidden ambushers instead of single target effects that only hit the bait. Thus, what the scout gives the party is actions, and lots of them. Surprise rounds to destroy enemies instead of having those same surprise rounds used against you, and fewer wasted actions when a player fails to account for what's actually against him. And in a game where battles are effectively over after the first round or so (which is often the case when save or lose and crowd control spells start dropping), that's critical.
JaronK