I have been to New Orleans several times. It certainly has some big down-sides, but I feel there is a lot worthwhile there as well. It certainly has a lot of character for a US city. It's interesting to watch the people and check out the city even if you aren't interested in the party aspect of it. There are some good restaurants as well.
I basically think of New Orelans like a second or third world country. There is poverty, dirt and crime but there is also a lot more for anyone interested in discovering it, and the people speak English.
I've never had a problem down there, but people certainly do as McAttack unfortunately experienced. Keep your wits around you and try not to walk around alone (though I have done it extensively and never had a problem). A group of several guys is certainly not going to have any problems as long as they stay in the main tourist areas of the French Quarter.
There is no doubt the city is dirty. Do not wear nice shoes or even pants out walking around in the Quarter at night, especially during a major event like Mardi Gras. The ground gets covered with a nasty gray slurry. If it rains hard even briefly, parts of the Quarter flood a couple inches deep with filthy water. There are millions of enormous rats and nutrias around, but you probably won't see them where there are lots of people.
As you would expect, all the hotels and restaurants in that area jack up prices for big events like Mardi Gras. You can stay across the river for less and take a cab back and forth, but I have never done it and I'm not sure you end up saving all that much by the time you add up all the cab fare. I like being able to walk back to my hotel anyway. Most of the older hotels are not very nice (there are some exceptions) even though they may cost $300 per night. Unless you are going to stay in one of the very nice / expensive hotels, try one of the newer chains - they are cleaner and have nicer and more modern amenities. If you get a hotel right in the heart of the Quarter, do not expect it to be very quiet.
I would not drive in the French Quarter other than to get to the hotel. Once there you park the car (or have it parked - most are valet) and don't touch it again until check-out. You simply don't "cruise" around the Quarter during Mari Gras. That is a complete joke. Anybody who has been there knows that. I do not see any real benefit to driving any kind of distance to get to New Orleans. Just fly and take a cab or limo from the airport.
If you are near a fight on the street, get away from it quickly. You will understand why once you see the police react to a street fight.
