Just my two-cents...
For noiseless containers, you'll have to have something that is collapsible so you can remove as much air as you drink to keep the water from sloshing: USGI two quart canteens, Camelbacks, Platypus and Nalgene flexible containers, Echo USGI canteens, etc.
Cheapest will be USGI canteens or Nalgene water bottles on sale. Sure you can use plastic soda/water bottles and the more robust Gatorade bottles. They'll work, but I wouldn't want them for rough use.
If you clean your water containers periodically you won't get that plastic taste. USGI canteens (not the Echo types) are probably worse. The best tasting comes from Stainless steel bottles.
Durability is most likely USGI, Nalgene and the stronger stainless bottles like Guyot/Nalgene and Klean Kanteens. I've seen the older Nalgene bottles dropped several feet (full of water) onto concrete and survive without damage. Another canteen system I really like is the NATO canteen. It has a wider mouth than the USGI canteen, and if you get the steel Crusader cup, it's more robust than the USGI cup; they are more expensive though.
Stainless bottles (single wall) are the heaviest, but most versatile IMHO. You can of course carry water in them, but you can also melt snow or boil water in them directly over the fire...handy for purifying water.
Regardless of your water bottle or canteen selection, I still recommend a nesting cup or mug. It takes up practically no extra space and very little weight. Recommendations are stainless steel, titanium or aluminum...there are several varieties to fit most bottles. Canteen cups are inexpensive and a must if you're running canteens.
As you put your BoB together, it doesn't hurt to have a few different methods. I've packed a Nalgene (plastic) water bottle because I like the wider mouth...it makes it easier to filter water into and I can then transfer to other bladders/containers easier; plus you can use a titanium or stainless nesting mug for a bowl, drinking cup, shovel/scoop, etc.). Camelbacks are excellent for hands-free hydration on the go and you can store that extra 2-3 liters of water or collapsible for less weight and really no loss storage loss inside your pack. I always pack a Platypus (4-liter), Nalgene 3-liter or MSR Dromedary bag...they're carried collapsed, but give me the ability to really stock up on water if needed. Also, I've added a small Klean Kanteen stainless bottle in my "break-away" bag or a smaller (empty) Platypus 1-Liter water pouch.
Lots of options and with a little practicing you can make a good system work for you.
ROCK6