An unscented bleach bottle may be ok in a pinch if very thoroughly rinsed, as far as the bleach itself is concerned, unlike most other chemicals since bleach is a chemical deliberately added to drinking water. I did a calculation for 55 gallon drums, not the small bottles, based on the assumption that bleach permeability was similar to water permeability and the amount of bleach that would be expected to leach out was likely to reach concentrations comparable to what you would deliberately add for water treatment purposes. I would fill the bottle and let it sit for many days to leach out then dump the bottle and refill. The plastic is HDPE which is safe but it may not be made to food grade standards as far as other substances it comes into contact with during manufacture or additives. It is also possible that the bleach may have caused some of the plastic to break down into abnormal byproducts. And it is possible use could lead to accidentally drinking out of an actual bottle of bleach. I would prefer to use it for non-drinking uses but would drink out of it if necessary. CDC says "Rinsed chlorine bleach bottles work well for water storage" http://emergency.cdc.gov/disasters/earthquakes/food.asp
NCSU, howeve, says " Chlorine bleach bottles may be a food approved plastic, but contain an anti-static agent which prevents accumulation of dust during storage and are thus not recommended."
Utah Department of Environmental Quality Drinking Water Division says:
Plastic bleach bottles and gallon milk containers are generally manufactured with non-durable plastic that promotes biodegradability and is undesirable for long term water storage. Water stored in non-durable plastic containers may become toxic over time from breakdown products from the plastic container walls.
The disposable, plastic milk bottle is thin-walled and tends to develop leaks easily. Liquid chlorine bleach bottles are made of thicker polyethylene plastic and may be used for water storage if the empty bottles are thoroughly rinsed with hot water and allowed to dry. However, the use of bleach bottles for water storage is not recommended, because of the potential danger of accidentally drinking bleach instead of water. If bleach bottles are used for water storage, remove the bleach label and write “WATER” indelibly across the bottle. Children may mistakenly associate the size and color of bleach bottles with acceptable sources for drinking water and mistakenly drink bleach. As such, bottles must be positively identified, and bleach must be kept out of the reach of children.
I don't buy that bleach bottles are made to biodegrade. Any modification to make biodegradable would be likely to cause the container to fail when exposed to concentrated bleach.
FEMA was quoted on one site as saying:
If you decide to re-use plastic storage containers, choose two-liter plastic soft drink bottles - not plastic jugs or cardboard containers that have had milk or juice in them. The reason is that milk protein or fruit sugars cannot be adequately removed from these containers and provide an environment for bacterial growth when water is stored in them
A quote attributed to CERT:
Q. Is it true that people should not re-use plastic bottles to store water for disasters?
A. This question is related to the one above. In a Master's Thesis by a student at the University of Idaho, which was not peer reviewed and had questionable scientific validation, the student asserted that certain types of plastic could release certain chemicals which he considered to be potentially carcinogenic. This assertion got widespread media attention last summer. In typical fashion, some media outlets passed on the information without obtaining verification from an independent source. This assertion started flying via email and in web postings from several organizations (but not Red Cross). Some of the public believed the news reports and became frightened, confused, or upset.
A variety of professional organizations and government agencies reviewed the claim, and found it to be untrue. The Food and Drug Administration (FDA), as well as the U.S. Department of Agriculture Food Safety and Inspection Service (USDA FSIS), carefully examined the claim and concluded, through research, that the claim was false. The FDA and USDA FSIS are members of our National Disaster Education Coalition and informed all of us the results of their independent research.
While the water bottling companies would like people to purchase bottled water, what the FDA and USDA FSIS have said is that if people select bottles like plastic soda bottles, and thoroughly clean and sanitize the bottles after use, the bottles can be refilled with tap water and stored for emergency use. Home-stored water should be replaced every six months. They further said that water should NOT be stored in plastic milk jugs. That's because milk has fats in it which get into the plastic, and those fats can not be sufficiently removed or cleaned out, even with bleach or other ordinary cleaning products such as disk soap or dishwashing detergent. Refilling these jugs poses a risk for the residual fats to foster bacterial and algae growth inside the jug. So using these containers to store water is NOT recommended."
FEMA also said: "If you decide to re-use plastic storage containers, choose two-liter plastic soft drink bottles - not plastic jugs or cardboard containers that have had milk or juice in them. The reason is that milk protein or fruit sugars cannot be adequately removed from these containers and provide an environment for bacterial growth when water is stored in them."
Same site also warns of dangers of lead glaze in old bathtubs and lead in old glass containers.
I did not find scientific studies of water quality when stored in bleach bottles.
Vinegar bottles would likely be ok, but the small amount of vinegar leached would not have anti-microbial benefits.
I would discard the water stored in reused containers periodically, during the first year, to remove any leached substances.