Pool water, after filter by your pool filter, would pass all federal regulations for potable water (as long as the Chlorine level is between.5 -5 ppm). Matter of fact, it's probably cleaner the tap water.
See, the water in a pool gets filtered everyday, whereas potable water only gets filtered once, and it's taken from surface water. Much of the water in a pool can be rain water, think distilled. As a pool filter runs it tends to capture smaller particles every day, until it's time for back washing.(especially if you have a DE filter.)
As for the chemicals that get added I've seen worse added to potable water, such as copper sulfate. Unless you like green looking water in your pool you'll never add that stuff. Pools chemicals can be kept to a minimum if you keep it clean and brush it occasionally.
I never add any algaecides, Chlorine can handle algae just fine. The only other thing I add is baking soda. That's better for you then the hydrated lime I add to the water at work.
The worst things in pool water would be cyanuric acid, which there is no federal regulation on for potable water, and calcium chloride, for raising the calcium level in concrete pools. Again, there's no limits of allowable calcium in potable water. It makes the water hard, but many places in this country have much harder water then what's in a pool.
As a side note, I've worked on city pools (license required) and now I'm a licensed plant operator making potable water.