In the United States, at least, tap water is safe enough for watering the garden, bathing, drinking, and putting into your baby’s formula. The EPA constantly monitors public drinking water for safety and contamination. Bottled waters, while popular, are rarely necessary; they testify more to the triumph of marketing than to any real problems with public water (In fact, bottled waters are more frequently contaminated with bacteria and chemicals than their tap cousins are, because the agency in charge of monitoring bottled waters, the FDA, does a less thorough job than the EPA does.).
Tap water sometimes appears cloudy or colored, not because it’s dirty but because it may bear trace deposits from pipes or storage containers. These deposits are unappealing, of course, but they aren’t toxic. Just let the water run for a little while to flush out sediments. The occasional strange odor in tap water derives not from toxicity but from gas buildup in the pipes. Water filters, which are almost as big an industry as bottled water, supposedly purify the water by removing heavy metals like mercury and lead as well as other bacteria and impurities. In reality, tap water contains none of those things anyway, so you may not want to spend money on an essentially superfluous system. Finally, tap water contains appropriate levels of fluoride, which filters do not remove.