Besides the outdoor pools available to the public during summer at Curry Village and Yosemite Valley Lodge, swimming is generally permitted in all bodies of water in the park. There are some exceptions. Swimming is prohibited in:

Hetch Hetchy Reservoir, and within one mile upstream along any tributary (e.g., Rancheria Falls).
Dana Fork of the Tuolumne River
Emerald Pool and Silver Apron (above Vernal Fall)
Lake Eleanor Reservoir (when posted)
Wawona Domestic Water Intake (and 100 yards upstream)
Swimming in the Merced River is a popular way to cool off–but help protect the river by entering and exiting only on sandy beaches. Swimming in rivers is not without hazards, including swift currents, cold water, and hazards within the river (e.g., trees).