Yup, basically, just start them slow. Some love it, some tolerate it, and some just flip out at the very idea.
A good way to start is by "pea fishing"- get a shallow dish, fill it with a inch of less of lukewarm water, sprinkle in some frozen peas (or corn or whatever your ratties like), and add rats. They should get the idea pretty quick. I did this a couple times, and one of my girls just loves the peas and dives right into the water without a second thought. The other girls sits there, drinks the water, and wishes the peas would come out on their own. So it just depends on the individual rat.
My wild rat Max hates water - I often take the girls into the bathroom with me while I have a bath and Max always makes sure she stays at least 2ft away from the bath. On the rare occasion she's fallen in she's NOT been a happy camper. I normally get the sulk treatment
I'd do as you suggest, test them with little bits of water, shallow water etc and see how they take to it. Mine seem to hate baths (they have the rubber duck phobia that cats have been endowed with) but they can all swim. Minus poor Gus - who once fell in the bath and froze in terror - and sunk! Silly boy (he was fine though - wet but happy enough for a towel dry)