Believe it or not, kittens don't need milk. And for some cats, cow's milk can actually cause diarrhea.
It is very important to make sure your kitten has a bowl of fresh, clean water at all times. If you suspect your kitten is not drinking enough, it may be that she can taste water treatment chemicals in the water, so you may want to try filtered or bottled water. Some cats even prefer to drink from flowing water sources. You can buy water fountains designed especially for cats. Some kittens don't like their whiskers rubbing the side of their water bowl, so a different kind of bowl may help. And don't forget that if she's eating dry, crunchy kitten food, it's essential you give her plenty of water.