In a blender, combine rice, cinnamon sticks, and 2 cups of water; blend until rice and cinnamon sticks break apart. Add remaining 2 cups of water and let rice mixture stand at room temperature overnight.
Strain the rice water into a pitcher and discard rice. Stir the milk, vanilla, and sugar into the rice water. Chill and stir before serving over ice.