In a saucepan over medium heat, combine water and rice and bring to a boil. Cover, reduce heat to low and simmer for 15 minutes. Remove saucepan from heat and let sit, covered, for 5 minutes.
Add milk and sugar to cooked rice, stirring well. Return saucepan to stove. Cook over medium heat, stirring continuously, for 15 to 20 minutes or until very thick and creamy. If pudding is too thick, thin with a little more milk until desired consistency.
Remove from heat and stir in vanilla extract or paste. Let rice pudding cool slightly before serving. If desired, sprinkle with ground cinnamon before serving.