I think it’s important for kids to have necessary fats and nutrients in their diets, regardless of the source (as long as it is a healthy source, obviously). It’s often a lot easier to get kids to drink milk or eat yogurt than to get them to take in non-dairy foods that supply calcium. I don’t think anyone would suggest that a young child eat cheeseburgers loaded with grease on a daily basis, or slather butter on everything that goes into their mouths, but their nutritional needs are different from adults. Getting your fats and calcium from whole milk, yogurt, cheese, or meats like chicken or steak is probably better than getting it from potato chips and ice cream. Ask any parent whose child is lactose intolerant or allergic to dairy, and I’m sure they could give you plenty of alternatives to whole milk.
We gave our kids whole milk until the youngest was two. Then we switched to 2%. I know families who continued to give kids whole milk past the age of two because the children were considered too thin by pediatricians. This is usually decided when a child’s growth shows a drop or his or her height starts to outstrip weight. I even knew a mom who was concerned that her daughter would grow up with unhealthy habits after the doctor suggested she go back to whole milk and give the child ice cream, cheese, and butter daily. She was very small for her age, and her growth rate had dropped. She followed the doctor’s advice, and later was able to gradually switch back to healthier levels of those fattier foods.