We've evolved to be able to adapt to virtually any diet our environment compels us into adpoting. However, thousands of years ago, our ancestors discovered that adopting dairy products as part of our diet proved a very effective way to stay healthy.
They were so convinced of this, that they invested significant resources into herding cattle, when they could have been foraging for soya beans. Clearly, the strategy paid off, otherwise the practise would have died out due to being a hopelessly ineffectual use of human energy.
Some societies in Africa base their entire diet around cow milk (supplementing it with milk of goats and camels, along with small amounts of vegetable matter when they can get hold of it).
A major practical benefit of milk is that an animal can be made to produce it all year around - unlike meat which is only available once from an animal; or vegetables, which are often seasonal. It's therefore very easy to see that it makes perfect evolutionary sense to drink milk - particularly in winter months when other food sources are scarce.
If we are thinking about our ecological footprint, we need to consider the major problems with international trade. It makes no sense flying food half way around the world, or expending vast amounts of energy on refrigeration, just to quench our guilt about eating meat. We should be looking to source all our food as locally as possible. This might mean a combination of seasonal vegetables, and local cheeses and meats. e.g. sheep make use of land which is completely unsuited to crop-growing. And pigs are an excellent 'food store' for excess vegetable protein which would otherwise rot.
This is the ideal, which is currently difficult to live up to. So I very rarely eat meat. I can't stand the sanctimonious attitude of vegans who depend on former rainforest produced food for most of their diets.