Greenpeace wants fewer cows and more regulation, saying the dairy industry has been allowed to run riot and wreak havoc on the environment.
The report said 94 percent of the river lengths in urban areas are not swimmable, that figure is 82 percent for rural river lengths.
It also found that between 1996 and 2012 more than 70,000 hectares of native vegetation was converted to pasture, forestry and urban areas.
Greenpeace spokesperson Steve Abel said the report painted a comprehensively damning picture of the dairy industry, which is harming biodiversity, rivers and lakes.
"Dairy bosses have been telling us they're well on the way to cleaning up their act, but the environmental statistics show the truth - the biggest degradations in New Zealand's environment in recent years have been caused by the dairy industry," Mr Abel said.
"This latest report from the Ministry for the Environment is a withering exposé of dairy's part in trashing the planet."
There needed to be a reduction in the number of cows, Mr Abel said.
"It means we still have agriculture and farming but we just have agriculture and farming that is not wall-to-wall cows basically.
"There are many other ways and many other things that we can farm that can be profitable and viable and help to feed our population."
He said no industry should be able to get away with harming the environment so severely.
Dairy sector defends ongoing improvements
However, the dairy sector is promising it is already cleaning up its act after the report highlighted it still has work to do.
A DairyNZ spokesperson, David Burger, said work was underway - with 98 percent of significant waterways fenced off, and farmers were working on ways to improve effluent discharge.
It took time for that to be picked up in scientific research because some contaminants took years or even decades to break down.
"Some of these contaminants take maybe two or three years to actually see a reduction in downstream whereas others, such as nitrogen, may takes decades or even more, depending on what catchment you're in."
Dr Burger said dairy farmers were working on environment plans that will be implemented over the next five years.
Grim reading but useful for making changes, Minister James Shaw says
Climate Change Minister James Shaw said the report made for grim reading, but it also gave a clear idea of what problems need to be addressed.
Significant changes were required from everyone, including how people get around, and how factories and farms work.
Mr Shaw said farmers would need to learn from the best experiments so they could apply them to their own land.
"Early consistent action is by far the lowest risk, least cost action. The longer we leave it the harder it gets, and the more expensive it will be and the greater risk we expose ourselves to."
He said the report reinforced the government's plan to clean up waterways and make New Zealand carbon-neutral.
Regional councils failing to protect waterways, says Fish and Game
Meanwhile, Fish and Game is blaming regional councils for allowing waterways to become too toxic for fish to live in. The environment report says three-quarters of freshwater fish are now threatened with, or at risk of, extinction.
Fish and Game chief executive, Martin Taylor, said regional councils were responsible for protecting the environment, but this report showed they had failed to do so.
Mr Taylor said there were no fish in rivers and lakes in parts of Taranaki, Canterbury and Southland because the councils had allowed destructive activities, such as intensive agriculture, to take off.