The scientists from Leicester and Loughborough Universities say harm is done even if people also exercise.
The study, published in Diabetologia, analysed 18 existing studies involving almost 800,000 people.
Diabetes UK said anyone who spent a lot of time sitting or lying down would "obviously benefit" from moving more.
The researchers say the opportunities for sedentary behaviour in modern society such as watching TV, sitting in a car or using a computer are "ubiquitous".
Of course, in modern society many people head to the gym for a burst of exercise to redress the balance.
But the research team, led by Dr Emma Wilmot from the Diabetes Group at the University of Leicester, says while going to the gym or pool after work is better than heading straight for the sofa, spending a long time sitting down remains bad for you.
Each of the studies they assessed used different measures - for example more or less than 14 hours a week watching TV, or self-reported sitting time of less than three hours a day to more than eight.
The researchers say this means it is not possible to give an absolute limit for how much sedentary time is bad for you.
But Dr Emma Wilmot, who led the study, said it was clear that those who sat the most had a higher risk of diabetes, heart disease and death than those who sat the least.
Continue reading the main story
We can have standing meetings, we can walk during the lunch break, and we can look to reduce TV viewing in the evenings by seeking out less sedentary behaviours_
Prof Stuart Biddle,
She said: "If a worker sits at their desk all day then goes to the gym, while their colleague heads home to watch TV, then the gym-goer will have better health outcomes.
"But there is still a health risk because of the amount of sitting they do.
"Comparatively, the risk for a waiter who is on their feet all day is going to be a lot lower."
She added: "People convince themselves they are living a healthy lifestyle, doing their 30 minutes of exercise a day.
"But they need to think about the other 23.5 hours."
The strongest associations in the analysis were between prolonged sitting and diabetes.
There is evidence that being sedentary negatively affects glucose levels and increases insulin resistance - but scientists do not yet know how.
Dr Wilmot said the study's message could help those at high risk of diabetes, such as obese people or those of South Asian ethnic origin, because it was an easy lifestyle change to make.
Prof Stuart Biddle, of Loughborough University, who also worked on the study, said: "There are many ways we can reduce our sitting time, such as breaking up long periods at the computer at work by placing our laptop on a filing cabinet.
"We can have standing meetings, we can walk during the lunch break, and we can look to reduce TV viewing in the evenings by seeking out less sedentary behaviours."
Dr Matthew Hobbs, head of research at Diabetes UK, said people should not be discouraged from exercising.
He added: "What is clear is that anyone who spends lots of time sitting or lying down would benefit from replacing some of that time by standing or walking.
"Aside from any direct effect reducing the amount of time you spend sitting down may have, getting more physical activity is a great way of helping maintain a healthy weight, which is the best way of minimising your risk of Type 2 diabetes."
BY BBC News