MSPs backed the bill, to tackle sectarianism with new jail terms of up to five years, by five votes to four.
But Labour, Tory and Lib Dem members refused to support the legislation as it stood.
First Minister Alex Salmond insisted backing the bill was the right thing to do.
The Offensive Behaviour at Football and Threatening Communications Bill was backed by the committee's SNP MSPs, but Labour, Liberal Democrat and Conservative members dissented.
Mr Salmond had hoped to secure the backing of all opposition parties.
The legislation aims to stamp out abusive behaviour, whether fans are watching matches in a stadium, in the pub or commenting online.
In its report, the committee recommended steps to ensure the measures are reviewed after a period of time.
And it also asked ministers to make clear what other action was being taken to tackle sectarianism and how much money was being spent on training police to deal with online hate crime.
The committee also accused football authorities of failing to take firm action to deal with offensive, football-related behaviour.
Continue reading the main storyWe all agree that the football authorities must shoulder some of the blame”End Quote Christine Grahame Justice committee convener "Over many years, they have allowed the issue to drift," concluded the committee report, adding: "If firm action had been taken earlier, offensive behaviour at football might have been stamped out, or at least significantly reduced."
MSPs also said the Scottish Football Association and Scottish Premier League must determine "once and for all" who has authority over disciplinary issues concerning the supporters of SPL clubs.
"We are dismayed that the two bodies do not appear to be close to resolving this issue at a time when clear leadership, and effective joint working, is badly needed," the report concluded.
The publication came after Labour leader Iain Gray said the bill had failed to win the country's support.
Ministers have already delayed the timetable for the bill amid concern from areas such as fans groups and, more recently, a Catholic bishop.Confusion has also surrounded exactly what would constitute a sectarian offence.
During first minister's questions at Holyrood, Mr Gray said: "We accept the need to legislate on sectarianism - that does not mean that any legislation has to be supported. It must garner the support of the country."
He asked: "It has not done that so far. Does the first minister not recognise that, as it stands, this legislation can't be supported?"
Wider debateIn response, Mr Salmond, who was seeking the backing of opposition parties for the bill, said: "All of us have to make a choice.
"We can either be part of the backlash or part of the solution. I truly hope that most people in this parliament decide to be part of the solution."
Leading anti-sectarian charity Nil By Mouth said the legislation was "too narrow" and argued that "less grandstanding" was needed to give way to a wider debate about the problem which was not confined to football.
The bill aims to tackle behaviour that leads to public disorder Strathclyde Police Assistant Chief Constable Campbell Corrigan said the new law was needed to tackle disorder and songs of "hatred, religion and terrorism".
He said: "There is no doubt that several organisations and groups are concerned about the focus that is being placed on football and that a view is being expressed about sectarianism in a wider sense and how prevalent it may or may not be in our society.
"My view on this is quite clear. I support this legislation because it seeks to deal with a problem that we know that we have.
"We do have people behaving inappropriately in and around football matches and we do still hear songs being sung that have nothing to do with football and everything to do with hatred, religion and terrorism."
Scotland's top prosecutor, Lord Advocate Frank Mulholland, previously moved to clarify that behaviour would not be seen as illegal without an "aggravating" factor.
SNP MSP Christine Grahame, convener of the Scottish Parliament justice committee, said "The majority of members support the bill as an important way forward in challenging unacceptable behaviour in 21st Century Scotland.
"A minority are not convinced, but we all agree that the football authorities must shoulder some of the blame.
"Their lack of action in joint working and clear leadership in working out who is in charge of disciplinary issues concerning SPL club fans must be addressed - and swiftly."
Earlier this week the Roman Catholic Bishop of Paisley, Philip Tartaglia, warned Mr Salmond of a "serious chill" in church relations with SNP ministers.
In a letter to Mr Salmond, the bishop said laws to tackle football-related sectarianism were a "distraction".
The first minister will meet the bishop on Friday to talk through the issue.
Meanwhile, the sectarian bill will now move forward to the final stage of parliamentary scrutiny.
