To be fair, it’s not a majority of football fans per se- not in my experience at least. But then I follow a small club with no hooligan element. Of course some individuals can still get overtly pissed and become an annoyance to fellow passengers, but for the most part it is not a widespread problem for many clubs.
And on those occasions when some fans have felt like singing throughout the journey back to London, which IME has always been good humoured and non-threatening (though still very annoying to some of the passengers in the vicinity, I suspect), it invariably tends to happen on one carriage where all the fans in that frame of mind end up congregating. Rather than happening on every carriage.
Furthermore, there will usually be fans of a number of other clubs on the same train, and I have never seen any bother, let alone fighting between sets of fans. I’m sure it does happen, but it’ll be the exception rather than the norm.
So in short, not all (or even the majority) of travelling football fans are problematic- but such incidents tend to be higher profile or stick in the mind more than others. I have seen far more instances on a per-journey basis of incidents, fights and a general feeling of hostility on Friday and Saturday nights from ordinary ‘revellers’ on London buses, the Tube and late evening trains than from football fans on weekend train journeys.