My main problem with series 4 is there's a Livia shaped hole in the story. Livia Soprano is my favourite character in all TV and her relationship with Tony is a big part of the first three series. I don't think they found anything to replace that element of the story in that series and it felt like something was lacking.
Then in some of the episodes in series 6 I found there were some clunky plot contrivances to make the stories that the show's creaters wanted to tell work, especially around emphasising the negative aspects of Tony's character.
Like the episode where Tony's gambling problem runs out of control revealing him to be a massive hypocrite and a bigger cunt to Hersh. But.. wait... gambling problem??? Where did that come from? Sure, he liked cards and hanging out at casinos, but it had never been hinted he had a gambling problem before and after the episode finished it was never referred to again.
Or when he goes to Las Vegas, revealing himself to be a selfish prick and hypocrite after he killed Chris. He goes to tell Chris' improbably beautiful ex, who'd been really close to Chris, about him dying and... wait a second... where did she come from? How come she'd never been mentioned in the previous 6 years if she was so close to Chris? When was she close to him? Of course, being improbably beautiful she falls straight into bed with Tony, who happlily takes drugs with her, which he'd always wanted to, even though that had never been suggested before.
Things like that were the kind of thing that the show hadn't done before. None of which takes away from the overall awesomeness of the show. And that ending was perfect.