It all depends on context IMO. If you are presenting a photograph in a factual context (ie a newspaper) then adding/removing objects/people etc from the image is misleading.
On the other hand, if you are presenting a photograph as art, in my book you can do whatever the hell you like.