I guess it's a bummer that you can't decide on the spur of the moment to take Monday and Friday off and have a long weekend somewhere.
But if you know far enough in advance, I don't see what the problem is with booking your holidays to fit those dates - unless they make you take them in school half term and summer holidays when they're stupendously expensive? In which case, surely colleagues with children would be more than happy to swap.
It would also be a bummer though if you wanted time off to attend a wedding or birthday or festival or some other fixed date event, but surely as PuddyTat says it should be possible to swap?