Thankfully I married a boater so weekends = boating.
Our first child was in the boat at 5 days old and has always been keen. Both kids (6 & 9) can't get enough.
Would agree that non-boaters just don't get it. We are gladly selfish about our boat time and just say no to events that would eat up precious summer weather.