The Prince of Wales spoke of the Royal Family's love of swimming while visiting a pool previously threatened with closure.