Do you think this is a bad idea?

So, we're moving on Saturday, so we will no longer have access to the swimming pool in our apartment complex. We haven't been swimming for months, and we kinda wanted to go swimming again before we move. We were planning on going swimming a month ago, but I had a baby on the day we were planning to go, and I've been bleeding ever since. I've finally stopped now, so we want to go swimming! It's an indoor heated pool, so we don't have to worry about it being too cold or anything. My bigger concern is our 4 week old. I don't really want to take him in the water, because I think he's far too young for that, but I know that my 17 month old would Love to go swimming. My husband can play with our older son in the water while I sit out with the baby, but I know all that he really wants to do is sit in the hot tub, and I'd kinda like to go swimming too. So would it be really unreasonable to put the baby in his carseat or his bouncer chair or something, and put him by the pool... not close enough that it's a hazard that he might fall in, but close enough that my husband who would be sitting in the hot tub could watch him and get to him quickly if there were a problem? That way I could be in the pool with our 17 month old. It's never crowded in there, more than likely there will be no one else in there but us. ... It makes sense when I think about it that way, but it also just kinda seems like a bad idea having us all in the water with the baby out by himself... what do you think?