If you're both grownups and in good head space, it doesn't matter what anyone else thinks is right or wrong. Hang out and see how it goes.