It's society's business just as it is society's business when anyone gets married. The state has a right to regulate marriage in order to promote the common good, which includes fostering a culture that will encourage strong marriages that produce children in order that the society continues. Is it an absolute right to legislate every aspect of bhavior associated with marriage? Obviously not. But neither is it a matter of privacy since the family is the building block of society.
And yet a society that is somewhat slow to permit gay marriage permits gay couples to adopt children in order that the society continues, and the vast majority of states do so.
Societies change, Chris.