ANSWER
Probably not.
What matters most in custody determinations is the “best interests of the child.” You can argue that infidelity reflects poorly on the parent’s character and you may or may not get a judge who thinks that lesbian infidelity is any more egregious than heterosexual infidelity but in most states there is no difference under the law.
Still, there are many other factors that a Court considers when evaluating the best interests standard. For example, the parent’s demonstrated care of the child and her involvement in the upbringing of the child is probably more demonstrative (than her lesbian affair) of whether or not she is a good physical custodian of the child. A parent’s availability/work schedule is important, the geographic proximity between the parents’ house, which parent is willing to include the other in the child’s life, the physical and emotional health of each parent, et cetera.
A lawyer can explain all the custody factors that a Judge in your jurisdiction would consider and the lawyer can work with you to develop more facts in order to evaluate whether or not sole physical custody is appropriate and in the best interests of your child.