There is a lack of knowledge of the influence of parental and offspring autism spectrum disorder (ASD) and/or attention-deficit/hyperactivity disorder (ADHD) symptoms on the quality of family climate. The number of affected children may play an important moderating role. 103 Families were recruited with at least one child with an ASD(+ADHD) diagnosis, one or more biological affected or unaffected siblings, and two participating biological parents. Parents and children were carefully screened for the presence of ASD and ADHD symptoms. Family climate (relationship and structure) was measured with the Dutch Family Environment Scale (FES). No overall differences were found between ASD, ASD + ADHD families and a norm group for family relationship and family structure. However, families with one affected child reported higher family relationship scores than the norm, whereas fathers and mothers of families with two or three affected children reported similar scores as the norm. Regarding the family structure index, fathers and mothers of families with one, two or three affected children reported similar scores as the norm data of the FES. As reported by both fathers and mothers, paternal and maternal ASD symptoms and paternal ADHD symptoms had a negative effect on family relationships, whereas predominately maternal ADHD had a negative impact on family structure. However, families with two or more affected children and families in which parents have increased ASD/ADHD symptoms may have more difficulties in maintaining a positive family climate.