The crucial distinction that must be made here is the difference between sex and gender. Sex is a biological term referring to the chromosomal makeup of an individual; gender refers to the characteristics associated with those sexes. It is reasonable to say that there are as many genders as people.
A classification of gender is a classification of a generalised societal role. Generalised societal roles derive from the needs of a society. These needs of a society are demanded by a specific biological function. Rejecting a generalised societal role is rejecting social gender classification.
Being a product of it, gender is as fickle as the mind. We don't yet fully know how the mind works, so it'd be strange to claim with absolute certainty that there are only two genders.