I can often correctly guess a lot about the nationalities, ethnicities, or genders of the organizers from the speaker list. Sometimes I can even correctly guess names of organizers.
I recommend playing the game. How much about the conference organizers can you guess from the list of speakers?
There are times when I've asked an organizer why the list of invited speakers is all male, and his reply is that the women in the field aren't good enough, the men are just better. I've gotten similar responses when a speaker (and organizer) list is disproportionately Dutch, or French, or of a particular ethnicity.
At a certain Ivy League university in the 1990s, the junior faculty attended a meeting where the (all-male) senior faculty decided which undergraduates would graduate summa cum laude in mathematics. Afterwards, some of the junior faculty told me they were upset and concerned because a female student with high grades in hard courses was passed over in favor of a male student with lower grades, after a senior faculty member said that the man reminded him of himself at that age.
People choose people who remind them of themselves. Then they rationalize it by saying that such people are better.
I like merit-based systems, and I'm not advocating for quotas. And if financial constraints mean that local speakers are preferred over those with more expensive travel costs, that's understandable. But sometimes it helps to be reminded to give full consideration to people different from oneself or one's friends. I hope that things have improved, and that the organizer-guessing game isn't as easy as it used to be.
The game has a second part. If the speaker and organizer lists are skewed in the same direction, ask yourself whether the argument that the over-represented group is just better feels right to you. If it does, do you belong to the favored group?