Alex, it is not unusual at this time of year for a flock of hens and young of the year to have few or no gobblers hanging out with them. There may be other gobblers about in a group somewhere that you have not seen. It could also be that the dominant gobblers in the area have harassed the subordinate birds enough that they have left the area, or are just staying away from the main flock.
Your question about the size of the gobblers is an interesting one. How are you making the judgement that the gobblers are all small? Do they appear to be small compared to the other turkeys in the flock?....or have you actually harvested one or more and weighed them? Gobblers should look larger (and darker) than hens, but this time of year, they may not be starting to strut yet, which is when they really look much bigger than the hens.
Regardless, if they are indeed small gobblers, then it could be due to genetics. However, unless that group of birds is somehow isolated from other flocks of turkeys in the area, the likelyhood that it is a genetic thing is pretty small, unless the small size is commonly occurring in the entire region. Same for the "bushy beard" syndrome.
Your speculation that the small size may be the result of nutrional deficiencies has merit, but if the entire flock of birds appears to be healthy otherwise, then I would not suspect that to be a problem either. I would not be too worried about it unless you start seeing indicators of distress in the flock....sick-looking birds, or gradual and significant decreases in the number of birds there.
It would be interesting to find out how big the gobblers are, to see if your observations about size are correct or not. If you get a chance to hunt them this spring, and kill one of those birds, make sure you weigh it and give us a report later.