We are talking a country that lives and breaths rugby football competing with countries where rugby union is perhaps 4th or 5th in terms of popular sport interest among "natives", if any at all. I'd say darts is possibly bigger than Union in the UK at present. Many countries where there is no professional rugby presence at all. Rugby Union, as a world concern, gets very little appreciation. Certainly in my quarter of the world. In the USA, the world largest sporting economy, rugby gets effectively zero national interest. Competes with high paying sports like nfl, nhl, nba, mlb to name but just 4. What athlete, in their right mind, would chose to play rugby.
So this is what competes with a country where, upon birth, a rugby ball is place in your hands, and your are told to go and run. Male or female.
Top athletes in other countries do not get the same introduction as New Zealanders do, and if they are determined to be athletic, at an early age, then it's likely to be in a different sport than rugby, to which they are likely never exposed in the first place. While putting a potentially narrow view on the situation, the national sides of many countries are formed with what is essentially the best of the rest, in terms of athletic ability.
Ask yourself what are the top sports in any countries who have a chance of upsetting the ABs in a match. England, Argentina, Ireland, oz, France, etc. Rugby is likely well down the pecking order of sports interest for many of their people. New Zealand, it's the reverse. Everyone plays rugby, and those who do not excel, play something else.
My own rugby experience started out of chance, and against personal wish to play at a footballing "grammar" school in the uk. I was 11 before I even kicked a rugby ball. I already had 8 or 9 years to make upon my New Zealand counterpart. Plus I still had an urge to play football, so my time was split between two sports. Even while playing rugby at a decent level. Even then, All the rugby players I knew had other sports interests. Which might not be so unique in of itself.
In developing rugby nations, like the USA, I bet most rugby players don't start till their teens. And the best will likely face competition from much more lucrative pursuits than rugby union.