People who say that are generally talking about the signup where you have to pick an instance. And then there’s the worry over which other servers yours federates with. If you isolate your attention to a single instance, then all those worries go away.
The same already happens on the fediverse in regards to mastodon itself. A lot of people discuss the fediverse almost wholly in terms of mastodon.
They are different because most users weren’t aware of XMPP. They weren’t making a conscious choice to use an open standard. The fediverse, on the other hand, has grown specifically because people are seeing the value of an open ecosystem.
When google started removing XMPP support, users weren’t aware and didn’t care (other than losing contact with a few holdouts). If Meta implements AP support and then removes that support or modifies it so that it breaks some of expectations of the fediverse, most users will move to instances that don’t use Meta extensions. Meta can not take your instance or make it use their extensions, so an open fediverse will always exist.