As documented, the Topics API has no particular minimums set about how many page visits/unique topic visits a user needs to have before they are classified. To prevent users from being classified into Topics with too little data, minimums should be put in place. For example:

In a week, if the user has not had at least 50 page visits, they can not be classified into any topics.
In a week, if the user has not had at least 10 page visits to a particular topic, they should not be classified into that topic

Those might not be exactly right, but some minimum usage component should be built into the system.