AI crawler access is not one switch
Separate controls for search discovery, possible model-training use and visits triggered by a user. The available controls and their effects differ by provider.
Inside Googlebot: demystifying crawling, fetching, and the bytes we process | Google Search Central Blog | Google for Developers
Google Search Centraldevelopers.google.com