Search AI Products and News

Apple encountered website blocking challenges when launching its AI features and needs to negotiate licensing agreements with major sites. Google, leveraging its market influence, is able to pressure publishers to allow AI access to content, whereas Apple faces more website blocks on its AI crawler, Applebot-Extended. 7% of websites have chosen to block this crawler, with data showing that 294 major websites have restricted access. OpenAI also faces similar challenges, with a higher blocking rate for its crawler. Google has strengthened its market position by combining AI summaries with traditional web scraping.

Recently, Wired magazine reported that Apple's content-scraping bot, Applebot, has faced collective resistance from multiple major media outlets, including well-known platforms such as Facebook, Instagram, The New York Times, and The Financial Times, which have chosen to block Applebot and deny it access to their website content. This phenomenon reflects the complex situation in the field of AI content scraping and the media industry's complicated attitude towards AI technology. Research shows that approximately 6% to 7% of websites have blocked Applebot, while up to