Meta's AI agents ran 1,097 experiments in 31 days to lift search activity by 0.23%

Meta's AI agents ran 1,097 experiments in 31 days to lift search activity by 0.23%

AI agents ran Meta's search experiments for a month

Meta researchers let AI agents run 1,097 machine-learning experiments in 31 days. The effort used about 360,000 GPU-hours and produced a model that raised search-session volume by 0.23% in a live test.

The work, described in a research paper published Nov. 18, shows how autonomous AI systems can develop and test improvements to large-scale recommendation models with limited human involvement. But the new model also made Meta's search interface slower to render, which raises questions about whether the small gain justified the computing cost.

The numbers behind the campaign

  • 1,097 candidate experiments completed in 31 days; 325 were accepted, a 29.6% acceptance rate.
  • An earlier version of the system, which ran tests one after another instead of in parallel, had a 15.6% acceptance rate.
  • About 360,000 GPU-hours were consumed, including variance probes and failed or retried jobs.
  • The live A/B test showed a relative 0.23% increase in search-session volume, with a 95% confidence interval of plus or minus 0.15 percentage points.
  • The search interface took longer to render after the change; the size of the slowdown was not disclosed.

What the research paper says

The system let AI agents propose changes to machine-learning models, write code, submit training jobs, and screen offline results. Engineers set the objective, steered the search, and reviewed the combined model before it was tested on live traffic.

The target was Meta's search-suggestion system, which decides what recommendations users see when they open the search bar before typing. Normally, improving such a system requires engineers to find changes, build them, and test them through repeated training cycles. The agents handled much of that work and ran many experiments at the same time.

The agents used Anthropic's Claude Opus 4.8 to explore architectural improvements and generate candidate changes. The researchers said the agents found several architectural improvements Meta's engineering team had not previously explored.

What the live test showed

After combining the accepted changes into one model, Meta tested it against its existing search-ranking system using live traffic. Search-session volume rose 0.23%. Other engagement measures, including participation and retention, moved in a positive direction.

The paper notes that Meta handles more than a billion search-ranking requests daily, so even a small improvement in user activity can matter at that scale.

However, the new model increased rendering time for the search interface. The researchers said this happened because the agents optimized against offline measures of search engagement, while rendering speed was monitored separately. They acknowledged the system could not independently catch every trade-off between its assigned targets and other parts of the user experience.

What is confirmed

These facts come from Meta's research paper as reported by CryptoSlate:

  • Agents completed 1,097 experiments in 31 days and 325 were accepted based on offline scores.
  • The final combined model was tested on live traffic and increased search-session volume by a relative 0.23%.
  • The same test showed a slowdown in how long the search surface took to render.
  • Meta estimates roughly 360,000 GPU-hours for the campaign.
  • Engineers kept control over research priorities and final deployment decisions.

What is still unclear

Meta did not disclose the financial cost of the campaign or compare the resources used with what a conventional engineering team would need. The paper also does not resolve whether the model will be permanently rolled out, and it does not include a matched comparison with human-only engineering.

The researchers argued the results show autonomous agents can do meaningful machine-learning engineering work on production-scale systems. Meta still has to decide whether the added search activity and engineering productivity justify the computing resources and slower interface before wider adoption.

Sources

Newisty Editorial Team
Written by

Newisty Editorial Team

Technology · Crypto · Digital Economy
View all posts

Newisty Editorial Team covers technology, cryptocurrency, digital products, online platforms, developer tools and the wider digital economy. Our content is researched from official sources, company announcements, public documentation, market data and other primary or reputable sources. Articles are reviewed and edited before publication for clarity, accuracy and useful context.

Comments (0)

Leave a comment
Your comment will appear publicly after submission.
No comments yet. Be the first to comment!