Scraping X is harder in 2026 than it was a few years ago. Logged-out access is mostly closed, X's terms forbid scraping without written consent, and the popular open-source scrapers either stopped working or now need real X accounts to run. You can still collect public data at small scale if you stay slow and within the rules. For anything regular or large, an API is usually less work.

We run TwittrAPI, a third-party API for public X data, so we have an interest in the second half of that sentence. The facts below come from X's own pages, court reporting and the scrapers' repositories, all linked under Sources and checked on 2026-10-10. None of this is legal advice.

What X's rules say about scraping

X's Terms of Service (the version effective October 9, 2026) say you may not access, crawl or search the services by any means other than X's published interfaces, unless a separate agreement with X allows it. A note adds that crawling or scraping in any form without prior written consent is expressly prohibited.

The terms also contain a liquidated damages clause. Anyone who breaks the terms, or helps others to, while requesting, viewing or accessing more than 1,000,000 posts in any 24-hour period is liable for $15,000 per 1,000,000 posts. The EU version states the amount in euros. According to the Knight First Amendment Institute, the clause first appeared in the terms that took effect on November 15, 2024.

robots.txt blocks all crawlers by default

x.com/robots.txt disallows everything for User-agent: *. Named search engine crawlers get their own rules: Googlebot and Bingbot may crawl most pages but not follower, following, likes or search pages. Several AI and social crawlers are blocked completely. A scraper that follows robots.txt has nothing to crawl.

Why scrapers get blocked

Three changes in 2023 broke most logged-out scraping, and none has been reversed in a way that helps scrapers.

The login wall. At the end of June 2023 Twitter started showing logged-out visitors a login prompt instead of posts, as TechCrunch reported.

Read limits per account. On July 1, 2023 Elon Musk announced daily limits on how many posts an account could read: 6,000 for verified accounts, 600 for unverified and 300 for new unverified accounts. He raised them the same day to 10,000, 1,000 and 500. He blamed "extreme levels of data scraping" (same TechCrunch report). Limits like this are why account-based scrapers spread their requests over many accounts.

Guest access removed. The widely used snscrape depended on logged-out access. Its issue "All Twitter scrapes are failing: blocked (404)" was opened on June 30, 2023 and is still open and pinned.

What still works, and what it costs you

Account-based scrapers

Tools such as twscrape still work by using logged-in X accounts. Its README says you need authorized accounts, either cookies (auth_token and ct0) or a login. It tracks rate limits per account and per endpoint and switches to another account when one runs out. It also notes that user timelines stop at about 3,200 tweets.

That is a working setup, with costs:

  • You need X accounts, and the scraping happens under their names. The terms above apply to those accounts.
  • Every limit is per account, so throughput depends on how many accounts you run.
  • X's internal endpoints are not a public interface. When they change, the tool breaks until someone fixes it.

Browser automation

Driving a real browser with Playwright or Selenium gets you whatever a logged-in user sees, at the speed a user sees it. It runs into the same login wall, the same per-account read limits and the same terms. It is also slow and heavy to run.

What we do not cover

Guides to scraping X often move on to rotating proxies, fake browser fingerprints and solving CAPTCHAs. Those exist to get around X's controls, which is what the terms prohibit, and it is an arms race you pay for in maintenance. We leave them out.

How to scrape without getting blocked

If you scrape anyway, at small scale, these keep you out of most trouble:

  1. Stay small and slow. A few hundred posts read at a human pace is a different thing from millions an hour. The $15,000 clause starts above 1,000,000 posts in 24 hours.
  2. Only public data. Never collect protected accounts, direct messages or anything behind a login you were not given.
  3. Cache everything. Do not fetch the same profile or post twice in a day.
  4. Stop on errors. A 429, a login prompt or an empty page means slow down or stop, not retry harder.
  5. Check the law where you are. Collecting personal data has rules of its own, separate from X's terms.

Two cases come up whenever scraping is discussed. Neither settles whether your project is allowed.

hiQ Labs v. LinkedIn. In April 2022 the Ninth Circuit held again that scraping publicly accessible pages is likely not access "without authorization" under the Computer Fraud and Abuse Act, even after a cease-and-desist letter (Fenwick summary). The ruling does not cover data behind a login, breach of contract or terms of service claims, copyright or trespass claims.

X Corp. v. Bright Data. In May 2024 Judge William Alsup dismissed X's case against the data company Bright Data. X had not alleged damage from the scraping, and its claims about the sale of public posts were preempted by copyright law (MediaPost). In November 2024 he let X refile its claims that scraping had impaired its servers (MediaPost). The parties settled in June 2025 on confidential terms (Bloomberg Law).

The short version: in the US, scraping public pages is not automatically a crime, but contract claims under the terms of service remain possible, and the terms now name a price.

When to use an API instead

Use an API when any of these is true:

  • You need data every day, not once.
  • You need more than a few thousand posts or profiles.
  • You do not want to manage X accounts, sessions and breakage.
  • Someone else (a client, a compliance team) needs to know where the data came from.

There are two kinds of API.

The official X API. You sign up at console.x.com and pay per use: $0.005 per post read and $0.010 per user read, with a cap of 3 million post reads per month on pay-per-usage. It is the only route that X supports, and the only one for posting or acting for users. X API pricing in 2026 has the details, and Twitter API rate limits explained covers its request limits.

Third-party APIs for public data. Providers such as TwittrAPI, twitterapi.io and SocialData run the collection and sell access to public data through their own REST endpoints and keys. You do not need an X developer account, X accounts or scraping infrastructure. They are not affiliated with X, they only return public data, and you should check that using one fits your own compliance needs. The cheapest Twitter API in 2026 compares their published prices.

Scraping yourselfOfficial X APIThird-party API
X's termsProhibited without consentAllowedCheck the provider and your use case
SetupAccounts, sessions, toolsDeveloper account, app, creditsProvider account, key
Breaks when X changesYes, you fix itX announces changesThe provider fixes it
Price for public readsYour time and infrastructure$5 per 1,000 postsCents per 1,000 calls or items
PostingNot covered hereYesUse the official API

Hosted scrapers sit in between: on Apify, independent developers publish Twitter actors that you run as jobs and pay per result. TwittrAPI vs Apify Twitter scrapers compares their prices, minimums and limits with an API.

With TwittrAPI a request is one HTTP call with a key header. The documentation shows the requests, the pricing page shows rates from $0.065 per 1,000 calls, and /start gives you a key with $0.10 of test credit. How to get an X API key without a developer account walks through it. Building a research dataset? Collecting X data for research has a Python collector and costs per million tweets. Only need a quick look? The free tools export an account's latest tweets to CSV, unroll a thread or find a user ID without code.

Frequently asked questions

It depends on what you collect, how and where you are. X's terms prohibit scraping without written consent and set liquidated damages above 1,000,000 posts in 24 hours. In the US, the hiQ v. LinkedIn ruling says scraping public pages is likely not a crime under the CFAA, but that does not rule out contract claims. Get legal advice for anything beyond personal research.

Can you scrape Twitter without logging in?

Very little. Since mid-2023 X shows logged-out visitors a login prompt, and robots.txt disallows all crawlers that are not specifically allowed. Tools that relied on logged-out access, such as snscrape, have been failing since June 2023.

Does snscrape still work for Twitter?

Not for Twitter. The snscrape issue reporting that all Twitter scrapes fail was opened on June 30, 2023 and is still open.

What is the safest way to get Twitter data?

The official X API is the route X supports. For reading public data at lower cost, a third-party API spares you the accounts and maintenance that scraping needs. Either way, collect only what you need and check the rules that apply to you.

Sources