使用Python snscrape twitter

Question

我刚刚得到一条信息，sncrape python 仍然错误，我尝试通过在 twi***scraper 行中添加 top = True 来解决它，但它仍然错误。 这是我的代码：

pd.options.display.max_colwidth = 500
query = "(music) lang:en  since:2023-01-01 until:2023-02-02" 
tweets = []
limit = 10
get_ipython().run_line_magic('time', '')
try:
    print("start scraping")
    for tweet in sntwitter.TwitterSearchScraper(query=query, top = True).get_items():
        if len(tweets) == limit :
            break
        else:
            tweets.append([tweet.date, tweet.user.username, tweet.content])
    df = pd.DataFrame(tweets, columns=['datetime', 'username', 'content'])
except Exception as e:
    print(e)

print("Finished")
print("-------")

有人可以解决这个错误吗？

Error retrieving https://api.twitter.com/2/search/adaptive.json?include_profile_interstitial_type=1&include_blocking=1&include_blocked_by=1&include_followed_by=1&include_want_retweets=1&include_mute_edge=1&include_can_dm=1&include_can_media_tag=1&skip_status=1&cards_platform=Web-12&include_cards=1&include_ext_alt_text=true&include_quote_count=true&include_reply_count=1&tweet_mode=extended&include_entities=true&include_user_entities=true&include_ext_media_color=true&include_ext_media_availability=true&send_error_codes=true&simple_quoted_tweets=true&q=%28Gempa%29+lang%3Aid++since%3A2023-01-01+until%3A2023-02-02&count=100&query_source=spelling_expansion_revert_click&pc=1&spelling_corrections=1&ext=mediaStats%2ChighlightedLabel: non-200 status code
4 requests to https://api.twitter.com/2/search/adaptive.json?include_profile_interstitial_type=1&include_blocking=1&include_blocked_by=1&include_followed_by=1&include_want_retweets=1&include_mute_edge=1&include_can_dm=1&include_can_media_tag=1&skip_status=1&cards_platform=Web-12&include_cards=1&include_ext_alt_text=true&include_quote_count=true&include_reply_count=1&tweet_mode=extended&include_entities=true&include_user_entities=true&include_ext_media_color=true&include_ext_media_availability=true&send_error_codes=true&simple_quoted_tweets=true&q=%28Gempa%29+lang%3Aid++since%3A2023-01-01+until%3A2023-02-02&count=100&query_source=spelling_expansion_revert_click&pc=1&spelling_corrections=1&ext=mediaStats%2ChighlightedLabel failed, giving up.

Answer 1

看来您的代码是正确的（尽管您的代码有一些警告，例如使用

mode = snscrape.modules.twitter.TwitterSearchScraperMode.TOP

代替

top = True

以及使用

rawContent

代替

content

。但我也遇到了同样的错误。当我使用伊朗 ISP（Twitter 在伊朗被政府禁止，我发现它与您的网络和 ISP 有关）我使用了 VPN（您也可以在可以访问免费互联网的服务器上运行您的代码）。已修复。

Answer 2

要抓取热门推文，请执行以下操作（因为 top=True 已弃用）：

import snscrape.modules.twitter as sntwitter

然后

mode_param = sntwitter.TwitterSearchScraperMode.TOP
(sntwitter.TwitterSearchScraper(search_query, mode = mode_param)