从instagram获得href的方法

问题描述 投票：-1回答：2

有没有办法从'https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/'获得'a href'，除了硒？在api的帮助下，我只能得到这种类型的图片链接：https://instagram.fhel6-1.fna.fbcdn.net/vp/b6c669ed3b5be0dc9c183412d738acac/5CEC3935/t51.2885-15/e35/c119.0.842.842/s240x240/49787501_1587577534678419_6308372780046107029_n.jpg?_nc_ht=instagram.fhel6-1.fna.fbcdn.net我不需要这个。我想得到这样的链接'https://www.instagram.com/p/BuGpLWsFioq/'。我正在尝试使用bs4和'lxml'解析器来执行此操作，但在html中使用NO'a href'得到结果。我需要知道是否有可能刮掉这些信息？很明显，javascript会生成更多信息。因此，除了selenium-webdriver之外，它是一种刮除这些数据的方法吗？

python api beautifulsoup instagram
2个回答
1
投票

您要查找的所有信息都在<script type=text/javacript>内

您可以使用以下正则表达式获取它：

from bs4 import BeautifulSoup as soup
import requests
import json
import re

def _get_json_footer(html):
    s = str(html)
    r = re.compile('"entry_data":(.*?),"gatekeepers"')
    m = r.search(s)
    if m:
        result = m.group(1)
    return json.loads(result)

url = 'https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/'
page = requests.get(url)
html = soup(page.text, 'html.parser')
json_footer = _get_json_footer(html)

tagpage = json_footer.get('TagPage')

然后，您可以在tagpage dict中导航以获取数据

编辑：

要获得帖子链接，您可以在tagpage词典中导航：

from bs4 import BeautifulSoup as soup
import requests
import json
import re

def _get_json_footer(html):
    s = str(html)
    r = re.compile('"entry_data":(.*?),"gatekeepers"')
    m = r.search(s)
    if m:
        result = m.group(1)
    return json.loads(result)

url = 'https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/'
page = requests.get(url)
html = soup(page.text, 'html.parser')
json_footer = _get_json_footer(html)

tagpage = json_footer.get('TagPage')

links = []
edges = tagpage[0].get('graphql',{}).get('hashtag',{}).get('edge_hashtag_to_media',{}).get('edges',[])
for e in edges:
    links.append("https://www.instagram.com/p/"+e.get('node',{}).get('shortcode','')+'/')

print(links)

OUTPUT：

['https://www.instagram.com/p/Bsh4UcdBRvY/', 'https://www.instagram.com/p/Bq8vAMRHtGB/', 'https://www.instagram.com/p/Bn_vfeWhcYL/', 'https://www.instagram.com/p/Bm1QRb2ntWL/', 'https://www.instagram.com/p/Bj5pLHAnVuY/', 'https://www.instagram.com/p/Bfn2QWiHKK5/', 'https://www.instagram.com/p/BfC4ZnTntq0/', 'https://www.instagram.com/p/BeomaB6Hb8-/', 'https://www.instagram.com/p/vYszwjyLdB/', 'https://www.instagram.com/p/sQI6Jfpi3f/', 'https://www.instagram.com/p/sO9oXPMr6K/', 'https://www.instagram.com/p/qzvHuCHUgH/', 'https://www.instagram.com/p/WdlKcCBW3w/']

您可以通过edge_hashtag_to_media更改密钥edge_hashtag_to_top_posts以获取其他值

0
投票

让我知道这就是你要照顾的。

from bs4 import BeautifulSoup
import requests
resp=requests.get("https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/")
html = resp.content
soup = BeautifulSoup(html,'html.parser')


for a in soup.find_all('link',rel='alternate',href=True):
    print "Found the URL:", a['href']

输出：

Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=en
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=fr
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=it
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=de
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=es
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=zh-cn
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=zh-tw
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=ja
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=ko
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=pt
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=pt-br
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=af
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=cs
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=da
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=el
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=fi
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=hr
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=hu
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=id
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=ms
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=nb
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=nl
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=pl
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=ru
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=sk
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=sv
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=th
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=tl
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=tr
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=hi
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=bn
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=gu
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=kn
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=ml
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=mr
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=pa
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=ta
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=te
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=ne
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=si
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=ur
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=vi
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=bg
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=fr-ca
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=ro
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=sr
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=uk
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=zh-hk
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=es-la
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=es-la
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=es-la
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=es-la
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=es-la
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=es-la
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=es-la
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=es-la
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=es-la
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=es-la
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=es-la
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=es-la
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=es-la
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=es-la
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=es-la
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=es-la
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=es-la
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=es-la
Found the URL: https://www.instagram.com/explore/tags/SOMEHASHTAGHERE/?hl=es-la
最新问题
© www.soinside.com 2019 - 2024. All rights reserved.