Python BeautifulSoup在特定标记之后提取文本

Question

我正在尝试使用beautifulsoup和python从网页中提取信息。我想在特定标签下面提取信息。要知道它是否是正确的标签，我想对其文本进行比较，然后在下一个立即标记中提取文本。 比如说，如果以下是HTML页面源的一部分，

<div class="row">
    ::before
    <div class="four columns">
        <p class="title">Procurement type</p>
        <p class="data strong">Services</p>
    </div>
  <div class="four columns">
      <p class="title">Reference</p>
      <p class="data strong">ANAJSKJD23423-Commission</p>
  </div>
  <div class="four columns">
      <p class="title">Funding Agency</p>
      <p class="data strong">Health Commission</p>
  </div>
  ::after
</div>
<div class="row">
    ::before
    ::after
</div>
<hr>
<div class="row">
    ::before
    <div class="twelve columns">
        <p class="title">Countries</p>
        <p class="data strong">
            <span class>Belgium</span>
            ", "
            <span class>France</span>
            ", "
            <span class>Luxembourg</span>
        </p>
        <p></p>
    </div>
    ::after
</div>

我想检查<p class="title">是否有文本值为Procurement type然后我想要打印服务 同样，如果<p class="title">的文本值为Reference，那么我想打印出ANAJSKJD23423-Commission，如果<p class="title">具有Countries的价值，那么打印出所有国家，即比利时，法国，卢森堡。

我知道我可以使用<p class="data strong">提取所有文本并将它们附加到列表中，然后使用索引来获取所有值。但问题是，这些<p class="title>的发生顺序并不固定......在某些地方，可以在采购类型之前提及国家。因此，我想要检查文本值，然后提取下一个立即标记的文本值。我还是BeautifulSoup的新手，所以对任何帮助表示赞赏。谢谢

Answer 1

你可以做很多事。你去吧。

from bs4 import BeautifulSoup
htmldata='''<div class="row">
    ::before
    <div class="four columns">
        <p class="title">Procurement type</p>
        <p class="data strong">Services</p>
    </div>
  <div class="four columns">
      <p class="title">Reference</p>
      <p class="data strong">ANAJSKJD23423-Commission</p>
  </div>
  <div class="four columns">
      <p class="title">Funding Agency</p>
      <p class="data strong">Health Commission</p>
  </div>
  ::after
</div>
<div class="row">
    ::before
    ::after
</div>
<hr>
<div class="row">
    ::before
    <div class="twelve columns">
        <p class="title">Countries</p>
        <p class="data strong">
            <span class>Belgium</span>
            ", "
            <span class>France</span>
            ", "
            <span class>Luxembourg</span>
        </p>
        <p></p>
    </div>
    ::after
</div>'''

soup=BeautifulSoup(htmldata,'html.parser')

items=soup.find_all('p', class_='title')
for item in items:
    if ('Procurement type' in item.text) or ('Reference' in item.text):
        print(item.findNext('p').text)

Answer 2

您还可以将:contains伪类与bs4 4.7.1一起使用。虽然我已经作为列表通过，但您可以将每个条件分开

from bs4 import BeautifulSoup as bs
import re

html = 'yourHTML'   
soup = bs(html, 'lxml')
items=[re.sub(r'\n\s+','', item.text.strip()) for item in soup.select('p.title:contains("Procurement type") + p, p.title:contains(Reference) + p, p.title:contains(Countries) + p')]
print(items)

---

输出：

Answer 3

您可以添加参数以在使用.find()或.find_all()时检查特定文本，然后使用.next_sibling或findNext()来获取包含内容的下一个标记

即：

soup.find('p', {'class':'title'}, text = 'Procurement type')

鉴于：

html = '''<div class="row">
    ::before
    <div class="four columns">
        <p class="title">Procurement type</p>
        <p class="data strong">Services</p>
    </div>
  <div class="four columns">
      <p class="title">Reference</p>
      <p class="data strong">ANAJSKJD23423-Commission</p>
  </div>
  <div class="four columns">
      <p class="title">Funding Agency</p>
      <p class="data strong">Health Commission</p>
  </div>
  ::after
</div>
<div class="row">
    ::before
    ::after
</div>
<hr>
<div class="row">
    ::before
    <div class="twelve columns">
        <p class="title">Countries</p>
        <p class="data strong">
            <span class>Belgium</span>
            ", "
            <span class>France</span>
            ", "
            <span class>Luxembourg</span>
        </p>
        <p></p>
    </div>
    ::after
</div>'''

你可以这样做：

from bs4 import BeautifulSoup     

soup = BeautifulSoup(html, 'html.parser')

alpha = soup.find('p', {'class':'title'}, text = 'Procurement type')
for sibling in alpha.next_siblings:
    try:
        print (sibling.text)
    except:
        continue

输出：

Services

要么

ref = soup.find('p', {'class':'title'}, text = 'Reference')
for sibling in ref.next_siblings:
    try:
        print (sibling.text)
    except:
        continue

输出：

ANAJSKJD23423-Commission    

要么

countries = soup.find('p', {'class':'title'}, text = 'Countries')
names = countries.findNext('p', {'class':'data strong'}).text.replace('", "','').strip().split('\n')
names = [name.strip() for name in names if not name.isspace()]

for country in names:
    print (country)

输出：

Belgium
France
Luxembourg