bs4 抓取无输出

Question

我是一个初学者，希望使用 bs4 来抓取图像中突出显示的数字，但结果我一直收到一个空列表。

import requests
from bs4 import BeautifulSoup

url = "https://www.set.or.th/en/market/product/stock/quote/KBANK/news"
response = requests.get(url)
soup = BeautifulSoup(response.content, "html.parser")

reports = soup.find_all('span', {'class' : 'd-none d-md-block fs-20px heading-font-family'})

print(reports)

截至 2024 年 8 月 20 日，我预计报告列表仅包含数字 316。

Answer 1

该网站从不同的地址动态加载新闻源。我还没有调查它提供错误数量的新闻项目的原因

import requests

url = "https://www.set.or.th/api/set/news/search?symbol=KBANK&fromDate=20%2F08%2F2019&toDate=20%2F08%2F2024&keyword=&lang=en"
headers = {
    'accept': 'application/json, text/plain, */*',
    'cookie': 'charlot=299a8d18-c22b-4436-a8b1-deb934022bb7;',
    'priority': 'u=1, i',
    'referer': 'https://www.set.or.th/en/market/product/stock/quote/KBANK/news',
    'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36'
}

response = requests.get(url, headers=headers, )
print('Total news: ', response.json()['totalCount'])
for news in response.json()['newsInfoList']:
    print(news['headline'], news['url'])

输出：

Total news:  356
Dissolution of a Subsidiary (K-SME Venture Capital Company Limited) https://www.set.or.th/en/market/news-and-alert/newsdetails?id=90886900&symbol=KBANK
Dissolution of a Subsidiary (KAPTURE ONE COMPANY LIMITED) https://www.set.or.th/en/market/news-and-alert/newsdetails?id=90886800&symbol=KBANK
Appointment of Director of KASIKORNBANK PCL https://www.set.or.th/en/market/news-and-alert/newsdetails?id=90111100&symbol=KBANK
Summary Statement of Assets and Liabilities as of 30 June 2024 https://www.set.or.th/en/market/news-and-alert/newsdetails?id=89990700&symbol=KBANK
Summary of the Bank and its subsidiaries' operating performance for the second quarter and six-month period ended 30 June 2024 https://www.set.or.th/en/market/news-and-alert/newsdetails?id=89994800&symbol=KBANK
Submission of the unaudited financial statements for the second quarter and six-month period ended 30 June 2024 https://www.set.or.th/en/market/news-and-alert/newsdetails?id=89991200&symbol=KBANK
Financial Performance Half Year For Financial Institution (F45) (Unaudited) https://www.set.or.th/en/market/news-and-alert/newsdetails?id=89991000&symbol=KBANK
Financial Statement Half Year 2024 (Unaudited) https://www.set.or.th/en/market/news-and-alert/newsdetails?id=17213457254820&symbol=KBANK
Summary Statement of Assets and Liabilities as of 31 May 2024 https://www.set.or.th/en/market/news-and-alert/newsdetails?id=89728800&symbol=KBANK
Notification of Establishment of KCLIMATE 1.5 COMPANY LIMITED https://www.set.or.th/en/market/news-and-alert/newsdetails?id=89688400&symbol=KBANK
Summary Statement of Assets and Liabilities as of 30 April 2024 https://www.set.or.th/en/market/news-and-alert/newsdetails?id=89382100&symbol=KBANK
Management Discussion and Analysis Quarter 1 Ending 31 March 2024 https://www.set.or.th/en/market/news-and-alert/newsdetails?id=89071000&symbol=KBANK
Financial Performance Quarter 1 (F45) (Reviewed) https://www.set.or.th/en/market/news-and-alert/newsdetails?id=89070100&symbol=KBANK
Financial Statement Quarter 1/2024 (Reviewed) https://www.set.or.th/en/market/news-and-alert/newsdetails?id=17156434556490&symbol=KBANK
Submission of the Reviewed Financial Statements for the three-month period ended 31 March 2024 and the Management Discussion and Analysis (MD&A) https://www.set.or.th/en/market/news-and-alert/newsdetails?id=89030400&symbol=KBANK
Announcement of Basel III - Pillar 3 Disclosures for the period ended 31 December 2023 https://www.set.or.th/en/market/news-and-alert/newsdetails?id=88314500&symbol=KBANK
Change of Board Structure of KASIKORNBANK, Resignation of Non-Executive and Executive Directors, Appointment of Presidents and Authorized Signatory https://www.set.or.th/en/market/news-and-alert/newsdetails?id=88193800&symbol=KBANK
Dissemination of Minutes of the General Meeting of Shareholders No.112 https://www.set.or.th/en/market/news-and-alert/newsdetails?id=88151300&symbol=KBANK
Summary Statement of Assets and Liabilities as of 31 March 2024 https://www.set.or.th/en/market/news-and-alert/newsdetails?id=88076600&symbol=KBANK
Summary of the Bank and its subsidiaries' operating performance for the first quarter ended 31 March 2024 https://www.set.or.th/en/market/news-and-alert/newsdetails?id=88077000&symbol=KBANK
Financial Performance Quarter 1 (F45) (Unreviewed) https://www.set.or.th/en/market/news-and-alert/newsdetails?id=88076900&symbol=KBANK
Financial Statement Quarter 1/2024 (Unreviewed) https://www.set.or.th/en/market/news-and-alert/newsdetails?id=17137422576380&symbol=KBANK
Submission of the unreviewed financial statements for the first quarter ended 31 March 2024 https://www.set.or.th/en/market/news-and-alert/newsdetails?id=88077100&symbol=KBANK
FORM TO REPORT ON NAMES OF MEMBERS AND SCOPE OF WORK OF THE AUDIT COMMITTEE https://www.set.or.th/en/market/news-and-alert/newsdetails?id=87983300&symbol=KBANK
Notification of Resolution of the General Meeting of Shareholders No. 112 https://www.set.or.th/en/market/news-and-alert/newsdetails?id=87989800&symbol=KBANK
Summary Statement of Assets and Liabilities as of 29 February 2024 https://www.set.or.th/en/market/news-and-alert/newsdetails?id=87254900&symbol=KBANK
Dissemination of Notice of the General Meeting of Shareholders No. 112 https://www.set.or.th/en/market/news-and-alert/newsdetails?id=87064700&symbol=KBANK
Management Discussion and Analysis Yearly Ending 31 December 2023 https://www.set.or.th/en/market/news-and-alert/newsdetails?id=86748800&symbol=KBANK
Financial Performance Yearly (F45) (Audited) https://www.set.or.th/en/market/news-and-alert/newsdetails?id=86748600&symbol=KBANK
Financial Statement Yearly 2023 (Audited) https://www.set.or.th/en/market/news-and-alert/newsdetails?id=17091569110160&symbol=KBANK
Submission of the Audited Financial Statements for the year ended 31 December 2023 and the Management Discussion and Analysis (MD&A) https://www.set.or.th/en/market/news-and-alert/newsdetails?id=86747200&symbol=KBANK
Dividend Payment, Schedule and Agenda for the General Meeting of Shareholders No. 112 https://www.set.or.th/en/market/news-and-alert/newsdetails?id=86700700&symbol=KBANK
Summary Statement of Assets and Liabilities as of 31 January 2024 https://www.set.or.th/en/market/news-and-alert/newsdetails?id=86123500&symbol=KBANK
KASIKORNBANK's 2024 Financial Targets https://www.set.or.th/en/market/news-and-alert/newsdetails?id=85695000&symbol=KBANK
Summary of the Bank and its subsidiaries' operating performance for the fourth quarter and for the year ended 31 December 2023 https://www.set.or.th/en/market/news-and-alert/newsdetails?id=85605600&symbol=KBANK
Summary Statement of Assets and Liabilities as of 31 December 2023 https://www.set.or.th/en/market/news-and-alert/newsdetails?id=85605200&symbol=KBANK
Financial Performance Yearly (F45) (Unaudited) https://www.set.or.th/en/market/news-and-alert/newsdetails?id=85605500&symbol=KBANK
Submission of the unaudited financial statements for the year ended 31 December 2023 https://www.set.or.th/en/market/news-and-alert/newsdetails?id=85605700&symbol=KBANK
Financial Statement Yearly 2023 (Unaudited) https://www.set.or.th/en/market/news-and-alert/newsdetails?id=17056215357060&symbol=KBANK
...
Unaudited Yearly financial performance (F45) https://www.set.or.th/en/market/news-and-alert/newsdetails?id=2020005788&symbol=KBANK
Financial Statement Yearly 2019 (Unaudited) https://www.set.or.th/en/market/news-and-alert/newsdetails?id=2020005787&symbol=KBANK
Summary of the Bank and subsidiaries' operating performance for Q4 and the year ended 31 December 2019 https://www.set.or.th/en/market/news-and-alert/newsdetails?id=2020005790&symbol=KBANK
Submittal of Financial Statements of Q4 and the year ended 31 December 2019 (Unaudited) https://www.set.or.th/en/market/news-and-alert/newsdetails?id=2020005784&symbol=KBANK
CLARIFICATION OF NEWS REPORTS ON KBANK'S JOINT INVESTMENT WITH AYEYARWADDY FARMERS DEVELOPMENT BANK https://www.set.or.th/en/market/news-and-alert/newsdetails?id=2020003571&symbol=KBANK
Summary Statement of Assets and Liabilities as of 30 November 2019 https://www.set.or.th/en/market/news-and-alert/newsdetails?id=2019123146&symbol=KBANK
Summary Statement of Assets and Liabilities as of 31 October 2019 https://www.set.or.th/en/market/news-and-alert/newsdetails?id=2019113596&symbol=KBANK
Management Discussion and Analysis Quarter 3 Ending 30 September 2019 https://www.set.or.th/en/market/news-and-alert/newsdetails?id=2019107582&symbol=KBANK
Reviewed financial performance Quarter 3 (F45) https://www.set.or.th/en/market/news-and-alert/newsdetails?id=2019107579&symbol=KBANK
Financial Statement Quarter 3/2019 (Reviewed) https://www.set.or.th/en/market/news-and-alert/newsdetails?id=2019107573&symbol=KBANK
Submittal of the reviewed Financial Statements for the three-month and nine-month periods ended 30 September 2019 and the Management Discussion and Analysis (MD&A) https://www.set.or.th/en/market/news-and-alert/newsdetails?id=2019107563&symbol=KBANK
Announcement of Basel III - Pillar 3 Disclosures for the period ended 30 June 2019 https://www.set.or.th/en/market/news-and-alert/newsdetails?id=2019101798&symbol=KBANK
KASIKORNBANK's 2020 Financial Targets https://www.set.or.th/en/market/news-and-alert/newsdetails?id=2019100644&symbol=KBANK
Summary Statement of Assets and Liabilities as of 30 September 2019 https://www.set.or.th/en/market/news-and-alert/newsdetails?id=2019099603&symbol=KBANK
Unreviewed financial performance Quarter 3 (F45) https://www.set.or.th/en/market/news-and-alert/newsdetails?id=2019099599&symbol=KBANK
Financial Statement Quarter 3/2019 (Unreviewed) https://www.set.or.th/en/market/news-and-alert/newsdetails?id=2019099598&symbol=KBANK
Summary of the Bank and its subsidiaries' operating performance for Q3 and 9 month ended 30 September 2019 https://www.set.or.th/en/market/news-and-alert/newsdetails?id=2019099601&symbol=KBANK
Submittal of Financial Statements of Q3 and 9 month ended 30 September 2019 (Unreviewed) https://www.set.or.th/en/market/news-and-alert/newsdetails?id=2019099595&symbol=KBANK
Appointment of Chief Executive Officer and Advisory to Management Committee https://www.set.or.th/en/market/news-and-alert/newsdetails?id=2019097446&symbol=KBANK
Notification of Issuance and Offering of Subordinated Notes to Overseas Investors https://www.set.or.th/en/market/news-and-alert/newsdetails?id=2019095377&symbol=KBANK
Establishment of AI GEN Company Limited https://www.set.or.th/en/market/news-and-alert/newsdetails?id=2019092010&symbol=KBANK
Summary Statement of Assets and Liabilities as of 31 August 2019 https://www.set.or.th/en/market/news-and-alert/newsdetails?id=2019091844&symbol=KBANK
Audited Half Year and Consolidated F/S (F45-3) https://www.set.or.th/en/market/news-and-alert/newsdetails?id=2019086218&symbol=KBANK
Management Discussion and Analysis Quarter 2 Ending 30 June 2019 https://www.set.or.th/en/market/news-and-alert/newsdetails?id=2019086220&symbol=KBANK
Submittal of the audited Financial Statements  for the three-month and six-month periods ended 30 June 2019 and the Management Discussion and Analysis (MD&A) https://www.set.or.th/en/market/news-and-alert/newsdetails?id=2019086214&symbol=KBANK
Financial Statement Half Year 2019 (Audited) https://www.set.or.th/en/market/news-and-alert/newsdetails?id=2019086217&symbol=KBANK
Rights of Shareholders to Propose Issues for Inclusion as AGM Agenda and Qualified Candidate for Nomination as KBank Director https://www.set.or.th/en/market/news-and-alert/newsdetails?id=2019086098&symbol=KBANK
Interim Dividend Payment for the First-Half of 2019 Operating Results https://www.set.or.th/en/market/news-and-alert/newsdetails?id=2019086096&symbol=KBANK
Summary Statement of Assets and Liabilities as of 31 July 2019 https://www.set.or.th/en/market/news-and-alert/newsdetails?id=2019082968&symbol=KBANK