选择器爬行没有结果

Question

我尝试抓取电影网站。没有错误，但没有结果没有iframe。我通过chrome> F12> Copy> Copy selector选择选择位置

网站链接是CGV movie

Element what i select

import requests
from bs4 import BeautifulSoup
import json
import os

## python file location
BASE_DIR = os.path.dirname(os.path.abspath(__file__))

req = requests.get('http://ticket.cgv.co.kr/Reservation/Reservation.aspx?MOVIE_CD=&MOVIE_CD_GROUP=&PLAY_YMD=&THEATER_CD=&PLAY_NUM=&PLAY_START_TM=&AREA_CD=&SCREEN_CD=&THIRD_ITEM=')
html = req.text
soup = BeautifulSoup(html, 'html.parser')
movie_titles = soup.select(
    '#movie_list > ul > li > a > span.text'
    )

data = {}

for title in movie_titles:
    data = title.text + "\n"

with open(os.path.join(BASE_DIR, 'result.json'), 'w+') as json_file:
    json.dump(data, json_file)

Answer 1

您想要的数据是动态加载的。使用像硒这样的方法，例如

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

url = 'http://ticket.cgv.co.kr/Reservation/Reservation.aspx?MOVIE_CD=&MOVIE_CD_GROUP=&PLAY_YMD=&THEATER_CD=&PLAY_NUM=&PLAY_START_TM=&AREA_CD=&SCREEN_CD=&THIRD_ITEM='
driver = webdriver.Chrome()
driver.get(url)
data = [item.text for item in WebDriverWait(driver,10).until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, "#movie_list > ul > li > a > span.text")))]
print(data)

您可以将css选择器缩短为

#movie_list .text