通过 JavaScript 寻呼机导航
问题描述 投票：0回答：2
网站 https://www.supralift.com/uk/itemsearch/results 使用纯基于 JavaScript 的分页器，它不会公开 url 中的任何参数，我可以更改这些参数并通过网站这种方式导航。
有人可以指导我如何模拟寻呼机导航按钮的点击吗？我的目标是收集整个网站上的所有广告链接。
查看 Chrome 控制台的“网络”选项卡，我发现该网站在 url
/api/search/item/summary{
"type": "about:blank",
"title": "Method Not Allowed",
"status": 405,
"detail": "Method 'GET' is not supported.",
"instance": "/api/search/item/summary"
}
为什么我可以在 Chrome 控制台中看到数据，但服务器拒绝以普通方式提供它们？
提前非常感谢。
2个回答
0
投票
投票
根据你的问题，你说这个端点
/api/search/item/summary405 Method not allowedPOSTAllow:
）。这是带有 json 正文的完整 POSTPOST /api/search/item/summary?size=100&page=0&sort=price,asc HTTP/2
Host: www.supralift.com
User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:130.0) Gecko/20100101 Firefox/130.0
Accept: application/json, text/plain, */*
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br
Referer: https://www.supralift.com/
Content-Type: application/json
Access-Control-Allow-Origin: https://www.supralift.com
Content-Length: 2664
Origin: https://www.supralift.com
Sec-Fetch-Dest: empty
Sec-Fetch-Mode: cors
Sec-Fetch-Site: same-origin
Priority: u=4
Te: trailers
{"searchType":null,"bundleId":null,"identification":{"slNumber":null,"serialNumber":null,"supplierProductNumber":null,"slOrSupplierProductNumber":null},"configuration":{"buildClass":null,"manufacturer":null,"buildSeries":null,"acDriven":null,"powerUnit":null,"fuelType":null,"mastType":null,"gearBox":null,"tyres":null,"typeSearch":null},"buildDates":{"month":null,"year":{"from":null,"to":null}},"dimensions":{"overallHeight":{"from":null,"to":null},"workingHours":{"from":null,"to":null},"loadCentreOfGravity":{"from":null,"to":null},"capacity":{"from":null,"to":null},"forkLength":{"from":null,"to":null},"towingCapacity":{"from":null,"to":null}},"price":{"price":{"from":null,"to":null,"currency":"GBP"}},"cabin":{"cabin":null,"height":{"from":null,"to":null},"platformHeight":{"from":null,"to":null}},"engine":{"manufacturer":null,"power":null},"battery":{"exists":null,"manufacturer":null,"batteryType":null,"voltage":{"from":null,"to":null},"capacity":{"from":null,"to":null},"buildDates":null},"batteryCharger":{"exists":null,"manufacturer":null,"voltage":{"from":null,"to":null},"current":{"from":null,"to":null},"buildDates":{"month":null,"year":{"from":null,"to":null}}},"location":{"distance":100,"postCode":null,"region":null,"countryState":null,"country":null,"countryOrNull":null},"container":{"containerType":null,"hubhoehe8Z3":{"from":null,"to":null},"hubhoehe8Z4":{"from":null,"to":null},"hubhoehe8Z5":{"from":null,"to":null},"hubhoehe8Z6":{"from":null,"to":null},"hubhoehe8Z7":{"from":null,"to":null},"hubhoehe8Z8":{"from":null,"to":null},"hubhoehe8Z6I3":{"from":null,"to":null},"hubhoehe8Z6I4":{"from":null,"to":null},"hubhoehe8Z6I5":{"from":null,"to":null},"hubhoehe8Z6I6":{"from":null,"to":null},"hubhoehe8Z6I7":{"from":null,"to":null},"hubhoehe8Z6I8":{"from":null,"to":null},"hubhoehe9Z6I3":{"from":null,"to":null},"hubhoehe9Z6I4":{"from":null,"to":null},"hubhoehe9Z6I5":{"from":null,"to":null},"hubhoehe9Z6I6":{"from":null,"to":null},"hubhoehe9Z6I7":{"from":null,"to":null},"hubhoehe9Z6I8":{"from":null,"to":null}},"offerDetails":{"offerBegin":null,"maxOfferAge":null,"activationDate":null,"offerFormat":null,"dealsOnly":null,"imagesOnly":null,"offerType":"SALE"},"additionalHydraulic":{"toValve":null,"complete":null},"liftAttributes":{"initialLift":null,"liftHeight":{"from":null,"to":null},"freeLift":{"from":null,"to":null},"liftPower":null},"isLicensedDealerOnly":null,"warranty":{"from":null,"to":null},"qualityRating":null,"attachments":null,"accessories":null,"customFields":null,"specialAttributes":{"explosionProof":null,"stainlessSteel":null,"autonomousMobileRobot":null},"freightTerm":null,"itemStatus":[],"backendSearch":false}
您应该使用什么工具来捕获此类请求？
嗯，我使用了 port swigger 的一个名为
Burp Suite如果您对 burp 套件感兴趣，请按照以下步骤操作：
- 下载社区版本（因为它是免费的）。
- 下载名为
的 Firefox/chrome 扩展程序。 （在谷歌上搜索Foxyproxy
）Foxypoxy - 使用浏览器设置 burp 代理。 （借助
扩展）Foxyproxy - 在目标浏览器上上传 burp 套件生成的 burp ca 证书。 （在您的浏览器中：
--> 搜索Settings
，谷歌搜索以获得更好的解决方案）。certificates
就是这样！
如果您在设置 Burp Suite 时遇到任何问题，请不要忘记用 google 搜索它。
希望这会有所帮助。
谢谢
0
投票
投票
如果没有代码示例，很难猜测您到底是如何实现这一点的。假设从包含的标签来看，可能是通过
httr::GET()rvest::read_html_live()无论如何，从 DevTools 的 Network 选项卡中您可以看到实际的请求方法，在本例中它是
POSTGEThttr::GET()POST要在 R 中发出相同的请求，您可以首先将请求复制为 cURL 并通过 https://curlconverter.com/r/ 或类似工具进行翻译。
或者如果
httr2httr2::curl_translate()请求方法 & 复制为 cURL（如果它是某些转换工具的输入，即使在 Windows 中，也可以使用 bash）：
以下内容基于
httr2::curl_translate()req_body_raw()POSThttr2req_perform_iterative()iterate_with_offset()resp_pagestotalPagesresp_completelastresp_completeTRUEmax_reqslibrary(httr2)
# prepare 1st request object, size increased
req <- request("https://www.supralift.com/api/search/item/summary") |>
req_url_query(
size = "2000",
page = "0",
) |>
req_body_raw('{"searchType":null,"bundleId":null,"identification":{"slNumber":null,"serialNumber":null,"supplierProductNumber":null,"slOrSupplierProductNumber":null},"configuration":{"buildClass":null,"manufacturer":null,"buildSeries":null,"acDriven":null,"powerUnit":null,"fuelType":null,"mastType":null,"gearBox":null,"tyres":null,"typeSearch":null},"buildDates":{"month":null,"year":{"from":null,"to":null}},"dimensions":{"overallHeight":{"from":null,"to":null},"workingHours":{"from":null,"to":null},"loadCentreOfGravity":{"from":null,"to":null},"capacity":{"from":null,"to":null},"forkLength":{"from":null,"to":null},"towingCapacity":{"from":null,"to":null}},"price":{"price":{"from":null,"to":null,"currency":"GBP"}},"cabin":{"cabin":null,"height":{"from":null,"to":null},"platformHeight":{"from":null,"to":null}},"engine":{"manufacturer":null,"power":null},"battery":{"exists":null,"manufacturer":null,"batteryType":null,"voltage":{"from":null,"to":null},"capacity":{"from":null,"to":null},"buildDates":null},"batteryCharger":{"exists":null,"manufacturer":null,"voltage":{"from":null,"to":null},"current":{"from":null,"to":null},"buildDates":{"month":null,"year":{"from":null,"to":null}}},"location":{"distance":100,"postCode":null,"region":null,"countryState":null,"country":null,"countryOrNull":null},"container":{"containerType":null,"hubhoehe8Z3":{"from":null,"to":null},"hubhoehe8Z4":{"from":null,"to":null},"hubhoehe8Z5":{"from":null,"to":null},"hubhoehe8Z6":{"from":null,"to":null},"hubhoehe8Z7":{"from":null,"to":null},"hubhoehe8Z8":{"from":null,"to":null},"hubhoehe8Z6I3":{"from":null,"to":null},"hubhoehe8Z6I4":{"from":null,"to":null},"hubhoehe8Z6I5":{"from":null,"to":null},"hubhoehe8Z6I6":{"from":null,"to":null},"hubhoehe8Z6I7":{"from":null,"to":null},"hubhoehe8Z6I8":{"from":null,"to":null},"hubhoehe9Z6I3":{"from":null,"to":null},"hubhoehe9Z6I4":{"from":null,"to":null},"hubhoehe9Z6I5":{"from":null,"to":null},"hubhoehe9Z6I6":{"from":null,"to":null},"hubhoehe9Z6I7":{"from":null,"to":null},"hubhoehe9Z6I8":{"from":null,"to":null}},"offerDetails":{"offerBegin":null,"maxOfferAge":null,"activationDate":null,"offerFormat":null,"dealsOnly":null,"imagesOnly":null,"offerType":"SALE"},"additionalHydraulic":{"toValve":null,"complete":null},"liftAttributes":{"initialLift":null,"liftHeight":{"from":null,"to":null},"freeLift":{"from":null,"to":null},"liftPower":null},"isLicensedDealerOnly":null,"warranty":{"from":null,"to":null},"qualityRating":null,"attachments":null,"accessories":null,"customFields":null,"specialAttributes":{"explosionProof":null,"stainlessSteel":null,"autonomousMobileRobot":null},"freightTerm":null,"itemStatus":[],"backendSearch":false}', "application/json")
# perform series of requests, increase `page` parameter until
# `resp_complete` returns TRUE or when reaching `max_reqs`
resps <-
req_perform_iterative(
req,
next_req = iterate_with_offset(
param_name = "page",
start = 0,
resp_pages = \(resp) resp_body_json(resp)$totalPages,
resp_complete = \(resp) resp_body_json(resp)$last,
),
# generate just first 2 requests as an example
max_reqs = 2
)
#> Iterating ■■■■■■■■■■■■■■■■ 50% | ETA: 3s
# list of responses:
resps
#> [[1]]
#> <httr2_response>
#> POST https://www.supralift.com/api/search/item/summary?size=2000&page=0
#> Status: 200 OK
#> Content-Type: application/json
#> Body: In memory (2043726 bytes)
#>
#> [[2]]
#> <httr2_response>
#> POST https://www.supralift.com/api/search/item/summary?size=2000&page=1
#> Status: 200 OK
#> Content-Type: application/json
#> Body: In memory (1933115 bytes)
单帧中的所有数据，2000（大小）* 2（max_reqs）= 4000行：
resps_data(resps, \(resp) resp_body_json(resp, simplifyVector = TRUE)$content) |>
tibble::as_tibble()
#> # A tibble: 4,000 × 18
#> id slNo type manufacturer powerUnit buildClass mastType liftHeight
#> <chr> <int> <chr> <chr> <chr> <chr> <chr> <int>
#> 1 66f8d101f… 1.29e7 R14S LINDE ELEKTRO SCHUBMAST… DREIFACH 6250
#> 2 66f8ccfaf… 1.29e7 EPL1… WEITERE ELEKTRO NIEDERHUB… <NA> 115
#> 3 66f8c894f… 1.29e7 27823 JUNGHEINRICH ELEKTRO WEITERE DREIFACH 5500
#> 4 66f8c886f… 1.29e7 27799 JUNGHEINRICH ELEKTRO WEITERE DREIFACH 5000
#> 5 66f8c595f… 1.29e7 T401… BOBCAT DIESEL TELESKOPA… <NA> 17000
#> 6 66f8c4e7f… 1.29e7 TFG … JUNGHEINRICH GAS VIERRADFR… DREIFACH 5000
#> 7 66f8b525f… 1.29e7 EFG … JUNGHEINRICH ELEKTRO VIERRADFR… DREIFACH 6000
#> 8 66f81676f… 1.28e7 R16G LINDE ELEKTRO SCHUBMAST… <NA> 6210
#> 9 66f80016f… 1.29e7 C500… COMBILIFT ELEKTRO SEITENSTA… DREIFACH 6000
#> 10 66f80015f… 1.29e7 H30D… LINDE DIESEL VIERRADFR… ZWEIFACH 3760
#> # ℹ 3,990 more rows
#> # ℹ 10 more variables: workingHours <int>, capacity <int>, yearOfBuild <int>,
#> # company <df[,6]>, offerType <chr>, isNew <lgl>, price <df[,4]>,
#> # qualityRating <int>, images <list>, specialRating <df[,4]>
创建于 2024-09-29，使用 reprex v2.1.1
最新问题
- 检查是否支持可选链接
- 枚举nodejs中的系统驱动器
- 即使在使用新存储桶和限定符重新引导后，AWS CDK 仍会引用旧的引导限定符
- Selenium-在 :before 部分中连续查找并单击“显示更多”按钮
- VSCode 远程 SSH - 失败：错误：没有到主机的路由
- 角度代码未显示文本框
- Android Studio 如何在 Windows 中使用 mega sdk
- jQuery：如何根据浏览器屏幕大小制作“实时分页器”而不刷新“内容”？
- jqgrid 分页不起作用...需要帮助
- 使用 jQuery 删除表格行的最佳方法是什么？
- 语法错误：无法在模块外部使用 import 语句 - Firebase 与 NodeJS
- 部署到托管服务时无法访问 HTTP 服务器
- c中是否有像python的a[-1]这样的数组反向索引？
- 目前我正在做params.node.beans.gos.gridOptions.rowData，有没有更好的方法来获取gridOptions.rowData？
- 如何获取一行DataFrame，使得该行也是一个DataFrame？
- 如何从存储为字符串的.xml中提取数据/节点？
- 我可以在VS2010中将IronRuby项目编译成DLL/exe文件吗？
- 如何安装旧版本的 Flash Player？
- 如何将 H1 在 div 中垂直居中？
- 如何使用playwright登录google帐户？
© www.soinside.com 2019 - 2024. All rights reserved.