Python get 请求生成的 HTML 与查看源代码不同

问题描述 投票：0回答：1
import requests
# Request to website and download HTML contents
url='https://beacon.schneidercorp.com/Application.aspx?AppID=165&LayerID=2145&PageTypeID=2&PageID=1104&KeyValue=0527427230'
req=requests.get(url, 'html.parser')

产生这个结果：

'<!DOCTYPE html><html lang="en-US"><head><title>Just a moment...</title><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta http-equiv="X-UA-Compatible" content="IE=Edge"><meta name="robots" content="noindex,nofollow"><meta name="viewport" content="width=device-width,initial-scale=1"><style>*{box-sizing:border-box;margin:0;padding:0}html{line-height:1.15;-webkit-text-size-adjust:100%;color:#313131}button,html{font-family:system-ui,-apple-system,BlinkMacSystemFont,Segoe UI,Roboto,Helvetica Neue,Arial,Noto Sans,sans-serif,Apple Color Emoji,Segoe UI Emoji,Segoe UI Symbol,Noto Color Emoji}@media (prefers-color-scheme:dark){body{background-color:#222;color:#d9d9d9}body a{color:#fff}body a:hover{color:#ee730a;text-decoration:underline}body .lds-ring div{border-color:#999 transparent transparent}body .font-red{color:#b20f03}body .pow-button{background-color:#4693ff;color:#1d1d1d}body #challenge-success-text{background-image:url(data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIzMiIgaGVpZ2h0PSIzMiIgZmlsbD0ibm9uZSIgdmlld0JveD0iMCAwIDI2IDI2Ij48cGF0aCBmaWxsPSIjZDlkOWQ5IiBkPSJNMTMgMGExMyAxMyAwIDEgMCAwIDI2IDEzIDEzIDAgMCAwIDAtMjZtMCAyNGExMSAxMSAwIDEgMSAwLTIyIDExIDExIDAgMCAxIDAgMjIiLz48cGF0aCBmaWxsPSIjZDlkOWQ5IiBkPSJtMTAuOTU1IDE2LjA1NS0zLjk1LTQuMTI1LTEuNDQ1IDEuMzg1IDUuMzcgNS42MSA5LjQ5NS05LjYtMS40Mi0xLjQwNXoiLz48L3N2Zz4=)}body #challenge-error-text{background-image:url(data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIzMiIgaGVpZ2h0PSIzMiIgZmlsbD0ibm9uZSI+PHBhdGggZmlsbD0iI0IyMEYwMyIgZD0iTTE2IDNhMTMgMTMgMCAxIDAgMTMgMTNBMTMuMDE1IDEzLjAxNSAwIDAgMCAxNiAzbTAgMjRhMTEgMTEgMCAxIDEgMTEtMTEgMTEuMDEgMTEuMDEgMCAwIDEtMTEgMTEiLz48cGF0aCBmaWxsPSIjQjIwRjAzIiBkPSJNMTcuMDM4IDE4LjYxNUgxNC44N0wxNC41NjMgOS41aDIuNzgzem0tMS4wODQgMS40MjdxLjY2IDAgMS4wNTcuMzg4LjQwNy4zODkuNDA3Ljk5NCAwIC41OTYtLjQwNy45ODQtLjM5Ny4zOS0xLjA1Ny4zODktLjY1IDAtMS4wNTYtLjM4OS0uMzk4LS4zODktLjM5OC0uOTg0IDAtLjU5Ny4zOTgtLjk4NS40MDYtLjM5NyAxLjA1Ni0uMzk3Ii8+PC9zdmc+)}}body{display:flex;flex-direction:column;min-height:100vh}body.no-js .loading-spinner{visibility:hidden}body.no-js .challenge-running{display:none}body.dark{background-color:#222;color:#d9d9d9}body.dark a{color:#fff}body.dark a:hover{color:#ee730a;text-decoration:underline}body.dark .lds-ring div{border-color:#999 transparent transparent}body.dark .font-red{color:#b20f03}body.dark .pow-button{background-color:#4693ff;color:#1d1d1d}body.dark #challenge-success-text{background-image:url(data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIzMiIgaGVpZ2h0PSIzMiIgZmlsbD0ibm9uZSIgdmlld0JveD0iMCAwIDI2IDI2Ij48cGF0aCBmaWxsPSIjZDlkOWQ5IiBkPSJNMTMgMGExMyAxMyAwIDEgMCAwIDI2IDEzIDEzIDAgMCAwIDAtMjZtMCAyNGExMSAxMSAwIDEgMSAwLTIyIDExIDExIDAgMCAxIDAgMjIiLz48cGF0aCBmaWxsPSIjZDlkOWQ5IiBkPSJtMTAuOTU1IDE2LjA1NS0zLjk1LTQuMTI1LTEuNDQ1IDEuMzg1IDUuMzcgNS42MSA5LjQ5NS05LjYtMS40Mi0xLjQwNXoiLz48L3N2Zz4=)}body.dark #challenge-error-text{background-image:url(data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIzMiIgaGVpZ2h0PSIzMiIgZmlsbD0ibm9uZSI+PHBhdGggZmlsbD0iI0IyMEYwMyIgZD0iTTE2IDNhMTMgMTMgMCAxIDAgMTMgMTNBMTMuMDE1IDEzLjAxNSAwIDAgMCAxNiAzbTAgMjRhMTEgMTEgMCAxIDEgMTEtMTEgMTEuMDEgMTEuMDEgMCAwIDEtMTEgMTEiLz48cGF0aCBmaWxsPSIjQjIwRjAzIiBkPSJNMTcuMDM4IDE4LjYxNUgxNC44N0wxNC41NjMgOS41aDIuNzgzem0tMS4wODQgMS40MjdxLjY2IDAgMS4wNTcuMzg4LjQwNy4zODkuNDA3Ljk5NCAwIC41OTYtLjQwNy45ODQtLjM5Ny4zOS0xLjA1Ny4zODktLjY1IDAtMS4wNTYtLjM4OS0uMzk4LS4zODktLjM5OC0uOTg0IDAtLjU5Ny4zOTgtLjk4NS40MDYtLjM5NyAxLjA1Ni0uMzk3Ii8+PC9zdmc+)}body.light{background-color:transparent;color:#313131}body.light a{color:#0051c3}body.light a:hover{color:#ee730a;text-decoration:underline}body.light .lds-ring div{border-color:#595959 transparent transparent}body.light .font-red{color:#fc574a}body.light .pow-button{background-color:#003681;border-color:#003681;color:#fff}body.light #challenge-success-text{background-image:url(data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIzMiIgaGVpZ2h0PSIzMiIgZmlsbD0ibm9uZSIgdmlld0JveD0iMCAwIDI2IDI2Ij48cGF0aCBmaWxsPSIjMzEzMTMxIiBkPSJNMTMgMGExMyAxMyAwIDEgMCAwIDI2IDEzIDEzIDAgMCAwIDAtMjZtMCAyNGExMSAxMSAwIDEgMSAwLTIyIDExIDExIDAgMCAxIDAgMjIiLz48cGF0aCBmaWxsPSIjMzEzMTMxIiBkPSJtMTAuOTU1IDE2LjA1NS0zLjk1LTQuMTI1LTEuNDQ1IDEuMzg1IDUuMzcgNS42MSA5LjQ5NS05LjYtMS40Mi0xLjQwNXoiLz48L3N2Zz4=)}body.light #challenge-error-text{background-image:url(data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIzMiIgaGVpZ2h0PSIzMiIgZmlsbD0ibm9uZSI+PHBhdGggZmlsbD0iI2ZjNTc0YSIgZD0iTTE2IDNhMTMgMTMgMCAxIDAgMTMgMTNBMTMuMDE1IDEzLjAxNSAwIDAgMCAxNiAzbTAgMjRhMTEgMTEgMCAxIDEgMTEtMTEgMTEuMDEgMTEuMDEgMCAwIDEtMTEgMTEiLz48cGF0aCBmaWxsPSIjZmM1NzRhIiBkPSJNMTcuMDM4IDE4LjYxNUgxNC44N0wxNC41NjMgOS41aDIuNzgzem0tMS4wODQgMS40MjdxLjY2IDAgMS4wNTcuMzg4LjQwNy4zODkuNDA3Ljk5NCAwIC41OTYtLjQwNy45ODQtLjM5Ny4zOS0xLjA1Ny4zODktLjY1IDAtMS4wNTYtLjM4OS0uMzk4LS4zODktLjM5OC0uOTg0IDAtLjU5Ny4zOTgtLjk4NS40MDYtLjM5NyAxLjA1Ni0uMzk3Ii8+PC9zdmc+)}a{background-color:transparent;color:#0051c3;text-decoration:none;transition:color .15s ease}a:hover{color:#ee730a;text-decoration:underline}.main-content{margin:8rem auto;max-width:60rem;width:100%}.heading-favicon{height:2rem;margin-right:.5rem;width:2rem}@media (width <= 720px){.main-content{margin-top:4rem}.heading-favicon{height:1.5rem;width:1.5rem}}.footer,.main-content{padding-left:1.5rem;padding-right:1.5rem}.main-wrapper{align-items:center;display:flex;flex:1;flex-direction:column}.font-red{color:#b20f03}.spacer{margin:2rem 0}.h1{font-size:2.5rem;font-weight:500;line-height:3.75rem}.h2{font-weight:500}.core-msg,.h2{font-size:1.5rem;line-height:2.25rem}.body-text,.core-msg{font-weight:400}.body-text{font-size:1rem;line-height:1.25rem}@media (width <= 720px){.h1{font-size:1.5rem;line-height:1.75rem}.h2{font-size:1.25rem}.core-msg,.h2{line-height:1.5rem}.core-msg{font-size:1rem}}#challenge-error-text{background-image:url(data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIzMiIgaGVpZ2h0PSIzMiIgZmlsbD0ibm9uZSI+PHBhdGggZmlsbD0iI2ZjNTc0YSIgZD0iTTE2IDNhMTMgMTMgMCAxIDAgMTMgMTNBMTMuMDE1IDEzLjAxNSAwIDAgMCAxNiAzbTAgMjRhMTEgMTEgMCAxIDEgMTEtMTEgMTEuMDEgMTEuMDEgMCAwIDEtMTEgMTEiLz48cGF0aCBmaWxsPSIjZmM1NzRhIiBkPSJNMTcuMDM4IDE4LjYxNUgxNC44N0wxNC41NjMgOS41aDIuNzgzem0tMS4wODQgMS40MjdxLjY2IDAgMS4wNTcuMzg4LjQwNy4zODkuNDA3Ljk5NCAwIC41OTYtLjQwNy45ODQtLjM5Ny4zOS0xLjA1Ny4zODktLjY1IDAtMS4wNTYtLjM4OS0uMzk4LS4zODktLjM5OC0uOTg0IDAtLjU5Ny4zOTgtLjk4NS40MDYtLjM5NyAxLjA1Ni0uMzk3Ii8+PC9zdmc+);padding-left:34px}#challenge-error-text,#challenge-success-text{background-repeat:no-repeat;background-size:contain}#challenge-success-text{background-image:url(data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIzMiIgaGVpZ2h0PSIzMiIgZmlsbD0ibm9uZSIgdmlld0JveD0iMCAwIDI2IDI2Ij48cGF0aCBmaWxsPSIjMzEzMTMxIiBkPSJNMTMgMGExMyAxMyAwIDEgMCAwIDI2IDEzIDEzIDAgMCAwIDAtMjZtMCAyNGExMSAxMSAwIDEgMSAwLTIyIDExIDExIDAgMCAxIDAgMjIiLz48cGF0aCBmaWxsPSIjMzEzMTMxIiBkPSJtMTAuOTU1IDE2LjA1NS0zLjk1LTQuMTI1LTEuNDQ1IDEuMzg1IDUuMzcgNS42MSA5LjQ5NS05LjYtMS40Mi0xLjQwNXoiLz48L3N2Zz4=);padding-left:42px}.text-center{text-align:center}.pow-button{background-color:#0051c3;border:.063rem solid #0051c3;border-radius:.313rem;color:#fff;font-size:.875rem;line-height:1.313rem;margin:2rem 0;padding:.375rem 1rem;transition-duration:.2s;transition-property:background-color,border-color,color;transition-timing-function:ease}.pow-button:hover{background-color:#003681;border-color:#003681;color:#fff;cursor:pointer}.footer{font-size:.75rem;line-height:1.125rem;margin:0 auto;max-width:60rem;width:100%}.footer-inner{border-top:1px solid #d9d9d9;padding-bottom:1rem;padding-top:1rem}.clearfix:after{clear:both;content:"";display:table}.clearfix .column{float:left;padding-right:1.5rem;width:50%}.diagnostic-wrapper{margin-bottom:.5rem}.footer .ray-id{text-align:center}.footer .ray-id code{font-family:monaco,courier,monospace}.core-msg,.zone-name-title{overflow-wrap:break-word}@media (width <= 720px){.diagnostic-wrapper{display:flex;flex-wrap:wrap;justify-content:center}.clearfix:after{clear:none;content:none;display:initial;text-align:center}.column{padding-bottom:2rem}.clearfix .column{float:none;padding:0;width:auto;word-break:keep-all}.zone-name-title{margin-bottom:1rem}}.loading-spinner{height:76.391px}.lds-ring{display:inline-block;position:relative}.lds-ring,.lds-ring div{height:1.875rem;width:1.875rem}.lds-ring div{animation:lds-ring 1.2s cubic-bezier(.5,0,.5,1) infinite;border:.3rem solid transparent;border-radius:50%;border-top-color:#313131;box-sizing:border-box;display:block;position:absolute}.lds-ring div:first-child{animation-delay:-.45s}.lds-ring div:nth-child(2){animation-delay:-.3s}.lds-ring div:nth-child(3){animation-delay:-.15s}@keyframes lds-ring{0%{transform:rotate(0)}to{transform:rotate(1turn)}}@media screen and (-ms-high-contrast:active),screen and (-ms-high-contrast:none){.main-wrapper,body{display:block}}.rtl .heading-favicon{margin-left:.5rem;margin-right:0}.rtl #challenge-success-text{background-position:100%;padding-left:0;padding-right:42px}.rtl #challenge-error-text{background-position:100%;padding-left:0;padding-right:34px}</style><meta http-equiv="refresh" content="390"></head><body class="no-js"><div class="main-wrapper" role="main"><div class="main-content"><noscript><div id="challenge-error-title"><div class="h2"><span id="challenge-error-text">Enable JavaScript and cookies to continue</span></div></div></noscript></div></div><script>(function(){window._cf_chl_opt={cvId: \'3\',cZone: "beacon.schneidercorp.com",cType: \'managed\',cNounce: \'28917\',cRay: \'89565d36ac397d1a\',cHash: \'47375b3742c7e0b\',cUPMDTk: "\\/Application.aspx?AppID=165&LayerID=2145&PageTypeID=2&PageID=1104&KeyValue=0527427230&html.parser&__cf_chl_tk=Of4B4c7stWB5SXvzpcRfYtsBNPOViNaNK0Ll9b5neUY-1718662168-0.0.1.1-4095",cFPWv: \'g\',cTTimeMs: \'1000\',cMTimeMs: \'390000\',cTplV: 5,cTplB: \'cf\',cK: "visitor-time",fa: "\\/Application.aspx?AppID=165&LayerID=2145&PageTypeID=2&PageID=1104&KeyValue=0527427230&html.parser&__cf_chl_f_tk=Of4B4c7stWB5SXvzpcRfYtsBNPOViNaNK0Ll9b5neUY-1718662168-0.0.1.1-4095",md: "rmmSJgeD56LpTOvrhy4DfR8o7RU_J0Mb91rJJIbLBXQ-1718662168-1.1.1.1-d25k6KHPu0f_TpOQSmgKRsOCg5vU130QkwpV5TbrFFcsoGgiTQp6MeQOCB5gFbnZVLkOOl_glh6FwjR9WNfDp_Nxb99nyFYlHPSMJLLLZXVUsvEaraTpJ0VYK5vR8RPw6JK4TMBg16JOTm6n8N78f49tUYNsgQVbO09II98fphgiKkrY6MTEDVrDY2QqDrGFrUVz7VD2Y3SjGeC1SmKdH9Jic4qnchy.dj1_DJiAJTLQju.3pS34xv1lkxSzMeoBOG.dSwRMG5NB0oICddop9iYkoUYkiimjE7lenhKgL2W_ZrrSICqiyaXm_2rdXmYQxIGacIUetCO9o2j7Kpw47PKH3lszsqLuQMoXOcws2zYJXjG..0WmLIgkabofpUlVrMnDxKMhuI9tKBjjwS32QH4gpaRG.a0PvXbL_EjqiZYRHXxE6.NFARlbOEnPc5lDJB7nlgdAnkyHYTKHcMmA2a.yyvd.YHloLQD_FP748XMIFq4GScn1a9YxnyP6TW2_yiU9ijT_.mhoGlyQKWuQaq52SK5H2j99N4_uHzm3uQZ234Tv8q0eQ4tEDnCvIWI0Mz9F8DXJFDv.DRl1K95GmpD4D1elwhOQqMS9imBfGGN17FzFWLdBXnG1ZbLUlRQ1XTOd6261ibwTsmo2ALqoFjMVN__mucN1dWcWGTdQTGYefKWORAyn28g8eo5Vj28qy7M3wYqdLulVphF57PY4QgvUS5Lj7GeHO4fI1xDL7VosTxCQYdLHt.kc62ctn7f7q4NmiaIMh_X6O7SzgefYNg_0HcnRDsu4f1hdpRoPCrSqYViL60OQrH9s9rFi1cs9yAdvT5KSRdEVKn6yZlGiKp25lw8kwZijkifdlrbUXjjMB13pVqAApSornOd3ZaB8LeTcmebdPn3uVZ3zefpFEqB.zreEN60ctDFywXHNRQJdemXKKw5en8vH9Z1htdOvSUR.zP.jGhVCd6pA5WeC45KkwcMQlB_FpoETsS8eZIxS5k9YXVSlQXCoO3m.rgayokIdNOeQ4CZMTgPx_C5HeXxpbT9IHRpTbsymZdb3MMzeHY89zFcNY5o6K0OFQPV7mzEv19BKRT7l6bI9USomkV2WixTOeZU_m.u_hPCAGibsMrLRGpJCxWckGpWEP3.Sgu5UuPGlGHviIxx9HtbLdiXnv.pwyP7.RTUkcBYcUCeVf3v2c17envOLFKEqNRclD.ZK4TPUZ8QxbeTKXfnKY3FCrSbaU9hfj0GV9k2A98NgGbn.NxOdppg4i7VlA251LtZsnuHRsGkcPP4vIs0GJMNmro9TLjgpDYoqvMenfaBCimq.eofRK5T.o2E2SixxXujdmnSlBMgg.3gGMgyzV.v7iXEYOpg5OFyxNFp37hNw3xcHdd1cZJs421FE8bGH1HyqFzkvym_qE.mx_.Ut8tK.OxJIHDfdO19YJqieJ9YExUR_9uSBscFznIbT_0ZEAl4VGvmpCRehACvjGuhC4q7.o3aWq1MvL3I_r7JG8YyniNQYfZ_mGNgTu5LGMI0Xnt7Pan.zBK6fXh9k.s3CWneSU_51R0p6yiKydsi_B9gw5eJLxsPBqS5PWDGfIlRV2Hj6v4GyEMezVHtuErfCxbkPoQd5BOFvz0en2SwNfGuAY01hdk2eJWlxUujTHnibT8ph7Oipg8_.AFhlXcrcKkpNHD265ZB9LtBsNaiUumlXf_MIGaKrJFaygM6vf8vVnpzR8Qed4K6ws9WLRME8ENh0gwGfZTZuw26qDoK4M97hLodwatbfBrD3eI_W4Nlp.rIYp1l6GzWLeKWdzA0kAQ",mdrd: "3etA5PVmQCVVM_aEgwlZiVjISxDftXwiCnxf_X7iNsU-1718662168-1.1.1.1-WsAyjo7VoGj0jqSHgoQg0Q9w5sxrSyBSjIIOK68mjmh2mU_tRXXlBf4QodpYxI1XBbA2eMzOWuH0ak4Lefjix59qg0xrz8Ue8czMo86uiGUG6PM04EzUdW3N60Mug9TPFH31V9xAa4xjYSPk8XyYz_NPch3ACGXpoaJrHqNdcGpjdr5Q_f75jrqWIL7JuKza0KapZpWaOQ_yylgCMRVfEvXWmYWkcFPg8Tru4xr6Oarx.yx3fJ1yD1vVoI01k8YG_l8BhAisKNdAPSGN9wxNKka3HKCpaMIwDGhSoO1JOrtyoZFZ0ZoN41RG0GhbojD9TfqNlLcJceq2GEwAum48cuESWLfbqmTvwRJ1_QsQsl2Sz8mHMlKuHqhXjf0garO2bWEMf_gOol8xOxQF_REmhjrKLhWp0UvgZntInAvMulzDsQ2z0LmlxApDvDDEcrIsAS1Gb_e9ochz3tvJVLAyyaMrjqb22.xOwPekNXCcLRcbYmv4Rwp35oUgSLaoRXfuTvZRXCextUuIL8t4YlQpN.KZ3KNjdXOyQRg91VIrjs5DPYSjlcn1DKtUoybSthNLNslrPmGqAGuTcWRCWXAs3dNPlQf2slrzYh9zVoCDO_wYACbt8m5m8q2UUxbDRF2fZetH9JaQCYD3ac6F0MM4MEQHK4NKfZQyJEbZCTP5yTU2Tnl6mPfLbR0vx5JqSQLzEyQvnKb3xFKAf7QBrgSDKhwTssYJ9_pmEbYojDJpjz0xP1kx1PsQY409f6nIMaZHVYFsTl6ivW_6wHuJtL64Ikwdt7JiHRkJnPKjlEwJnToGvHi9MjDP1y_rBOUOQD4DsZzdwkza20ls4i9wZvNGfM8oVqhkMplteNarLSIO6VM5LvaaPJ729twPPT.PV9SEUCCcFFwXba.NZKX5i6gbCytqOuxrL_AzOospag5Oj4RPN3PVI_vRMqYGG7ZSfTbzW9v38WYrz2_seYLyH4.kY8anvNVguqFTBnpi_RwcdwJjkPUMgmnikDtaudhsZpq7RO8ru5TSpgTuonsKZV.ALgmYTHOHTrfzID_2zoQ5SUgoQDkcPt8hmRh8V.yhckcXSgIoLyX4n_MS_7k3Ua.5sosqjL5XKWItOGw7L34lOc3cKubpoFh3Ye.MdVq87yUceA6Z9oyXYSjhrAhBzk5vsrobARiAX2Zc9qlyt8ZNrOF63kcnZhQCEU1xmr1fGwSm17S3eok0fGHr5bVxCQhDUXOXEjsh3lcF_LLKSveWvjI8IHvzTW8eXAuR3aNxatBlvMpRwC9B25.xTn.D2AUj0nV8yeN2GS38FP2ejd1A7g_j2RcrWtrUFp1j7eiuK_wDyyjDE.hSClKZbDA4uTmdw1Rxf1Q6gF_mVH8LxG9azBm5yqAHWke5DEK8qoYfkjOgVwXEL_qwtwu7eyVqD_ZoinrOKDDJwPwJ2KS.bZVfVWAbPXutM8XoI41juZUgJB1xuaI2D.CIls0EdnELw5O0b_BEIiWifzGpohrMWquq8WG0aoDdR8hTet3cHyU5GU535AmCBAZO0VI03sh4YbxJvVBXDPl9EfEZaCe.QFtpc56ej4if7DwOAKvH2dzCKY9fHKOxolu0H5ZOBSnYqcVeAtOfF9fP3INh12jlN338zbAaXsgihoWiALgSvmLo41IxmjUC435vEnFnLhdPvtP10iLWgZeBcaEdTscqd6F5ojTSNhxdwsYrJp3hM2BDpRseSUY_rUAlBWiV4zwERwsQphlnbgfN9fHc6RGdcK.JHW.Ji.y4kATTHIEQKSOxe.Ih0dSu.l8QD54Lv6dnfl8i.W_At0.9G6P4GKPDpgoNRiRvDMh89r5U5kEkNUK7Ly8HqeTJL4hNRZDji13ZN2SUR71XdNWzjyKtMbsHoZacVgqZRLZfZJPF6yY_hIECo99pf7f0Md_oYblwUjcnq4NcstjFm_sDmHmFd69uRODRvyamP8oGI3uK6fOyhuP6_pEfurdEikUz3apG1DO3dx9JKTV0fIcrDbi7gqHuAiNVs22qExP4EE68GrZQykD3RQz64a6gpkR2fIW4m4dJL2R_L9xnHq59iRVYRFh2Py7rwLzvTJIasm0RKtgHpscW8SWViFFTPIxfElDEe_Egc_S61i3gxt1GZf1uPT7s40t3cAw",cRq: {ru: \'aHR0cHM6Ly9iZWFjb24uc2NobmVpZGVyY29ycC5jb20vQXBwbGljYXRpb24uYXNweD9BcHBJRD0xNjUmTGF5ZXJJRD0yMTQ1JlBhZ2VUeXBlSUQ9MiZQYWdlSUQ9MTEwNCZLZXlWYWx1ZT0wNTI3NDI3MjMwJmh0bWwucGFyc2Vy\',ra: \'cHl0aG9uLXJlcXVlc3RzLzIuMzEuMA==\',rm: \'R0VU\',d: \'u4f3Yobgwf5u6zv7Vyez37fDJMRVaKuOq4Y5paFsnFojnKglovP/CR4xXE9H/B8aGKiWmpGN1XHcP0Vz7qLFRqmmeeJP/GfPv83oDcqCgaDYYPTkl8T8DCoO5yplG3tvPrUquNAWpqzfVWiWIZOybBY9qdP5Noun/yrL94UndnQoz8xRNNujnm8jjICAgAVNC6YW4tAAMTbgyLk3y1J1aaCh029b2aCcPaEbI5000bwD2yX6pG8JSfkDfBRWoBJM7O9+WMUzelXC2Xim15NCEyUmQAsRduvTyqn9aVV0vOTJ4KsQHeCpD5+BDuE49N08SoRSpWXoapg8TSEmP6gIQyN8iqw1uX6eoWvVK9eFXuERXmpzBNANK5R3ba3dN+cEsmdkTaAIPjMlewlmn8HqdZV2dqT7mLG+dXp0CcFsRCa3D0l1saSNF5eE57Xsbn/V52VBim3pJZXT/KoWtfE1wllrKE13g12B19oTZnV1aTNn1CzzOpy3510FgxQOl/BpjXIDr07u4uyzmoQJiFNqpE8kqv6Dun5xNhqPDFu+NzXY2P6c6neXnFpR1nsz/67CBJIJaCf7C0isThSlIYB2B3d6H6X2eHXmXpFHEUEghPU=\',t: \'MTcxODY2MjE2OC4wMDAwMDA=\',cT: Math.floor(Date.now() / 1000),m: \'wDCh8QRn8G04Q366J+IK+KpGMLciSRvuB2gMZ5fhyf0=\',i1: \'9WrJz2g1uAB/7FMTNOMfKw==\',i2: \'JDZfN4djPTXHy+E32CWj5Q==\',zh: \'9PbOsqhSKsLpjRc2OkMWk0h/2bQLoa45/u0P1gLq0HY=\',uh: \'YE9XOpG5TeHmhA1zfs5mxC8CrRZzq2a/+r+OU7dliYQ=\',hh: \'z1PLu2KPJfY2I4YzfUIJyk5iSeOJDT8VV7UBO5WLEL4=\',}};var cpo = document.createElement(\'script\');cpo.src = \'/cdn-cgi/challenge-platform/h/g/orchestrate/chl_page/v1?ray=89565d36ac397d1a\';window._cf_chl_opt.cOgUHash = location.hash === \'\' && location.href.indexOf(\'#\') !== -1 ? \'#\' : location.hash;window._cf_chl_opt.cOgUQuery = location.search === \'\' && location.href.slice(0, location.href.length - window._cf_chl_opt.cOgUHash.length).indexOf(\'?\') !== -1 ? \'?\' : location.search;if (window.history && window.history.replaceState) {var ogU = location.pathname + window._cf_chl_opt.cOgUQuery + window._cf_chl_opt.cOgUHash;history.replaceState(null, null, "\\/Application.aspx?AppID=165&LayerID=2145&PageTypeID=2&PageID=1104&KeyValue=0527427230&html.parser&__cf_chl_rt_tk=Of4B4c7stWB5SXvzpcRfYtsBNPOViNaNK0Ll9b5neUY-1718662168-0.0.1.1-4095" + window._cf_chl_opt.cOgUHash);cpo.onload = function() {history.replaceState(null, null, ogU);}}document.getElementsByTagName(\'head\')[0].appendChild(cpo);}());</script></body></html>'

这与以下内容非常不同： 结果来自： 查看源代码：https://beacon.schneidercorp.com/Application.aspx?AppID=165&LayerID=2145&PageTypeID=4&PageID=1108&KeyValue=0527427230

我是网络抓取新手...

python html web-scraping request get
1个回答
0
投票

不同之处在于，当您使用浏览器访问 URL 时，它会处理从中接收到的脚本。

对 

requests.get()
的调用不会进行任何额外的处理，因此您会看到来自服务器的原始响应。

您可以尝试使用解析返回代码的库，为您提供最终的网页，就像您在浏览器中看到的那样，但使用 Selenium 等测试工具之一可能更简单。使用它，您可以查询模拟任何您想要的浏览器的页面，添加潜在的重要标志，然后继续解析结果。这是一个简单的例子：

from selenium import webdriver

URL = ""

driver = webdriver.Firefox()
driver.get(URL)

with open("webpage.html", 'wb') as f:
    f.write(driver.page_source)

driver.close()
最新问题
© www.soinside.com 2019 - 2024. All rights reserved.