我对如何使用vba在网络上单击和搜索有疑问

Question

我对如何使用vba在网络上单击和搜索有疑问。我已经编写了代码，但是找不到如何单击此Web中的按钮

Sub LEISearch()

    'dimension (declare or set aside memory for) our variables
    Dim objIE As InternetExplorer 'special object variable representing the IE browser
    Dim LEI As HTMLLinkElement 'special object variable for an <a> (link) element
    Dim y As Integer 'integer variable we'll use as a counter
    Dim result As String 'string variable that will hold our result link'
    Dim result2 As String


    'initiating a new instance of Internet Explorer and asigning it to objIE
    Set objIE = New InternetExplorer

    'make IE browser visible (False would allow IE to run in the background)
    objIE.Visible = True

    'navigate IE to this web page (a pretty neat search engine really)
    objIE.navigate "https://www.gmeiutility.org/search.jsp?keyWord"

    'wait here a few seconds while the browser is busy
    Do While objIE.Busy = True Or objIE.readyState <> 4: DoEvents: Loop

    'in the search box put cell "A2" value, the word "in" and cell "C1" value
    objIE.document.getElementById("searchInput").Value = _
        Sheets("Macro1").Range("A1").Value

    'click the 'go' button
    Set LEIButton = objIE.document.getElementsByClassName("hiddenSubmitButton")
    LEIButton.Focus
    LEIButton.Click


    'wait again for the browser
    Do While objIE.Busy = True Or objIE.readyState <> 4: DoEvents: Loop
End Sub

Answer 1

此案例是一个很好的，干净的网络抓取示例，因此我将借此机会介绍教育文章。

我极力建议避免在可能的情况下使用IE抓取网站。这是非常低效的。尤其是在这种情况下，可能会有多页结果。相反，您可以使用HTTP requests。

HTTP请求是一种从服务器请求内容的结构化方法。在这种情况下，我们要向服务器发送关键字并获取相应的搜索结果。

要了解此请求的外观，您必须在单击带有放大镜的按钮时检查网络流量。您可以通过浏览器的开发人员工具（如果使用Firefox，则为Ctrl + Shift + E）来做到这一点：

如果查看请求的标题和参数，您将看到url，正文和标题的外观。在这种特殊情况下，所有参数都被编码到url中，并且标头对于请求成功并非必不可少，因此您只需要url。

URL的某些参数是关键字，每页结果数和页数。

响应的有效负载为json格式。您可以使用this之类的工具检查其结构。它是这样的：

基本上，JSON响应由您指定的尽可能多的结果组成，应在每页显示（或更少）。要获取下一页，您需要使用相同的关键字发送新请求，但指定新的页码，依此类推。

事实上，正如您所看到的，该网站提供的数据比浏览器上显示的数据还要多，这可能很有用。

下面的代码搜索关键字test，同时每页请求25个结果。发送第一个请求以查找其中有多少页结果，然后代码循环遍历所有页并将结果打印在工作表中。

TL; DR

Option Explicit

Sub main()
Dim sht As Worksheet
Dim totalNumberOfPages As Long
Dim searchResults As Object
Dim pageNumber As Long
Dim results() As String
Dim entity As Object
Dim i As Long, j As Long
Dim rng As Range

Set sht = ThisWorkbook.Worksheets("Name of your Worksheet")
''''''First request to find out the number of pages''''''
Set searchResults = XHRrequest("test", 25, 1)           '
totalNumberOfPages = searchResults("totalPages")        '
'''''''''''''''''''''''''''''''''''''''''''''''''''''''''

'''''''''''''''''''Loop through all the pages''''''''''''''''''''''''''''''''
For pageNumber = 1 To totalNumberOfPages Step 1                             '
    Set searchResults = XHRrequest("test", 25, pageNumber)                  '
    ReDim results(1 To searchResults("entitySearchResult").Count, 1 To 7)   '
    i = 0                                                                   '
'''''''''''write the results in an array''''''''''''''''''''''''''''        '
    For Each entity In searchResults("entitySearchResult")         '        '
        i = i + 1                                                  '        '
        results(i, 1) = entity("LEINumber")                        '        '
        results(i, 2) = entity("legalName")                        '        '
        results(i, 3) = entity("city")                             '        '
        results(i, 4) = entity("headquartersCountry")              '        '
        results(i, 5) = entity("recordStatus")                     '        '
        results(i, 6) = entity("renewalStatus")                    '        '
        results(i, 7) = entity("entityStatus")                     '        '
    Next entity                                                    '        '
''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''        '
'''''''''''''''write all the results in the worksheet in one go'''''        '
    With sht                                                       '        '
        Set rng = .Range("A" & .Rows.Count).End(xlUp).Offset(1, 0) '        '
    End With                                                       '        '
    rng.Resize(UBound(results, 1), UBound(results, 2)) = results   '        '
''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''        '
Next pageNumber                                                             '
'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
End Sub

Public Function XHRrequest(ByVal key As String, ByVal resultsPerPage As Long, ByVal pageNumber As Long) As Object
Dim req As New WinHttpRequest
Dim url As String

url = "https://www.gmeiutility.org/actions/Search/?isPendingValidationChecked=true&isSearchAllLOUChecked=true&keyWord=" & key & "&page=" & pageNumber & "&resultsPerPage=" & resultsPerPage & "&searchType=baseSearch" 'build the URL according to the parameters

'''''''''Send the HTTP request'''''''''''''''''''''''''''''''
With req                                                    '
    .Open "POST", url, False                                '
    .send                                                   '
    Set XHRrequest = JsonConverter.ParseJson(.responseText) '
End With                                                    '
'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
End Function

出于演示目的，上面的代码在名为Name of your Worksheet的工作表中打印所有数据。

如果需要执行多次搜索，则可以轻松地修改代码以最适合您的需求。更具体地说，您可以循环浏览多个关键字，并使用这些关键字而不是“测试”来调用XHRrequest函数。

以下是输出示例：

您将需要在项目中添加以下参考（VBE>工具>参考）：

Microsoft WinHTTP Services version 5.1
Microsoft HTML Objects Library
Microsoft Scripting Runtime

您还需要将此JSON parser添加到您的项目中。请按照链接中的安装说明进行操作，然后就可以开始使用了。