我正在尝试使用 XML-RPC 从网站加载数据。到目前为止,这种方法效果很好,但不适用于需要登录的页面。我为此发送了我的用户名和密码。 我从同事那里知道cookies也是需要的。即 PHPSESSID 和另一个,我们称之为 OTHER_COOKIE ;) 我从浏览器中提取这些。
但是,我已经两天无法实际使用这些cookie了。我找不到让 XmlRpcClient 获取这些 cookie 的方法。我总是得到的答案是我无权访问这些页面。可能有像 XmlRpcTransport 或类似的类应该用于此目的,但我无法进一步了解。文档情况很糟糕。
这是至少适用于不需要登录的页面的代码。
由 DeepL.com 翻译(免费版)
(如果它起作用:我正在尝试阅读的页面是“MoinMoinWiki”。)
private final XmlRpcClient client;
public MoinMoinWikiClient(String baseUrl, String username, String password) throws Exception {
String endpointUrl = baseUrl + "/moinmoin/?action=xmlrpc2";
/*
EDIT: I now realise that using a username and password here is insane. That's exactly what cookies are for. I only need them for a manual login to create the cookies. Sorry, that was a bit stupid of me
*/
XmlRpcClientConfigImpl config = new XmlRpcClientConfigImpl() {
@Override
public String getBasicUserName() {
return username;
}
@Override
public String getBasicPassword() {
return password;
}
};
config.setServerURL(new URL(endpointUrl));
this.client = new XmlRpcClient();
client.setConfig(config);
String examplePageName = "any_page";
HashMap<String, Object> result = (HashMap<String, Object>)client.execute("getPageInfo", new Object[]{examplePageName});
info("Initialized XML-RPC client with URL: " + xmlRpcEndpoint);
}
我已经研究了两天的解决方案和文档。我问了ChatGPI很长时间,很详细;)
无数假设的解决方案都以类未知且无法添加到 Maven 的事实告终。例如CookieManager、CookiesStore ...
public MoinMoinWikiClient(String wikiUrl, String username, String password) throws Exception {
String endpointUrl = wikiUrl + "/moinmoin/?action=xmlrpc2";
/*
EDIT: I now realise that using a username and password here is insane. That's exactly what cookies are for. I only need them for a manual login to create the cookies. Sorry, that was a bit stupid of me
*/
XmlRpcClientConfigImpl config = new XmlRpcClientConfigImpl() {
@Override
public String getBasicUserName() {
return username;
}
@Override
public String getBasicPassword() {
return password;
}
};
config.setServerURL(new URL(endpointUrl));
BasicCookieStore cookieStore = new BasicCookieStore();
List<Cookie> cookies = new ArrayList<>();
cookies.add(new org.apache.http.impl.cookie.BasicClientCookie("PHPSESSID", Main.PHPSESSID));
cookies.add(new org.apache.http.impl.cookie.BasicClientCookie("OTHER_COOKIE", Main.OTHER_COOKIE));
cookieStore.addCookies(cookies.toArray(new Cookie[0]));
CloseableHttpClient httpClient = HttpClients.custom()
.setDefaultCookieStore(cookieStore)
.build();
this.client = new XmlRpcClient();
client.setConfig(config);
}
我终于成功了...! 这是代码:
import com.hacon.wiki.moinmoin.Config;
import org.apache.commons.httpclient.HttpClient;
import org.apache.commons.httpclient.cookie.CookiePolicy;
import org.apache.commons.httpclient.Cookie;
import org.apache.xmlrpc.XmlRpcException;
import org.apache.xmlrpc.client.XmlRpcClient;
import org.apache.xmlrpc.client.XmlRpcClientConfigImpl;
import org.apache.xmlrpc.client.XmlRpcCommonsTransportFactory;
import java.net.MalformedURLException;
import java.net.URL;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.stream.Collectors;
public class XmlRpcManager {
private String domain;
private List<Cookie> cookies;
public XmlRpcManager(String domain) {
this.domain = domain;
this.cookies = new ArrayList<>();
cookies.add(new Cookie(".xxx.de", "PHPSESSID", Config.PHPSESSID, "/", null, true));
cookies.add(new Cookie(".xxx.de", "XXXX_COOKIENAME", Config.XXXX_COOKIENAME, "/", null, true));
}
public List<String> getAllPages(String filter) {
String xmlRpcUrl = "https://xxxxxx/moinmoin/?action=xmlrpc2";
HttpClient httpClient = new HttpClient();
httpClient.getParams().setCookiePolicy(CookiePolicy.BROWSER_COMPATIBILITY);
if (cookies != null) {
for (Cookie cookie : cookies) {
httpClient.getState().addCookie(new org.apache.commons.httpclient.Cookie(
cookie.getDomain(),
cookie.getName(),
cookie.getValue(),
cookie.getPath(),
cookie.getExpiryDate(),
cookie.getSecure()
));
}
}
XmlRpcClientConfigImpl config = new XmlRpcClientConfigImpl();
try {
config.setServerURL(new URL(xmlRpcUrl));
} catch (MalformedURLException e) {
System.err.println("bad url format >> " + xmlRpcUrl);
throw new RuntimeException(e);
}
XmlRpcClient client = new XmlRpcClient();
XmlRpcCommonsTransportFactory factory = new XmlRpcCommonsTransportFactory(client);
factory.setHttpClient(httpClient);
client.setTransportFactory(factory);
client.setConfig(config);
Object result = null;
try {
result = client.execute("getAllPages", new Object[]{});
} catch (XmlRpcException e) {
System.err.println("failed");
throw new RuntimeException(e);
}
Object[] pagesArray = (Object[]) result;
List<String> pageNames = Arrays.stream(pagesArray)
.map(Object::toString)
.filter(s -> filter.isEmpty() || s.contains(filter))
.collect(Collectors.toList());
System.out.println("Found: " + String.valueOf(pageNames.size()));
return pageNames;
}
}