如何在 R 中上传多个文件而不收到 421 Too Many Connections 错误？

问题描述 投票：0回答：3

我需要使用 

RCurl
将超过 65 个文件上传到我的托管 ftp 服务器，但在上传多个文件后我收到错误：

 < 421 Too many connections (8) from this IP
* We got a 421 - timeout!
* Closing connection 291

从我在日志中看到的内容来看，它显示 

Connection #283 to host ftp.myserver.com left intact
，因此无需进一步授权，直到连接在 15 分钟不活动后过期 
< 220 You will be disconnected after 15 minutes of inactivity.
。

至少这是我的理解，因为出于某种原因，它会在第二个文件之后重新发送用户和密码：

> USER [email protected]
< 331 User [email protected] OK. Password required
> PASS mypassword
< 230 OK. Current restricted directory is /

我知道有多个授权请求（每个文件一个），因此我淹没了服务器。

是否可以先打开与 FTP 服务器的连接，然后上传所有文件？就像 FileZilla 一样？

请找到我使用的 R 代码：

for(file in list.files(localPath)) {

  ftpUpload(paste0(localPath,file), userpwd = userpwd, to=paste0("ftp://", ftpPath, file), verbose=TRUE)

}

另一个 SO 成员在这里也有同样的问题，但看起来我已经使用了类似于向他提供的响应的东西，但它不起作用。 使用R上传多个文件

谢谢你

编辑（添加日志）：

> for(file in list.files(localPath)) {
+   
+   print(paste0("ftp://", ftpPath, file))
+   ftpUpload(paste0(localPath,file), userpwd = userpwd, to=paste0("ftp://", ftpPath, file), verbose=TRUE)
+ 
+ }
[1] "ftp://ftp.mywebsite.com/data/csv/data_private_allocation-1.csv"
* Hostname was NOT found in DNS cache
*   Trying 11.22.33.44...
* Connected to ftp.mywebsite.com (11.22.33.44) port 21 (#310)
< 220---------- Welcome to Pure-FTPd [privsep] [TLS] ----------
< 220-You are user number 2 of 1000 allowed.
< 220-Local time is now 08:50. Server port: 21.
< 220-This is a private system - No anonymous login
< 220-IPv6 connections are also welcome on this server.
< 220 You will be disconnected after 15 minutes of inactivity.
> USER [email protected]
< 331 User [email protected] OK. Password required
> PASS mypassword
< 230 OK. Current restricted directory is /
> PWD
< 257 "/" is your current location
* Entry path is '/'
> CWD portfolios
* ftp_perform ends with SECONDARY: 0
< 250 OK. Current directory is /portfolios
> CWD csv
< 250 OK. Current directory is /data/csv
> EPSV
* Connect data stream passively
< 229 Extended Passive mode OK (|||44516|)
* Hostname was NOT found in DNS cache
*   Trying 11.22.33.44...
* Connecting to 11.22.33.44 (11.22.33.44) port 44516
* Connected to ftp.mywebsite.com (11.22.33.44) port 21 (#310)
> TYPE I
< 200 TYPE is now 8-bit binary
> STOR data_private_allocation-1.csv
< 150 Accepted data connection
* Remembering we are in dir "data/csv/"
< 226-File successfully transferred
< 226 0.180 seconds (measured here), 1.15 Kbytes per second
* Connection #310 to host ftp.mywebsite.com left intact
[1] "ftp://ftp.mywebsite.com/data/csv/data_private_allocation-2.csv"
* Hostname was NOT found in DNS cache
*   Trying 11.22.33.44...
* Connected to ftp.mywebsite.com (11.22.33.44) port 21 (#311)
< 220---------- Welcome to Pure-FTPd [privsep] [TLS] ----------
< 220-You are user number 3 of 1000 allowed.
< 220-Local time is now 08:50. Server port: 21.
< 220-This is a private system - No anonymous login
< 220-IPv6 connections are also welcome on this server.
< 220 You will be disconnected after 15 minutes of inactivity.
> USER [email protected]
< 331 User [email protected] OK. Password required
> PASS mypassword
< 230 OK. Current restricted directory is /
> PWD
< 257 "/" is your current location
* Entry path is '/'
> CWD portfolios
* ftp_perform ends with SECONDARY: 0
< 250 OK. Current directory is /portfolios
> CWD csv
< 250 OK. Current directory is /data/csv
> EPSV
* Connect data stream passively
< 229 Extended Passive mode OK (|||45077|)
* Hostname was NOT found in DNS cache
*   Trying 11.22.33.44...
* Connecting to 11.22.33.44 (11.22.33.44) port 45077
* Connected to ftp.mywebsite.com (11.22.33.44) port 21 (#311)
> TYPE I
< 200 TYPE is now 8-bit binary
> STOR data_private_allocation-2.csv
< 150 Accepted data connection
* Remembering we are in dir "data/csv/"
< 226-File successfully transferred
< 226 0.207 seconds (measured here), 1.00 Kbytes per second
* Connection #311 to host ftp.mywebsite.com left intact
[1] "ftp://ftp.mywebsite.com/data/csv/data_private_allocation-3.csv"
* Hostname was NOT found in DNS cache
*   Trying 11.22.33.44...
* Connected to ftp.mywebsite.com (11.22.33.44) port 21 (#312)
< 220---------- Welcome to Pure-FTPd [privsep] [TLS] ----------
< 220-You are user number 4 of 1000 allowed.
< 220-Local time is now 08:50. Server port: 21.
< 220-This is a private system - No anonymous login
< 220-IPv6 connections are also welcome on this server.
< 220 You will be disconnected after 15 minutes of inactivity.
> USER [email protected]
< 331 User [email protected] OK. Password required
> PASS mypassword
< 230 OK. Current restricted directory is /
> PWD
< 257 "/" is your current location
* Entry path is '/'
> CWD portfolios
* ftp_perform ends with SECONDARY: 0
< 250 OK. Current directory is /portfolios
> CWD csv
< 250 OK. Current directory is /data/csv
> EPSV
* Connect data stream passively
< 229 Extended Passive mode OK (|||40390|)
* Hostname was NOT found in DNS cache
*   Trying 11.22.33.44...
* Connecting to 11.22.33.44 (11.22.33.44) port 40390
* Connected to ftp.mywebsite.com (11.22.33.44) port 21 (#312)
> TYPE I
< 200 TYPE is now 8-bit binary
> STOR data_private_allocation-3.csv
< 150 Accepted data connection
* Remembering we are in dir "data/csv/"
< 226-File successfully transferred
< 226 0.189 seconds (measured here), 1.12 Kbytes per second
* Connection #312 to host ftp.mywebsite.com left intact
[1] "ftp://ftp.mywebsite.com/data/csv/data_private_allocation-4.csv"
* Hostname was NOT found in DNS cache
*   Trying 11.22.33.44...
* Connected to ftp.mywebsite.com (11.22.33.44) port 21 (#313)
< 220---------- Welcome to Pure-FTPd [privsep] [TLS] ----------
< 220-You are user number 5 of 1000 allowed.
< 220-Local time is now 08:50. Server port: 21.
< 220-This is a private system - No anonymous login
< 220-IPv6 connections are also welcome on this server.
< 220 You will be disconnected after 15 minutes of inactivity.
> USER [email protected]
< 331 User [email protected] OK. Password required
> PASS mypassword
< 230 OK. Current restricted directory is /
> PWD
< 257 "/" is your current location
* Entry path is '/'
> CWD portfolios
* ftp_perform ends with SECONDARY: 0
< 250 OK. Current directory is /portfolios
> CWD csv
< 250 OK. Current directory is /data/csv
> EPSV
* Connect data stream passively
< 229 Extended Passive mode OK (|||46329|)
* Hostname was NOT found in DNS cache
*   Trying 11.22.33.44...
* Connecting to 11.22.33.44 (11.22.33.44) port 46329
* Connected to ftp.mywebsite.com (11.22.33.44) port 21 (#313)
> TYPE I
< 200 TYPE is now 8-bit binary
> STOR data_private_allocation-4.csv
< 150 Accepted data connection
* Remembering we are in dir "data/csv/"
< 226-File successfully transferred
< 226 0.187 seconds (measured here), 1.11 Kbytes per second
* Connection #313 to host ftp.mywebsite.com left intact
[1] "ftp://ftp.mywebsite.com/data/csv/data_private_allocation_protected-1.csv"
* Hostname was NOT found in DNS cache
*   Trying 11.22.33.44...
* Connected to ftp.mywebsite.com (11.22.33.44) port 21 (#314)
< 220---------- Welcome to Pure-FTPd [privsep] [TLS] ----------
< 220-You are user number 6 of 1000 allowed.
< 220-Local time is now 08:50. Server port: 21.
< 220-This is a private system - No anonymous login
< 220-IPv6 connections are also welcome on this server.
< 220 You will be disconnected after 15 minutes of inactivity.
> USER [email protected]
< 331 User [email protected] OK. Password required
> PASS mypassword
< 230 OK. Current restricted directory is /
> PWD
< 257 "/" is your current location
* Entry path is '/'
> CWD portfolios
* ftp_perform ends with SECONDARY: 0
< 250 OK. Current directory is /portfolios
> CWD csv
< 250 OK. Current directory is /data/csv
> EPSV
* Connect data stream passively
< 229 Extended Passive mode OK (|||46556|)
* Hostname was NOT found in DNS cache
*   Trying 11.22.33.44...
* Connecting to 11.22.33.44 (11.22.33.44) port 46556
* Connected to ftp.mywebsite.com (11.22.33.44) port 21 (#314)
> TYPE I
< 200 TYPE is now 8-bit binary
> STOR data_private_allocation_protected-1.csv
< 150 Accepted data connection
* Remembering we are in dir "data/csv/"
< 226-File successfully transferred
< 226 0.186 seconds (measured here), 2.75 Kbytes per second
* Connection #314 to host ftp.mywebsite.com left intact
[1] "ftp://ftp.mywebsite.com/data/csv/data_private_allocation_protected-2.csv"
* Hostname was NOT found in DNS cache
*   Trying 11.22.33.44...
* Connected to ftp.mywebsite.com (11.22.33.44) port 21 (#315)
< 220---------- Welcome to Pure-FTPd [privsep] [TLS] ----------
< 220-You are user number 7 of 1000 allowed.
< 220-Local time is now 08:50. Server port: 21.
< 220-This is a private system - No anonymous login
< 220-IPv6 connections are also welcome on this server.
< 220 You will be disconnected after 15 minutes of inactivity.
> USER [email protected]
< 331 User [email protected] OK. Password required
> PASS mypassword
< 230 OK. Current restricted directory is /
> PWD
< 257 "/" is your current location
* Entry path is '/'
> CWD portfolios
* ftp_perform ends with SECONDARY: 0
< 250 OK. Current directory is /portfolios
> CWD csv
< 250 OK. Current directory is /data/csv
> EPSV
* Connect data stream passively
< 229 Extended Passive mode OK (|||43730|)
* Hostname was NOT found in DNS cache
*   Trying 11.22.33.44...
* Connecting to 11.22.33.44 (11.22.33.44) port 43730
* Connected to ftp.mywebsite.com (11.22.33.44) port 21 (#315)
> TYPE I
< 200 TYPE is now 8-bit binary
> STOR data_private_allocation_protected-2.csv
< 150 Accepted data connection
* Remembering we are in dir "data/csv/"
< 226-File successfully transferred
< 226 0.190 seconds (measured here), 2.69 Kbytes per second
* Connection #315 to host ftp.mywebsite.com left intact
[1] "ftp://ftp.mywebsite.com/data/csv/data_private_allocation_protected-3.csv"
* Hostname was NOT found in DNS cache
*   Trying 11.22.33.44...
* Connected to ftp.mywebsite.com (11.22.33.44) port 21 (#316)
< 220---------- Welcome to Pure-FTPd [privsep] [TLS] ----------
< 220-You are user number 8 of 1000 allowed.
< 220-Local time is now 08:50. Server port: 21.
< 220-This is a private system - No anonymous login
< 220-IPv6 connections are also welcome on this server.
< 220 You will be disconnected after 15 minutes of inactivity.
> USER [email protected]
< 331 User [email protected] OK. Password required
> PASS mypassword
< 230 OK. Current restricted directory is /
> PWD
< 257 "/" is your current location
* Entry path is '/'
> CWD portfolios
* ftp_perform ends with SECONDARY: 0
< 250 OK. Current directory is /portfolios
> CWD csv
< 250 OK. Current directory is /data/csv
> EPSV
* Connect data stream passively
< 229 Extended Passive mode OK (|||43399|)
* Hostname was NOT found in DNS cache
*   Trying 11.22.33.44...
* Connecting to 11.22.33.44 (11.22.33.44) port 43399
* Connected to ftp.mywebsite.com (11.22.33.44) port 21 (#316)
> TYPE I
< 200 TYPE is now 8-bit binary
> STOR data_private_allocation_protected-3.csv
< 150 Accepted data connection
* Remembering we are in dir "data/csv/"
< 226-File successfully transferred
< 226 0.196 seconds (measured here), 2.61 Kbytes per second
* Connection #316 to host ftp.mywebsite.com left intact
[1] "ftp://ftp.mywebsite.com/data/csv/data_private_allocation_protected-4.csv"
* Hostname was NOT found in DNS cache
*   Trying 11.22.33.44...
* Connected to ftp.mywebsite.com (11.22.33.44) port 21 (#317)
< 220---------- Welcome to Pure-FTPd [privsep] [TLS] ----------
< 220-You are user number 9 of 1000 allowed.
< 220-Local time is now 08:50. Server port: 21.
< 220-This is a private system - No anonymous login
< 220-IPv6 connections are also welcome on this server.
< 220 You will be disconnected after 15 minutes of inactivity.
> USER [email protected]
< 331 User [email protected] OK. Password required
> PASS mypassword
< 230 OK. Current restricted directory is /
> PWD
< 257 "/" is your current location
* Entry path is '/'
> CWD portfolios
* ftp_perform ends with SECONDARY: 0
< 250 OK. Current directory is /portfolios
> CWD csv
< 250 OK. Current directory is /data/csv
> EPSV
* Connect data stream passively
< 229 Extended Passive mode OK (|||48536|)
* Hostname was NOT found in DNS cache
*   Trying 11.22.33.44...
* Connecting to 11.22.33.44 (11.22.33.44) port 48536
* Connected to ftp.mywebsite.com (11.22.33.44) port 21 (#317)
> TYPE I
< 200 TYPE is now 8-bit binary
> STOR data_private_allocation_protected-4.csv
< 150 Accepted data connection
* Remembering we are in dir "data/csv/"
[1] "ftp://ftp.mywebsite.com/data/csv/data_coins_average-1.csv"
< 226-File successfully transferred
< 226 0.184 seconds (measured here), 2.77 Kbytes per second
* Connection #317 to host ftp.mywebsite.com left intact
* Hostname was NOT found in DNS cache
*   Trying 11.22.33.44...
* Connected to ftp.mywebsite.com (11.22.33.44) port 21 (#318)
< 421 Too many connections (8) from this IP
* We got a 421 - timeout!
* Closing connection 318
Error in function (type, msg, asError = TRUE)  : 
  Uploading to a URL without a file name!
r ftp rcurl
3个回答
0
投票

这是一种解决方法，而不是解决我的问题，但它的作用就像一个魅力。我创建了一个文本文件，其中包含 

ftp
命令，然后使用 R 
ftp
启动 Ubuntu 
system()
。

ftpcommands.txt：

open ftp.mywebsite.com
user [email protected] mypassword
lcd ~/R/data/csv
cd data/csv
prompt
mput *.csv
bye

然后是

R
命令：

system("ftp -n < ~/R/data/ftpcommands.txt")
0
投票

看起来您正在为每个文件打开一个新连接，这不是“文件传输协议”的用途。一些不同的解决方案：

在
    R
  1. 代码中打开更少的连接。 限制
    2. curl
  2. 使用的最大并发连接数。 增加
    3. FTP服务器
  3. 配置中允许的并发连接限制。 （只有当您确实需要时才执行此操作，正如我提到的，不要为每个文件打开一个连接。）
    4. 查看
此详细答案

以更好地了解问题。不要误解我的意思，同时拥有多个连接对性能有好处，但对每个文件来说却不是这样，这绝对太多了。大多数 FTP 客户端使用 2。

0
投票
RCurl::ftpUpload()

保持连接打开，但随后在每次新传输时建立新连接。

我找到了一个解决方案，通过设置 
options = list(forbid.reuse = TRUE)
 强制关闭第一个连接，允许您在后续上传时建立一个新连接，而不会遇到 421 太多连接。
    

	

    

    


  

  

    
最新问题


  





  

    © www.soinside.com 2019 - 2024. All rights reserved.