Snakemake：本地计算机上的SFTP

Question

我从本地计算机与远程服务器上的ssh连接我在远程服务器上运行Snakemake我想使用本地计算机上的文件作为规则的输入当然，由于我在服务器上运行Snakemake，因此服务器成为本地计算机，而本地计算机成为远程计算机（对于Snakemake）。

from snakemake.remote.SFTP import RemoteProvider

# I am not sure about the private key, is it the one I have on the server ?
# I have the same result with or without private_key anyway

# SFTP = RemoteProvider(port=22, username="myusername", private_key="/path/to/.ssh/id_rsa")
SFTP = RemoteProvider(port=22, username="myusername")

configfile : "config.json"

localrules: copyBclLocalToCluster

rule all:
    input:
        "copycluster.txt"

rule copyBclLocalToCluster:
    input:
        SFTP.remote("adress:path/to/filelocal.txt")
    output:
        "copycluster.txt"
    shell:
        "scp {input} {output}"

-----------------------------------------
Building DAG of jobs...
MissingInputException in line 26 of /path/to/Snakefile:
Missing input files for rule copyBclLocalToCluster:
adress:path/to/filelocal.txt

https://snakemake.readthedocs.io/en/stable/snakefiles/remote_files.html必须使用主机（域或IP地址）以及远程服务器上文件的绝对路径来指定所使用的远程文件地址。如果服务器上的SSH守护进程正在侦听RemoteProvider或remote（）的每个实例中22以外的端口，则可以指定端口：

文档说端口不应该是端口22，但是为什么呢？我真的很想使用它，因为我不知道如何配置另一个端口，而且我什至不确定是否拥有这样做的权利。

这真的是端口问题吗？或者我只是不了解如何在Snakemake中使用SFTP。

将本地计算机上的文件用作我的makenmake的输入的最佳方法是什么？

编辑

我使用了错误的私钥作为参数，因为当我尝试使用错误的私钥时，与没有指定私钥的情况下，我会遇到相同的错误。当我指定好的私钥时：

SFTP = RemoteProvider(port=22, username="myusername", private_key="/path/to/.ssh/id_rsa")
-----------------------------
Building DAG of jobs...
Traceback (most recent call last):
  File "/work/username/miniconda3/envs/RNAseq_snakemake/lib/python3.6/site-packages/snakemake/__init__.py", line 627, in snakemake
    batch=batch,
  File "/work/username/miniconda3/envs/RNAseq_snakemake/lib/python3.6/site-packages/snakemake/workflow.py", line 560, in execute
    dag.init()
  File "/work/username/miniconda3/envs/RNAseq_snakemake/lib/python3.6/site-packages/snakemake/dag.py", line 168, in init
    job = self.update([job], progress=progress)
  File "/work/username/miniconda3/envs/RNAseq_snakemake/lib/python3.6/site-packages/snakemake/dag.py", line 693, in update
    progress=progress,
  File "/work/username/miniconda3/envs/RNAseq_snakemake/lib/python3.6/site-packages/snakemake/dag.py", line 769, in update_
    progress=progress,
  File "/work/username/miniconda3/envs/RNAseq_snakemake/lib/python3.6/site-packages/snakemake/dag.py", line 693, in update
    progress=progress,
  File "/work/username/miniconda3/envs/RNAseq_snakemake/lib/python3.6/site-packages/snakemake/dag.py", line 757, in update_
    if not file.exists:
  File "/work/username/miniconda3/envs/RNAseq_snakemake/lib/python3.6/site-packages/snakemake/io.py", line 260, in exists
    return self.exists_remote
  File "/work/username/miniconda3/envs/RNAseq_snakemake/lib/python3.6/site-packages/snakemake/io.py", line 133, in wrapper
    v = func(self, *args, **kwargs)
  File "/work/username/miniconda3/envs/RNAseq_snakemake/lib/python3.6/site-packages/snakemake/io.py", line 312, in exists_remote
    return self.remote_object.exists()
  File "/work/username/miniconda3/envs/RNAseq_snakemake/lib/python3.6/site-packages/snakemake/remote/SFTP.py", line 85, in exists
    with self.sftpc() as sftpc:
  File "/work/username/miniconda3/envs/RNAseq_snakemake/lib/python3.6/contextlib.py", line 81, in __enter__
    return next(self.gen)
  File "/work/username/miniconda3/envs/RNAseq_snakemake/lib/python3.6/site-packages/snakemake/remote/SFTP.py", line 79, in sftpc
    conn = pysftp.Connection(*args_to_use, **kwargs_to_use)
  File "/work/username/miniconda3/envs/RNAseq_snakemake/lib/python3.6/site-packages/pysftp-0.2.9-py3.6.egg/pysftp/__init__.py", line 143, in __init__
    self._transport.connect(**self._tconnect)
  File "/work/username/miniconda3/envs/RNAseq_snakemake/lib/python3.6/site-packages/paramiko/transport.py", line 1333, in connect
    self.auth_publickey(username, pkey)
  File "/work/username/miniconda3/envs/RNAseq_snakemake/lib/python3.6/site-packages/paramiko/transport.py", line 1580, in auth_publickey
    return self.auth_handler.wait_for_response(my_event)
  File "/work/username/miniconda3/envs/RNAseq_snakemake/lib/python3.6/site-packages/paramiko/auth_handler.py", line 250, in wait_for_response
    raise e
paramiko.ssh_exception.AuthenticationException: Authentication failed.

如果我在服务器上的控制台上尝试sftp myusername@adress:path/to/filelocal.txt .，则可以正常工作

为什么在snakemake中不起作用？

Answer 1

因此，我在Snakemake的源代码中进行了快速又肮脏的搜索，Snakemake使用了ftputil，它需要用户名和密码。当您不提供Snakemake的ssh-key路径时，此密码将默认为None，然后将其传递给ftputil。

请参见Snakemake source。

我同意默认行为应默认为更明智的行为，例如~/.ssh/id_rsa.pub，但不幸的是，它不是。