我最近尝试在我用 Snakemake 制作的 jupyter notebook 中使用 Scanpy python 包。 Scanpy 安装在我在 Snakemake 的 .yaml 中明确说明的 conda 环境中。 运行作业时：
snakemake --cores 1 results/output.h5ad --use-conda
conda环境加载成功，但是Snakemake没有找到模块报错：
Building DAG of jobs...
Using shell: /bin/bash
Provided cores: 1 (use --cores to define parallelism)
Rules claiming more threads will be scaled down.
Job stats:
job count min threads max threads
------- ------- ------------- -------------
load_h5 1 1 1
total 1 1 1
Select jobs to execute...
[Fri Feb 24 13:35:29 2023]
rule load_h5:
input: input/filtered_feature_bc_matrix.h5
output: results/output.h5ad
jobid: 0
reason: Missing output files: results/output.h5ad
resources: tmpdir=/var/folders/w7/zvnr_nqd4f3_2kdw0259s26r0000gq/T
Activating conda environment: .snakemake/conda/8d45bb2abfce310beb1752237b93c097_
Traceback (most recent call last):
File "/Users/usr/miniconda3/bin/jupyter-nbconvert", line 10, in <module>
sys.exit(main())
File "/Users/usr/miniconda3/lib/python3.9/site-packages/jupyter_core/application.py", line 277, in launch_instance
return super().launch_instance(argv=argv, **kwargs)
File "/Users/usr/miniconda3/lib/python3.9/site-packages/traitlets/config/application.py", line 1041, in launch_instance
app.start()
File "/Users/usr/miniconda3/lib/python3.9/site-packages/nbconvert/nbconvertapp.py", line 418, in start
self.convert_notebooks()
File "/Users/usr/miniconda3/lib/python3.9/site-packages/nbconvert/nbconvertapp.py", line 592, in convert_notebooks
self.convert_single_notebook(notebook_filename)
File "/Users/usr/miniconda3/lib/python3.9/site-packages/nbconvert/nbconvertapp.py", line 555, in convert_single_notebook
output, resources = self.export_single_notebook(
File "/Users/usr/miniconda3/lib/python3.9/site-packages/nbconvert/nbconvertapp.py", line 483, in export_single_notebook
output, resources = self.exporter.from_filename(
File "/Users/usr/miniconda3/lib/python3.9/site-packages/nbconvert/exporters/exporter.py", line 198, in from_filename
return self.from_file(f, resources=resources, **kw)
File "/Users/usr/miniconda3/lib/python3.9/site-packages/nbconvert/exporters/exporter.py", line 217, in from_file
return self.from_notebook_node(
File "/Users/usr/miniconda3/lib/python3.9/site-packages/nbconvert/exporters/notebook.py", line 36, in from_notebook_node
nb_copy, resources = super().from_notebook_node(nb, resources, **kw)
File "/Users/bduc1/miniconda3/lib/python3.9/site-packages/nbconvert/exporters/exporter.py", line 153, in from_notebook_node
nb_copy, resources = self._preprocess(nb_copy, resources)
File "/Users/bduc1/miniconda3/lib/python3.9/site-packages/nbconvert/exporters/exporter.py", line 349, in _preprocess
nbc, resc = preprocessor(nbc, resc)
File "/Users/bduc1/miniconda3/lib/python3.9/site-packages/nbconvert/preprocessors/base.py", line 48, in __call__
return self.preprocess(nb, resources)
File "/Users/bduc1/miniconda3/lib/python3.9/site-packages/nbconvert/preprocessors/execute.py", line 100, in preprocess
self.preprocess_cell(cell, resources, index)
File "/Users/usr/miniconda3/lib/python3.9/site-packages/nbconvert/preprocessors/execute.py", line 121, in preprocess_cell
cell = self.execute_cell(cell, index, store_history=True)
File "/Users/usr/miniconda3/lib/python3.9/site-packages/jupyter_core/utils/__init__.py", line 168, in wrapped
return loop.run_until_complete(inner)
File "/Users/usr/miniconda3/lib/python3.9/site-packages/nest_asyncio.py", line 90, in run_until_complete
return f.result()
File "/Users/usr/miniconda3/lib/python3.9/asyncio/futures.py", line 201, in result
raise self._exception
File "/Users/usr/miniconda3/lib/python3.9/asyncio/tasks.py", line 256, in __step
result = coro.send(None)
File "/Users/usr/miniconda3/lib/python3.9/site-packages/nbclient/client.py", line 1021, in async_execute_cell
await self._check_raise_for_error(cell, cell_index, exec_reply)
File "/Users/usr/miniconda3/lib/python3.9/site-packages/nbclient/client.py", line 915, in _check_raise_for_error
raise CellExecutionError.from_cell_and_msg(cell, exec_reply_content)
nbclient.exceptions.CellExecutionError: An error occurred while executing the following cell:
------------------
# start coding here
import scanpy as sc
------------------
---------------------------------------------------------------------------
ModuleNotFoundError Traceback (most recent call last)
/var/folders/w7/zvnr_nqd4f3_2kdw0259s26r0000gq/T/ipykernel_34677/1324428204.py in <cell line: 2>()
1 # start coding here
----> 2 import scanpy as sc
ModuleNotFoundError: No module named 'scanpy'
ModuleNotFoundError: No module named 'scanpy'
[Fri Feb 24 13:35:35 2023]
Error in rule load_h5:
jobid: 0
input: input/filtered_feature_bc_matrix.h5
output: results/output.h5ad
conda-env: /Users/bduc1/projects/wspace/project_brm/.snakemake/conda/8d45bb2abfce310beb1752237b93c097_
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
为了确保 Snakemake 创建的环境正常工作，我激活了 Snakemake 生成的环境：
conda activate .snakemake/conda/8d45bb2abfce310beb1752237b93c097_
这行得通，所以我检查使用的是哪个 Python：
which python
/Users/usr/projects/wspace/project/.snakemake/conda/8d45bb2abfce310beb1752237b93c097_/bin/python
这看起来不错，所以我尝试使用 Scanpy：
$ python3
Python 3.10.8 | packaged by conda-forge | (main, Nov 22 2022, 08:27:35) [Clang 14.0.6 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import scanpy as sc
>>> test = sc.read_10x_h5("/Users/usr/projects/wspace/project/input/filtered_feature_bc_matrix.h5")
/Users/usr/projects/wspace/project/.snakemake/conda/8d45bb2abfce310beb1752237b93c097_/lib/python3.10/site-packages/anndata/_core/anndata.py:1830: UserWarning: Variable names are not unique. To make them unique, call `.var_names_make_unique`.
所以 Scanpy 在这种环境下工作！ 但是不知何故 Snakemake 找不到 Scanpy 模块！
我在 macOS Monterey 12.5.1 上使用 Snakemake 7.22.0.
任何帮助将不胜感激！
Snakefile 规则是：
rule load_h5:
output:
"results/output.h5ad"
input:
"input/filtered_feature_bc_matrix.h5"
conda:
"envs/scanpy_min.yaml"
notebook:
"notebooks/01_load_scanpy.py.ipynb"
scanpy_min.yaml 是：
name: scanpy_min
channels:
- conda-forge
- bioconda
- defaults
dependencies:
- anndata=0.8.0=pyhd8ed1ab_1
- appdirs=1.4.4=pyh9f0ad1d_0
- appnope=0.1.3=pyhd8ed1ab_0
- arpack=3.7.0=hefb7bc6_2
- asttokens=2.2.1=pyhd8ed1ab_0
- backcall=0.2.0=pyh9f0ad1d_0
- backports=1.0=pyhd8ed1ab_3
- backports.functools_lru_cache=1.6.4=pyhd8ed1ab_0
- blosc=1.21.2=hebb52c4_0
- brotli=1.0.9=hb7f2c08_8
- brotli-bin=1.0.9=hb7f2c08_8
- brotlipy=0.7.0=py310h90acd4f_1005
- bzip2=1.0.8=h0d85af4_4
- c-ares=1.18.1=h0d85af4_0
- ca-certificates=2022.12.7=h033912b_0
- cached-property=1.5.2=hd8ed1ab_1
- cached_property=1.5.2=pyha770c72_1
- certifi=2022.12.7=pyhd8ed1ab_0
- cffi=1.15.1=py310ha78151a_3
- charset-normalizer=2.1.1=pyhd8ed1ab_0
- colorama=0.4.6=pyhd8ed1ab_0
- comm=0.1.2=pyhd8ed1ab_0
- contourpy=1.0.7=py310ha23aa8a_0
- cryptography=39.0.0=py310hdd0c95c_0
- cycler=0.11.0=pyhd8ed1ab_0
- debugpy=1.6.6=py310h7a76584_0
- decorator=5.1.1=pyhd8ed1ab_0
- entrypoints=0.4=pyhd8ed1ab_0
- et_xmlfile=1.1.0=pyhd8ed1ab_0
- executing=1.2.0=pyhd8ed1ab_0
- fonttools=4.38.0=py310h90acd4f_1
- freetype=2.12.1=h3f81eb7_1
- glpk=5.0=h3cb5acd_0
- gmp=6.2.1=h2e338ed_0
- h5py=3.8.0=nompi_py310h5555e59_100
- hdf5=1.12.2=nompi_h48135f9_101
- icu=70.1=h96cf925_0
- idna=3.4=pyhd8ed1ab_0
- igraph=0.10.3=h020c493_0
- importlib-metadata=6.0.0=pyha770c72_0
- importlib_metadata=6.0.0=hd8ed1ab_0
- ipykernel=6.20.2=pyh736e0ef_0
- ipython=8.8.0=pyhd1c38e8_0
- jedi=0.18.2=pyhd8ed1ab_0
- joblib=1.2.0=pyhd8ed1ab_0
- jpeg=9e=hac89ed1_2
- jupyter_client=7.4.9=pyhd8ed1ab_0
- jupyter_core=5.1.5=py310h2ec42d9_0
- kiwisolver=1.4.4=py310ha23aa8a_1
- krb5=1.20.1=h049b76e_0
- lcms2=2.14=h29502cd_1
- leidenalg=0.9.1=py310h7a76584_0
- lerc=4.0.0=hb486fe8_0
- libaec=1.0.6=hf0c8a7f_1
- libblas=3.9.0=16_osx64_openblas
- libbrotlicommon=1.0.9=hb7f2c08_8
- libbrotlidec=1.0.9=hb7f2c08_8
- libbrotlienc=1.0.9=hb7f2c08_8
- libcblas=3.9.0=16_osx64_openblas
- libcurl=7.87.0=h6df9250_0
- libcxx=14.0.6=hccf4f1f_0
- libdeflate=1.17=hac1461d_0
- libedit=3.1.20191231=h0678c8f_2
- libev=4.33=haf1e3a3_1
- libffi=3.4.2=h0d85af4_5
- libgfortran=5.0.0=11_3_0_h97931a8_27
- libgfortran5=11.3.0=h082f757_27
- libiconv=1.17=hac89ed1_0
- libjpeg-turbo=2.1.4=hb7f2c08_0
- liblapack=3.9.0=16_osx64_openblas
- libllvm11=11.1.0=h8fb7429_5
- libnghttp2=1.51.0=he2ab024_0
- libopenblas=0.3.21=openmp_h429af6e_3
- libpng=1.6.39=ha978bb4_0
- libsodium=1.0.18=hbcb3906_1
- libsqlite=3.40.0=ha978bb4_0
- libssh2=1.10.0=h47af595_3
- libtiff=4.5.0=hee9004a_2
- libwebp-base=1.2.4=h775f41a_0
- libxcb=1.13=h0d85af4_1004
- libxml2=2.10.3=hb9e07b5_0
- libzlib=1.2.13=hfd90126_4
- llvm-openmp=15.0.7=h61d9ccf_0
- llvmlite=0.39.1=py310h2bfb868_1
- lz4-c=1.9.4=hf0c8a7f_0
- matplotlib-base=3.6.3=py310he725631_0
- matplotlib-inline=0.1.6=pyhd8ed1ab_0
- metis=5.1.0=h2e338ed_1006
- mpfr=4.1.0=h0f52abe_1
- munkres=1.1.4=pyh9f0ad1d_0
- natsort=8.2.0=pyhd8ed1ab_0
- ncurses=6.3=h96cf925_1
- nest-asyncio=1.5.6=pyhd8ed1ab_0
- networkx=3.0=pyhd8ed1ab_0
- numba=0.56.4=py310h62db5c2_0
- numexpr=2.8.3=py310hecf8f37_1
- numpy=1.23.5=py310h1b7c290_0
- openjpeg=2.5.0=h13ac156_2
- openpyxl=3.1.0=py310h90acd4f_0
- openssl=3.0.8=hfd90126_0
- packaging=23.0=pyhd8ed1ab_0
- pandas=1.5.3=py310hecf8f37_0
- parso=0.8.3=pyhd8ed1ab_0
- patsy=0.5.3=pyhd8ed1ab_0
- pexpect=4.8.0=pyh1a96a4e_2
- pickleshare=0.7.5=py_1003
- pillow=9.4.0=py310hab5364c_0
- pip=22.3.1=pyhd8ed1ab_0
- platformdirs=2.6.2=pyhd8ed1ab_0
- pooch=1.6.0=pyhd8ed1ab_0
- prompt-toolkit=3.0.36=pyha770c72_0
- psutil=5.9.4=py310h90acd4f_0
- pthread-stubs=0.4=hc929b4f_1001
- ptyprocess=0.7.0=pyhd3deb0d_0
- pure_eval=0.2.2=pyhd8ed1ab_0
- pycparser=2.21=pyhd8ed1ab_0
- pygments=2.14.0=pyhd8ed1ab_0
- pynndescent=0.5.8=pyh1a96a4e_0
- pyopenssl=23.0.0=pyhd8ed1ab_0
- pyparsing=3.0.9=pyhd8ed1ab_0
- pysocks=1.7.1=pyha2e5f31_6
- pytables=3.7.0=py310h90ba602_3
- python=3.10.8=he7542f4_0_cpython
- python-dateutil=2.8.2=pyhd8ed1ab_0
- python-igraph=0.10.3=py310hedfac68_0
- python_abi=3.10=3_cp310
- pytz=2022.7.1=pyhd8ed1ab_0
- pyzmq=25.0.0=py310hf615a82_0
- readline=8.1.2=h3899abd_0
- requests=2.28.2=pyhd8ed1ab_0
- scanpy=1.9.1=pyhd8ed1ab_0
- scikit-learn=1.2.1=py310hcebe997_0
- scipy=1.10.0=py310h240c617_0
- seaborn=0.12.2=hd8ed1ab_0
- seaborn-base=0.12.2=pyhd8ed1ab_0
- session-info=1.0.0=pyhd8ed1ab_0
- setuptools=66.1.1=pyhd8ed1ab_0
- six=1.16.0=pyh6c4a22f_0
- snappy=1.1.9=h225ccf5_2
- stack_data=0.6.2=pyhd8ed1ab_0
- statsmodels=0.13.5=py310h936d966_2
- stdlib-list=0.8.0=pyhd8ed1ab_0
- suitesparse=5.10.1=h7aff33d_1
- tbb=2021.7.0=hb8565cd_1
- texttable=1.6.7=pyhd8ed1ab_0
- threadpoolctl=3.1.0=pyh8a188c0_0
- tk=8.6.12=h5dbffcc_0
- tornado=6.2=py310h90acd4f_1
- tqdm=4.64.1=pyhd8ed1ab_0
- traitlets=5.8.1=pyhd8ed1ab_0
- typing-extensions=4.4.0=hd8ed1ab_0
- typing_extensions=4.4.0=pyha770c72_0
- tzdata=2022g=h191b570_0
- umap-learn=0.5.3=py310h2ec42d9_0
- unicodedata2=15.0.0=py310h90acd4f_0
- urllib3=1.26.14=pyhd8ed1ab_0
- wcwidth=0.2.6=pyhd8ed1ab_0
- wheel=0.38.4=pyhd8ed1ab_0
- xorg-libxau=1.0.9=h35c211d_0
- xorg-libxdmcp=1.1.3=h35c211d_0
- xz=5.2.6=h775f41a_0
- zeromq=4.3.4=he49afe7_1
- zipp=3.11.0=pyhd8ed1ab_0
- zstd=1.5.2=hbc0c0cd_6
prefix: /Users/usr/miniconda3/envs/scanpy_min
如果有人遇到同样的问题，我找到了答案。 在感兴趣的环境中只有 jupyter_core 和 jupyter_client 是不够的。 我使用 mamba 安装了 jupyter lab，它解决了所有问题！