===== dummy =====
==== tasks ====
* [[conda environment file to install different packages from different channels]]
* tags | conda-forge::foo
==== Remove an environment ====
To do it with no questions asked
conda env remove --name foo --yes
Ask for confirmation
conda env remove --name foo
If the environment was installed in a non standard location
conda env remove -p foo
where foo is the path of the environment.
tags | delete
related commands:
conda info --envs
conda remove --help
==== update environment ====
conda env update -f environment.yml
where environment.yml is the path of the environment file
Ref:- https://conda.io/docs/commands/env/conda-env-update.html
==== update conda ====
conda update -n base -c defaults conda --repodata-fn=repodata.json
conda update -n -c defaults --yes conda
For example
conda update -n base -c defaults --yes conda
To specify a particular version number, X.Y.Z (ex:- 4.5.4)
conda update -n base -c defaults --yes conda=X.Y.Z
To downgrade to a specific version
conda config --set allow_conda_downgrades true
conda install -n base -c defaults conda=4.6.14
tags | upgrade conda
Ref:-
* https://anaconda.org/anaconda/conda - shows the latest conda version available
==== command cheat sheet ====
^ purpose ^ command ^
| List all packages in the current environment | conda list |
| List the version of a package in all the conda environments |
for env in `conda info --envs | stuff.py | cut -f 1 -d ' '`
do
echo $env
conda list -n $env
echo
done
where stuff.py is from https://github.com/KamarajuKusumanchi/rutils/blob/master/python3/stuff.py
For example
for env in `conda info --envs | stuff.py | cut -f 1 -d ' '`
do
echo $env
conda list agate -n $env
echo
done gives something like
...
py310
# packages in environment at :
#
# Name Version Build Channel
agate 1.7.1 py310haa95532_0 defaults
agate-dbf 0.2.2 py_0 defaults
agate-excel 0.2.5 pyhd3eb1b0_0 defaults
agate-sql 0.5.8 pyhd3eb1b0_0 defaults
py311
# packages in environment at :
#
# Name Version Build Channel
agate 1.7.1 py311haa95532_0 defaults
agate-dbf 0.2.2 py_0 defaults
agate-excel 0.2.5 pyhd3eb1b0_0 defaults
agate-sql 0.5.8 pyhd3eb1b0_0 defaults
...
|
| Show the channel urls | conda list --show-channel-urls |
| Search packages available in Anaconda | conda search -f |
==== find the package version ====
tags | check package version, list packages in environment, show the package version
To find the version of a package installed
conda list -n env_name package_name
For example
$ conda list pandas -n py39
# packages in environment at C:\ProgramData\Continuum\Anaconda\envs\py39:
#
# Name Version Build Channel
pandas 1.2.4 py39hd77b12b_0
pandas-datareader 0.9.0 py_0
You can also use this technique to check if a package is installed.
For multiple packages
conda list -n env_name | grep -E "pkgA|pkgB"
For example
$ conda list -n py39 | grep -E "pandas|numpy"
numpy 1.20.1 py39h34a8a5c_0
numpy-base 1.20.1 py39haf7ebc8_0
pandas 1.2.4 py39hd77b12b_0
pandas-datareader 0.9.0 py_0
Ref:- https://conda.io/docs/commands/conda-list.html
==== list available package versions ====
List all packages in the current environment
$ conda list
# packages in environment at C:\ProgramData\Continuum\Anaconda\envs\numpy_issue:
#
# Name Version Build Channel
certifi 2019.3.9 py27_0
numpy 1.16.4 pypi_0 pypi
pip 19.1.1 py27_0
python 2.7.16 hcb6e200_0
setuptools 41.0.1 py27_0
sqlite 3.28.0 h0c8e037_0
vc 9 h7299396_1
vs2008_runtime 9.00.30729.1 hfaea7d5_1
wheel 0.33.4 py27_0
wincertstore 0.2 py27hf04cefb_0
Show the channel urls
$ conda list --show-channel-urls
# packages in environment at C:\ProgramData\Continuum\Anaconda\envs\numpy_issue:
#
# Name Version Build Channel
certifi 2019.3.9 py27_0 defaults
numpy 1.16.4 pypi_0 pypi
pip 19.1.1 py27_0 defaults
python 2.7.16 hcb6e200_0 defaults
setuptools 41.0.1 py27_0 defaults
sqlite 3.28.0 h0c8e037_0 defaults
vc 9 h7299396_1 defaults
vs2008_runtime 9.00.30729.1 hfaea7d5_1 defaults
wheel 0.33.4 py27_0 defaults
wincertstore 0.2 py27hf04cefb_0 defaults
Related:
* conda search -f - search packages available in Anaconda
==== reduce disk space usage ====
conda clean --dry-run --all
conda clean --all
Ref:-
* https://stackoverflow.com/questions/48706548/how-to-free-disk-space-taken-up-by-anaconda
* https://github.com/conda/conda/issues/8630 - recommends ''conda clean --all'' to fix ''conda json.decoder.JSONDecodeError: Unterminated string starting at: line xxxxxx column xx (char xxxxxxxx)''
==== where is the package cache ====
conda info
tags | conda cache directory
Related commands:
conda clean --all --dry-run
conda clean --packages --dry-run
==== where is my condarc file? ====
$ conda info | grep "user config file"
user config file : C:\Users\raju\.condarc
==== documentation links ====
* conda install - https://docs.conda.io/projects/conda/en/latest/commands/install.html
* conda update - https://docs.conda.io/projects/conda/en/latest/commands/update.html
* conda config - https://docs.conda.io/projects/conda/en/latest/commands/config.html
* pinning packages - https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-pkgs.html#preventing-packages-from-updating-pinning
==== which solver am I using? ====
Use
conda config --show solver
or
conda config --show-sources
Sample output:
$ conda config --show solver
solver: classic
$ conda config --show solver
solver: libmamba
$conda config --show-sources
==> C:\Users\root\.condarc <==
solver: libmamba
==> cmd_line <==
debug: False
json: False
See also:-
* https://conda.github.io/conda-libmamba-solver/getting-started/ - tells how to switch the default solver from classic to libmamba.
==== when was a conda environment created? ====
conda env list | grep -v '^#' | perl -lane 'print $F[-1]' | xargs ls -lrt1d
For example
% conda env list | grep -v '^#' | perl -lane 'print $F[-1]' | xargs ls -lrt1d
drwxr-xr-x 10 rajulocal rajulocal 4096 Apr 4 2020 /opt/rajulocal/miniconda3/envs/rutils
drwxr-xr-x 23 rajulocal rajulocal 4096 Nov 8 2021 /opt/rajulocal/miniconda3/envs/py39
drwxr-xr-x 13 rajulocal rajulocal 4096 Jun 6 2022 /opt/rajulocal/miniconda3/envs/play_ground
drwxr-xr-x 27 rajulocal rajulocal 4096 Feb 28 2023 /opt/rajulocal/miniconda3/envs/py310
drwxr-xr-x 13 rajulocal rajulocal 4096 May 20 2023 /opt/rajulocal/miniconda3/envs/test_yfinance
drwxr-xr-x 18 rajulocal rajulocal 4096 Jun 30 00:45 /opt/rajulocal/miniconda3
drwxr-xr-x 27 rajulocal rajulocal 4096 Jul 11 01:07 /opt/rajulocal/miniconda3/envs/py311
drwxr-xr-x 24 rajulocal rajulocal 4096 Nov 12 04:44 /opt/rajulocal/miniconda3/envs/market_data_processor
Ref: https://stackoverflow.com/questions/69109316/how-can-i-see-the-creation-dates-for-my-conda-environments
===== activate conda environment =====
tags | activate conda environment through shell script
==== source activate vs. conda activate ====
For conda >= 4.4.0, use "conda activate" instead of "source activate". The former is faster and the command remains same across OSes.
From https://github.com/conda/conda/blob/master/CHANGELOG.md#440-2017-12-20 -> "New Feature Highlights" section
> **conda activate**: The logic and mechanisms underlying environment activation have been reworked. With conda 4.4, ''conda activate'' and ''conda deactivate'' are now the preferred commands for activating and deactivating environments. You'll find they are much more snappy than the ''source activate'' and ''source deactivate'' commands from previous conda versions. The ''conda activate'' command also has advantages of (1) being universal across all OSes, shells, and platforms, and (2) not having path collisions with scripts from other packages like python virtualenv's activate script.
See also: [[#conda activate]], [[#source activate]]
==== conda activate ====
# As described in https://github.com/conda/conda/issues/8186
# The conda activate command is failing with
# conda.sh: line 55: PS1: unbound variable
# Until the issue is fixed, set +eu before calling "source activate"
# and set -eu afterwards
set +eux
# If the conda.sh is not sourced, then 'conda activate' is giving
# CommandNotFoundError: Your shell has not been properly configured
# to use 'conda activate'.
source "$(dirname $(which conda))/../etc/profile.d/conda.sh"
conda activate $ENV_NAME
set -eux
tested with conda 4.8.3 on a machine running RHEL 7.6
See also: [[#source activate vs. conda activate]], [[#source activate]]
==== source activate ====
# As described in https://github.com/conda/conda/issues/8186
# The conda activate command is failing with
# conda.sh: line 55: PS1: unbound variable
# Until the issue is fixed, set +eu before calling "source activate"
# and set -eu afterwards
set +eux
source activate $ENV_NAME
set -eux
tested with conda 4.8.3 on a machine running RHEL 7.6
See also: [[#source activate vs. conda activate]], [[#conda activate]]
===== Sample environment files =====
==== sample environment file ====
name: sample_py36
channels:
- conda-forge
- defaults
dependencies:
- python=3.6.8
- pandas=0.24.1
- pip:
- pywinauto
- fastavro==0.21.18
==== environment file to install pip package from a local repository ====
One way to specify a wheel file stored in a local directory as a pip dependency in a conda environment file is to do something like
name: test_ibapi
channels:
- defaults
dependencies:
- python=3.11
- pip
- pip:
- file:///home/rajulocal/software/dist/ibapi-10.22.1-py3-none-any.whl
but this will not be portable when you move to a different machine since the wheel file path (/home/rajulocal/software/dist/ibapi-10.22.1-py3-none-any.whl) is hard coded. A better approach is to run
% pip config set global.find-links file:///home/rajulocal/software/dist
which will modify pip.conf as
% cat ~/.config/pip/pip.conf
[global]
find-links = file:///home/rajulocal/software/dist
and then specify ibapi as a normal pip dependency.
name: test_ibapi
channels:
- defaults
dependencies:
- python=3.11
- pip
- pip:
- ibapi
Ref:-
* https://pip.pypa.io/en/stable/cli/pip_config/ - shows the syntax for the 'pip config' command
* https://pip.pypa.io/en/stable/topics/configuration/ - shows the locations of default pip configuration files.
* https://manpages.debian.org/bookworm/python3-pip/pip.1.en.html - where I found the --find-links option
tags | --find-links, pip config local packages, how to specify local repo location for pip packages, conda install pip package from a "local directory"
==== specify package version with git+https ====
name: test_chessli
channels:
- defaults
dependencies:
- python=3.11
- pandas=1.5.3
- pip
- pip:
- chessli
- git+https://github.com/lervag/apy.git@v0.6.0#egg=apy
Ref:-
* https://stackoverflow.com/questions/47617552/how-can-i-specify-a-custom-git-branch-in-my-pipfile - where I found the answer
* https://github.com/lervag/apy/tags?after=v0.9.1 - versions of the api package < 0.9.1
tags | pip "git+https" release, conda "git+https" release, conda git depend on a "specific release", specify package version with git+https in conda environment file
===== dummy =====
==== create an environment on the fly ====
Sample commands:
conda create -n python ipython prompt_toolkit=2 jupyter pandas numpy scipy scikit-learn matplotlib
conda create -n python=3.9 pandas pandas-datareader
==== commands I came across ====
$ conda config --add pinned_packages defaults::conda
To revert
$ conda config --remove-key pinned_packages
==== What is the difference between conda, pip, and virtualenv? ====
* https://docs.conda.io/projects/conda/en/latest/commands.html#conda-vs-pip-vs-virtualenv-commands -> scroll down to see a table that shows how to accomplish different tasks by using either conda or a combination pip and virtualenv.
==== conda release numbers ====
* After 4.14.0 (released on Aug 3, 2022), conda changed to CalVer versioning system. The release after 4.14.0 is 22.9.0 (released on Sep 20, 2022).
See also:
* https://github.com/conda/conda/releases - conda release history
* https://calver.org/ - CalVer - Calendar Versioning
==== twsapi conda environment file ====
* Build a wheel file from the zip file.
* See for example https://github.com/KamarajuKusumanchi/market_data_processor/blob/master/src/scripts/build_twsapi.sh which builds a wheel file from the TWS python api zip file (https://interactivebrokers.github.io/downloads/twsapi_macunix.1022.01.zip)
* Configure pip to install its wheels from a local wheel directory pip config set global.find-links file:///home/rajulocal/software/dist
* Install the package
* See for example https://github.com/KamarajuKusumanchi/rutils/blob/master/python3/envs/env_test_ibapi.yml
tags | conda envrionment install package from zip file, conda install zip package, install zip file