conda_notes []

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
===== dummy =====
==== tasks ====
  * [[conda environment file to install different packages from different channels]]
    * tags | conda-forge::foo

==== Remove an environment ====
To do it with no questions asked
<code>
conda env remove --name foo --yes
</code>

Ask for confirmation
<code>
conda env remove --name foo
</code>

If the environment was installed in a non standard location
<code>
conda env remove -p foo
</code>
where foo is the path of the environment.

tags | delete

related commands:
<code>
conda info --envs
conda remove --help
</code>

==== update environment ====
<code>
conda env update -f environment.yml
</code>
where environment.yml is the path of the environment file

Ref:- https://conda.io/docs/commands/env/conda-env-update.html

==== update conda ====
<code>
conda update -n base -c defaults conda --repodata-fn=repodata.json
</code>

<code>
conda update -n <env_name> -c defaults --yes conda
</code>
For example
<code>
conda update -n base -c defaults --yes conda
</code>
To specify a particular version number, X.Y.Z (ex:- 4.5.4)
<code>
conda update -n base -c defaults --yes conda=X.Y.Z
</code>

To downgrade to a specific version
<code>
conda config --set allow_conda_downgrades true
conda install -n base -c defaults conda=4.6.14
</code>

tags | upgrade conda

Ref:-
  * https://anaconda.org/anaconda/conda - shows the latest conda version available



==== command cheat sheet ====

^ purpose ^ command ^
| List all packages in the current environment | conda list | 
| List the version of a package in all the conda environments | <WRAP>
<code>
for env in `conda info --envs | stuff.py | cut -f 1 -d ' '`
do
  echo $env
  conda list <package_name> -n $env
  echo
done
</code>
where stuff.py is from https://github.com/KamarajuKusumanchi/rutils/blob/master/python3/stuff.py

For example <code>
for env in `conda info --envs | stuff.py | cut -f 1 -d ' '`
do
  echo $env
  conda list agate -n $env
  echo
done </code> gives something like <code>
...
py310
# packages in environment at <munged_path>:
#
# Name                    Version                   Build  Channel
agate                     1.7.1           py310haa95532_0    defaults
agate-dbf                 0.2.2                      py_0    defaults
agate-excel               0.2.5              pyhd3eb1b0_0    defaults
agate-sql                 0.5.8              pyhd3eb1b0_0    defaults

py311
# packages in environment at <munged_path>:
#
# Name                    Version                   Build  Channel
agate                     1.7.1           py311haa95532_0    defaults
agate-dbf                 0.2.2                      py_0    defaults
agate-excel               0.2.5              pyhd3eb1b0_0    defaults
agate-sql                 0.5.8              pyhd3eb1b0_0    defaults
...
</code> 
</WRAP> |
| Show the channel urls | conda list --show-channel-urls |
| Search packages available in Anaconda | conda search -f <package_name> |

==== find the package version ====
tags | check package version, list packages in environment, show the package version

To find the version of a package installed
<code>
conda list -n env_name package_name
</code>
For example
<code>
$ conda list pandas -n py39
# packages in environment at C:\ProgramData\Continuum\Anaconda\envs\py39:
#
# Name                    Version                   Build  Channel
pandas                    1.2.4            py39hd77b12b_0
pandas-datareader         0.9.0                      py_0
</code>

You can also use this technique to check if a package is installed.

For multiple packages
<code>
conda list -n env_name | grep -E "pkgA|pkgB"
</code>
For example
<code>
$ conda list -n py39 | grep -E "pandas|numpy"
numpy                     1.20.1           py39h34a8a5c_0
numpy-base                1.20.1           py39haf7ebc8_0
pandas                    1.2.4            py39hd77b12b_0
pandas-datareader         0.9.0                      py_0
</code>

Ref:- https://conda.io/docs/commands/conda-list.html

==== list available package versions ====

List all packages in the current environment

<code>
$ conda list
# packages in environment at C:\ProgramData\Continuum\Anaconda\envs\numpy_issue:
#
# Name                    Version                   Build  Channel
certifi                   2019.3.9                 py27_0
numpy                     1.16.4                   pypi_0    pypi
pip                       19.1.1                   py27_0
python                    2.7.16               hcb6e200_0
setuptools                41.0.1                   py27_0
sqlite                    3.28.0               h0c8e037_0
vc                        9                    h7299396_1
vs2008_runtime            9.00.30729.1         hfaea7d5_1
wheel                     0.33.4                   py27_0
wincertstore              0.2              py27hf04cefb_0
</code>

Show the channel urls

<code>
$ conda list --show-channel-urls
# packages in environment at C:\ProgramData\Continuum\Anaconda\envs\numpy_issue:
#
# Name                    Version                   Build  Channel
certifi                   2019.3.9                 py27_0    defaults
numpy                     1.16.4                   pypi_0    pypi
pip                       19.1.1                   py27_0    defaults
python                    2.7.16               hcb6e200_0    defaults
setuptools                41.0.1                   py27_0    defaults
sqlite                    3.28.0               h0c8e037_0    defaults
vc                        9                    h7299396_1    defaults
vs2008_runtime            9.00.30729.1         hfaea7d5_1    defaults
wheel                     0.33.4                   py27_0    defaults
wincertstore              0.2              py27hf04cefb_0    defaults
</code>

Related:

  * conda search -f <package_name> - search packages available in Anaconda

==== reduce disk space usage ====
<cli>
conda clean --dry-run --all
conda clean --all
</cli>

Ref:-
  * https://stackoverflow.com/questions/48706548/how-to-free-disk-space-taken-up-by-anaconda
  * https://github.com/conda/conda/issues/8630 - recommends ''conda clean --all'' to fix ''conda json.decoder.JSONDecodeError: Unterminated string starting at: line xxxxxx column xx (char xxxxxxxx)''

==== where is the package cache ====
<code>
conda info
</code>

tags | conda cache directory

Related commands:
<code>
conda clean --all --dry-run
conda clean --packages --dry-run
</code>

==== where is my condarc file? ====
<code>
$ conda info | grep "user config file"
       user config file : C:\Users\raju\.condarc
</code>

==== documentation links ====
  * conda install - https://docs.conda.io/projects/conda/en/latest/commands/install.html
  * conda update - https://docs.conda.io/projects/conda/en/latest/commands/update.html
  * conda config - https://docs.conda.io/projects/conda/en/latest/commands/config.html
  * pinning packages - https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-pkgs.html#preventing-packages-from-updating-pinning

==== which solver am I using? ====
Use
<code>
conda config --show solver
</code>
or
<code>
conda config --show-sources
</code>


Sample output:
<code>
$ conda config --show solver
solver: classic
</code>

<code>
$ conda config --show solver
solver: libmamba
</code>

<code>
$conda config --show-sources
==> C:\Users\root\.condarc <==
solver: libmamba

==> cmd_line <==
debug: False
json: False
</code>

See also:-
  * https://conda.github.io/conda-libmamba-solver/getting-started/ - tells how to switch the default solver from classic to libmamba.

==== when was a conda environment created? ====
<code>
conda env list | grep -v '^#' | perl -lane 'print $F[-1]' | xargs ls -lrt1d
</code>

For example
<code>
 % conda env list | grep -v '^#' | perl -lane 'print $F[-1]' | xargs ls -lrt1d
drwxr-xr-x 10 rajulocal rajulocal 4096 Apr  4  2020 /opt/rajulocal/miniconda3/envs/rutils
drwxr-xr-x 23 rajulocal rajulocal 4096 Nov  8  2021 /opt/rajulocal/miniconda3/envs/py39
drwxr-xr-x 13 rajulocal rajulocal 4096 Jun  6  2022 /opt/rajulocal/miniconda3/envs/play_ground
drwxr-xr-x 27 rajulocal rajulocal 4096 Feb 28  2023 /opt/rajulocal/miniconda3/envs/py310
drwxr-xr-x 13 rajulocal rajulocal 4096 May 20  2023 /opt/rajulocal/miniconda3/envs/test_yfinance
drwxr-xr-x 18 rajulocal rajulocal 4096 Jun 30 00:45 /opt/rajulocal/miniconda3
drwxr-xr-x 27 rajulocal rajulocal 4096 Jul 11 01:07 /opt/rajulocal/miniconda3/envs/py311
drwxr-xr-x 24 rajulocal rajulocal 4096 Nov 12 04:44 /opt/rajulocal/miniconda3/envs/market_data_processor
</code>

Ref: https://stackoverflow.com/questions/69109316/how-can-i-see-the-creation-dates-for-my-conda-environments

===== activate conda environment =====
tags | activate conda environment through shell script
==== source activate vs. conda activate ====
For conda >= 4.4.0, use "conda activate" instead of "source activate". The former is faster and the command remains same across OSes.

From https://github.com/conda/conda/blob/master/CHANGELOG.md#440-2017-12-20 -> "New Feature Highlights" section

> **conda activate**: The logic and mechanisms underlying environment activation have been reworked. With conda 4.4, ''conda activate'' and ''conda deactivate'' are now the preferred commands for activating and deactivating environments. You'll find they are much more snappy than the ''source activate'' and ''source deactivate'' commands from previous conda versions. The ''conda activate'' command also has advantages of (1) being universal across all OSes, shells, and platforms, and (2) not having path collisions with scripts from other packages like python virtualenv's activate script.

See also: [[#conda activate]], [[#source activate]]
==== conda activate ====
<code>
# As described in https://github.com/conda/conda/issues/8186
# The conda activate command is failing with
# conda.sh: line 55: PS1: unbound variable
# Until the issue is fixed, set +eu before calling "source activate"
# and set -eu afterwards
set +eux
# If the conda.sh is not sourced, then 'conda activate' is giving
# CommandNotFoundError: Your shell has not been properly configured
# to use 'conda activate'.
source "$(dirname $(which conda))/../etc/profile.d/conda.sh"
conda activate $ENV_NAME
set -eux
</code>

tested with conda 4.8.3 on a machine running RHEL 7.6

See also: [[#source activate vs. conda activate]], [[#source activate]]
==== source activate ====

<code>
# As described in https://github.com/conda/conda/issues/8186
# The conda activate command is failing with
# conda.sh: line 55: PS1: unbound variable
# Until the issue is fixed, set +eu before calling "source activate"
# and set -eu afterwards
set +eux
source activate $ENV_NAME
set -eux
</code>

tested with conda 4.8.3 on a machine running RHEL 7.6

See also: [[#source activate vs. conda activate]], [[#conda activate]]

===== Sample environment files =====

==== sample environment file ====
<code>
name: sample_py36
channels:
  - conda-forge
  - defaults
dependencies:
  - python=3.6.8
  - pandas=0.24.1
  - pip:
    - pywinauto
    - fastavro==0.21.18
</code>

==== environment file to install pip package from a local repository ====
One way to specify a wheel file stored in a local directory as a pip dependency in a conda environment file is to do something like
<code>
name: test_ibapi
channels:
  - defaults
dependencies:
  - python=3.11
  - pip
  - pip:
    - file:///home/rajulocal/software/dist/ibapi-10.22.1-py3-none-any.whl
</code>

but this will not be portable when you move to a different machine since the wheel file path (/home/rajulocal/software/dist/ibapi-10.22.1-py3-none-any.whl) is hard coded. A better approach is to run

<code>
 % pip config set global.find-links file:///home/rajulocal/software/dist
</code>
which will modify pip.conf as
<code>
 % cat ~/.config/pip/pip.conf 
[global]
find-links = file:///home/rajulocal/software/dist
</code>
and then specify ibapi as a normal pip dependency.
<code>
name: test_ibapi
channels:
  - defaults
dependencies:
  - python=3.11
  - pip
  - pip:
    - ibapi
</code>
Ref:-
  * https://pip.pypa.io/en/stable/cli/pip_config/ - shows the syntax for the 'pip config' command
  * https://pip.pypa.io/en/stable/topics/configuration/ - shows the locations of default pip configuration files.
  * https://manpages.debian.org/bookworm/python3-pip/pip.1.en.html - where I found the <nowiki>--find-links</nowiki> option

tags | <nowiki>--find-links</nowiki>, pip config local packages, how to specify local repo location for pip packages, conda install pip package from a "local directory"

==== specify package version with git+https ====
<code>
name: test_chessli
channels:
  - defaults
dependencies:
  - python=3.11
  - pandas=1.5.3
  - pip
  - pip:
    - chessli
    - git+https://github.com/lervag/apy.git@v0.6.0#egg=apy
</code>

Ref:-
  * https://stackoverflow.com/questions/47617552/how-can-i-specify-a-custom-git-branch-in-my-pipfile - where I found the answer
  * https://github.com/lervag/apy/tags?after=v0.9.1 - versions of the api package < 0.9.1

tags | pip "git+https" release, conda "git+https" release, conda git depend on a "specific release", specify package version with git+https in conda environment file

===== dummy =====

==== create an environment on the fly ====
Sample commands:
<code>
conda create -n <myenv> python ipython prompt_toolkit=2 jupyter pandas numpy scipy scikit-learn matplotlib

conda create -n <myenv> python=3.9 pandas pandas-datareader
</code>

==== commands I came across ====
<code>
$ conda config --add pinned_packages defaults::conda
</code>
To revert
<code>
$ conda config --remove-key pinned_packages
</code>

==== What is the difference between conda, pip, and virtualenv? ====
  * https://docs.conda.io/projects/conda/en/latest/commands.html#conda-vs-pip-vs-virtualenv-commands -> scroll down to see a table that shows how to accomplish different tasks by using either conda or a combination pip and virtualenv.

==== conda release numbers ====
  * After 4.14.0 (released on Aug 3, 2022), conda changed to CalVer versioning system. The release after 4.14.0 is 22.9.0 (released on Sep 20, 2022).

See also:
  * https://github.com/conda/conda/releases - conda release history
  * https://calver.org/ - CalVer - Calendar Versioning

==== twsapi conda environment file ====
  * Build a wheel file from the zip file.
    * See for example https://github.com/KamarajuKusumanchi/market_data_processor/blob/master/src/scripts/build_twsapi.sh which builds a wheel file from the TWS python api zip file (https://interactivebrokers.github.io/downloads/twsapi_macunix.1022.01.zip)
  * Configure pip to install its wheels from a local wheel directory <code>pip config set global.find-links file:///home/rajulocal/software/dist</code>

  * Install the package
    * See for example https://github.com/KamarajuKusumanchi/rutils/blob/master/python3/envs/env_test_ibapi.yml

tags | conda envrionment install package from zip file, conda install zip package, install zip file