This is an old revision of the document!
Table of Contents
dummy
useful links
- exception handling in python (tutswiki.com)
- very easy to follow; high information density; well thought out examples
- Online python compiler (programiz.com)
- can be useful while doing live coding interviews.
- Facts and myths about Python names and values (nedbatchelder.com) - well explained.
validate inputs
Sample snippet:
day = 'foo' valid_days = ['sun', 'mon', 'tue'] assert day in valid_days, '{} is not a valid day. '\ 'It should be one of {}'.format(day, valid_days)
Sample output:
--------------------------------------------------------------------------- AssertionError Traceback (most recent call last) <ipython-input-4-70d26d6f2c6d> in <module> ----> 1 assert day in valid_days, '{} is not a valid day. '\ 2 'It should be one of {}'.format(day, valid_days) AssertionError: foo is not a valid day. It should be one of ['sun', 'mon', 'tue']
where is a module imported from
tags | module path
one way:
import importlib.util importlib.util.find_spec("foo")
Alternative:
import imp imp.find_module("foo")
Note:
- The imp module is deprecated in favour of importlib. So use the first approach whenever possible.
get the function name
my_function.__name__
Ref:- https://stackoverflow.com/questions/251464/how-to-get-a-function-name-as-a-string
empty file
Task: If input file is empty, write an empty output file
import os if os.stat(input_file).st_size == 0: logger.info('Input file is empty.') logger.info('Writing an empty output file.') with open(output_file, 'w') as fp: pass
config files
- https://docs.python.org/3/library/configparser.html#configparser.ConfigParser.defaults - start from here to see the list of functions available
- use extension .cfg for config files.
- based on: https://docs.python.org/3/library/configparser.html → search for .cfg
- another choice is .ini . But I prefer .cfg over .ini
- more info on INI: https://en.wikipedia.org/wiki/INI_file
python mailing list
python reference
- operator precedence - https://docs.python.org/3/reference/expressions.html#operator-precedence
python tutorials
python release cycle
- https://python-release-cycle.glitch.me/ - useful to check when a Python version was released and when it would be EOL.
isinstance experiments
In [1]: isinstance(True, int) Out[1]: True In [2]: type(True) is int Out[2]: False
Using Python 3.9.5
write an extra line to file
with open('filename.txt', 'w') as fh: fh.write('\n')
Do not use os.linesep as a line terminator when writing files opened in text mode (the default); use a single '\n' instead, on all platforms.
Ref:- https://docs.python.org/3/library/os.html#os.linesep
See also:- https://stackoverflow.com/questions/4025760/python-file-write-creating-extra-carriage-return
find unused variables
pip install pylint pylint module_name.py find . -name "*.py" | xargs pylint
check if something is float
% python Python 3.11.4 (main, Jul 5 2023, 13:45:01) [GCC 11.2.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> isinstance(12.0, float) True >>> isinstance(12.0, int) False >>> isinstance(12, float) False
check if something is int
% python Python 3.11.4 (main, Jul 5 2023, 13:45:01) [GCC 11.2.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> isinstance(12, int) True >>> isinstance(12, float) False >>> isinstance(12.0, int) False
Python comparisons
$ python Python 3.9.17 (main, Jul 5 2023, 20:47:11) [MSC v.1916 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> 4 == True False >>> 1 == True True
Chained comparisons
Q: In Python 3 < 4 == True
evaluates to False.
$ python Python 3.9.17 (main, Jul 5 2023, 20:47:11) [MSC v.1916 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> 3 < 4 == True False
Why?
A: The <
and ==
are comparison operators. If a, b, c, …, y, z are expressions and op1, op2, …, opN are comparison operators, then in python a op1 b op2 c … y opN z is equivalent to a op1 b and b op2 c and … y opN z, except that each expression is evaluated at most once.
So 3 < 4 == True
is equivalent to 3 < 4 and 4 == True
which evaluates to 'True and False which turns out to be
False.
Ref:- https://docs.python.org/3/reference/expressions.html#comparisons
===== tasks =====
* get urls in a url
* find depth
===== difference between =====
==== os.getenv vs. os.environ.get ====
What is the difference between os.getenv vs os.environ.get?
None. I prefer to use os.getenv as it is less number of characters to type.
To experiment, use difference between os.getenv vs. os.environ.get.ipynb (github.com/KamarajuKusumanchi).
Sample usage from https://github.com/dpguthrie/yahooquery/blob/master/yahooquery/base.py#LL926C1-L936C1
<code>
def init(self, **kwargs):
…
username = os.getenv(“YF_USERNAME”) or kwargs.get(“username”)
password = os.getenv(“YF_PASSWORD”) or kwargs.get(“password”)
if username and password:
self.login(username, password)
</code>
===== Snippets =====
=== import from another project ====
Say we have two projects cloned into two directories - github/project1 and github/project2. To import stuff from project1 into a file in project2
<code>
import os
import sys
# Note: os.path.dirname(file) can give an empty string,
# use os.path.dirname(os.path.abspath(file)) instead.
# Ref:- See Dmitry Trofimov's comment in https://stackoverflow.com/a/918178/6305733
dirname = os.path.dirname(os.path.abspath(file))
# Change the relative path as per your needs
newdir = os.path.abspath(os.path.join(dirname, '../project1'))
if newdir not in sys.path:
sys.path.insert(0, newdir)
</code>
tags | import from many levels up, import from another directory, relative directory name, path relative to current file
==== check for minimum python version ====
<code>
import sys
if sys.version_info[:2] < (3, 7):
raise RuntimeError('Requires Python 3.7 or greater')
</code>
==== print exception and continue ====
<code>
from traceback import format_exc
try:
# my code
except Exception as exc:
print 'caught exception while doing foo'
print format_exc()
print exc
</code>
==== print exception to log file ====
<code>
from traceback import format_exc
try:
# my code
except Exception as exc:
logger.error('Caught exception while doing stuff.')
logger.error(format_exc())
logger.error(exc)
</code>
When the code is executed in pycharm and the exception is encountered, the traceback is printed in both pycharm window and in the file.
To test the above code, you can raise a simple exception using
<code>
raise ValueError(“test exception handling.”)
</code>
==== sleep for 10 seconds ====
tags | sleep for N seconds
<code>
from time import sleep
sleep(10)
</code>
Ref:- https://realpython.com/python-sleep/ - well written article that talks about how you might use the sleep function in real life. Worth reading end-to-end once.
===== Reading files =====
==== read file into a list ====
Given
<code>
$ cat -A great.txt
kamaraju$
kamaraj $
kamara $
kamar $
kama $
kam $
ka $
k $
</code>
where the $ denotes the end of line.
<code>
$ du -b great.txt
72 great.txt
$ wc great.txt
8 8 72 great.txt
</code>
To read it
<code>
In [3]:
file = 'great.txt'
with open(file) as fh:
contents = [line.rstrip('\n') for line in fh]
print(contents)
['kamaraju', 'kamaraj ', 'kamara ', 'kamar ', 'kama ', 'kam ', 'ka ', 'k ']
In [4]:
len(contents)
Out[4]:
8
</code>
Ref:- https://stackoverflow.com/questions/15233340/getting-rid-of-n-when-using-readlines
You can also do it using readlines.
<code>
In [5]:
file = 'great.txt'
with open(file) as fh:
lines = fh.readlines()
lines = [line.rstrip('\n') for line in lines]
print(lines)
['kamaraju', 'kamaraj ', 'kamara ', 'kamar ', 'kama ', 'kam ', 'ka ', 'k ']
</code>
tags | file readlines strip newline
Another approach is to use read() with splitlines().
<code>
In [6]:
file = 'great.txt'
with open(file) as fh:
lines = fh.read().splitlines()
print(lines)
['kamaraju', 'kamaraj ', 'kamara ', 'kamar ', 'kama ', 'kam ', 'ka ', 'k ']
</code>
===== programming notes =====
==== which python executable am I running? ====
<code>
import sys
print(sys.executable)
</code>
For example
<code>
$ python
Python 3.11.5 | packaged by Anaconda, Inc. | (main, Sep 11 2023, 13:26:23) [MSC v.1916 64 bit (AMD64)] on win32
Type “help”, “copyright”, “credits” or “license” for more information.
»> import sys
»> print(sys.executable)
C:\Users\kkusuman\AppData\Local\conda\conda\envs\py311\python.exe
</code>
===== programming related =====
==== get the home directory ====
<code>
import os
home = os.path.expanduser(“~”)
</code>
Ref:- https://stackoverflow.com/questions/4028904/what-is-a-cross-platform-way-to-get-the-home-directory
==== write file relative to home directory ====
<code>
import os
home = os.path.expanduser(“~”)
path = os.path.join(home, “x”, “data.txt”)
</code>
==== round function ====
The expression
round(n, r) rounds floating-point expression $n$ to the $10^{-r}$ decimal digit; for example,
round(n, -2) rounds floating-point value $n$ to the hundreds place $(10^2)$. Similarly,
round(n, 3)'' rounds floating-point value n to the thousandths place $(10^{-3})$.
format related
assert variable in a list
$ cat junk51.py letters = ["a", "b", "c"] x = "1" assert x in letters, "{} is not a valid letter. It should be one of {}".format( x, letters ) $ python junk51.py Traceback (most recent call last): File "C:\Users\kkusuman\x\junk51.py", line 3, in <module> assert x in letters, "{} is not a valid letter. It should be one of {}".format( AssertionError: 1 is not a valid letter. It should be one of ['a', 'b', 'c']
stack overflow links I came across
memory consumed by a function
call different functions based on variable
tags | passing function, function as value in dictionary, dispatch
__all__
- https://stackoverflow.com/questions/44834/what-does-all-mean-in-python - good explanation by Alec Thomas
- https://docs.python.org/3/tutorial/modules.html#importing-from-a-package - link to documentation; referenced in the above stackoverflow link.
dummy
- https://superuser.com/questions/1633073/why-are-tar-xz-files-15x-smaller-when-using-pythons-tar-library-compared-to-mac - Why are tar.xz files 15x smaller when using Python's tar library compared to macOS tar?
came across
dummy
- os.path.splitext(path) - Get the extension of a path