Scripting >> Python >> Cheatsheet >> Python 3 modules cheatsheet 1

Jump to : calendar | collections | csv | ctypes | datetime | html.parser | glob | itertools | json | math | pillow | PyYAMLoperator | os | random | re | requests | subprocess | tqdm | unittest | zipfile

Import ModuleName

Important !!! - Do not name your python script the same as the module being imported!!! e.g. if you want to import "calendar" module, do not name you script as calendar.py ... it will import your script instead. 


calendar
References:help('calendar') / https://docs.python.org/3/library/calendar.html

 

Classes:

calendar.Calendar(firstweekday=0)

Creates a Calendar object. firstweekday is an integer specifying the first day of the week. 0 is Monday (the default), 6 is Sunday

calendar.TextCalendar().formatyear(yyyy)

Used to generate plain text calendars

calendar.HTMLCalendar(firstweekday=0)

Used to generate HTML calendars

calendar.LocaleTextCalendar(firstweekday=0, locale=None)

subclass of TextCalendar can be passed a locale name in the constructor and will return month and weekday names in the specified local

calendar.LocaleHTMLCalendar(firstweekday=0, locale=None)

subclass of HTMLCalendar can be passed a locale name in the constructor and will return month and weekday names in the specified locale

 

 

 

 

 

 

Go Top

calendar.Calendar instance Methods:

iterweekdays()

Return an iterator for the week day numbers that will be used for one week

itermonthdates(year, month)

Return an iterator for the month month (1–12) in the year year.

itermonthdays(year, month)

Return an iterator for the month month in the year year similar to itermonthdates(), but not restricted by the datetime.date range. Days returned will simply be day of the month numbers. For the days outside of the specified month, the day number is 0.

itermonthdays2(year, month)

Return an iterator for the month month in the year year similar to itermonthdates(), but not restricted by the datetime.date range. Days returned will be tuples consisting of a day of the month number and a week day number.

itermonthdays3(year, month)

Return an iterator for the month month in the year year similar to itermonthdates(), but not restricted by the datetime.date range. Days returned will be tuples consisting of a year, a month and a day of the month numbers.

itermonthdays4(year, month)

Return an iterator for the month month in the year year similar to itermonthdates(), but not restricted by the datetime.date range. Days returned will be tuples consisting of a year, a month, a day of the month, and a day of the week numbers.

monthdatescalendar(year, month)

Return a list of the weeks in the month month of the year as full weeks. Weeks are lists of seven datetime.date objects.

monthdays2calendar(year, month)

Return a list of the weeks in the month month of the year as full weeks. Weeks are lists of seven tuples of day numbers and weekday numbers.

monthdayscalendar(year, month)

Return a list of the weeks in the month month of the year as full weeks. Weeks are lists of seven day numbers.

yeardatescalendar(year, width=3)

Return the data for the specified year ready for formatting. The return value is a list of month rows. Each month row contains up to width months (defaulting to 3). Each month contains between 4 and 6 weeks and each week contains 1–7 days. Days are datetime.date objects.

yeardays2calendar(year, width=3)

Return the data for the specified year ready for formatting (similar to yeardatescalendar()). Entries in the week lists are tuples of day numbers and weekday numbers. Day numbers outside this month are zero.

yeardayscalendar(year, width=3)

Return the data for the specified year ready for formatting (similar to yeardatescalendar()). Entries in the week lists are day numbers. Day numbers outside this month are zero.

 

 

 

  

collections
References: help('collections') / https://docs.python.org/3/library/collections.html

Examples

This module implements specialized container datatypes providing alternatives to Python's general purpose built-in containers, dict,
list, set, and tuple.

namedtuple  
  • factory function for creating tuple subclasses with named fields

deque       

  • list-like container with fast appends and pops on either end
  • .append(..)
  • .appendleft(..)
  • .clear(..)
  • .copy(..)
  • .count(..)
  • .extend(..)
  • .extendleft(..)
  • .index(..)
  • .insert(..)
  • .pop(..)
  • .popleft(..)
  • .remove(..)
  • reverse(..)
  • .rotate(..)


ChainMap    

  • dict-like class for creating a single view of multiple mappings


Counter     

  • dict subclass for counting hashable objects


OrderedDict 

  • dict subclass that remembers the order entries were added


defaultdict 

  • dict subclass that calls a factory function to supply missing values


UserDict    

  • wrapper around dictionary objects for easier dict subclassing


UserList    

  • wrapper around list objects for easier list subclassing


UserString  

  • wrapper around string objects for easier string subclassing

namedtuple example 1

>>> from collections import namedtuple
>>> Point = namedtuple('Point','x,y')
>>> pt1 = Point(1,2)
>>> pt2 = Point(3,4)
>>> dot_product = ( pt1.x * pt2.x ) +( pt1.y * pt2.y )
>>> print dot_product
11

namedtuple example 2
>>> from collections import namedtuple
>>> Car = namedtuple('Car','Price Mileage Colour Class')
>>> xyz = Car(Price = 100000, Mileage = 30, Colour = 'Cyan', Class = 'Y')
>>> print xyz
Car(Price=100000, Mileage=30, Colour='Cyan', Class='Y')
>>> print xyz.Class
Y

namedtuple example 3
#input line 1 N = no. of students
#input line 2 = column headings that include MARKS
#next N lines, columns of data for each headingfrom collections import namedtuple
N = int(input())
Student = namedtuple('Student',input().split())
print(sum([ int(Student._make(input().split()).MARKS)  for i in range(N) ]) / N)

Counter example

>>> from collections import Counter
>>> shoeList = [1,2,3,4,5,1,2,3,4,5,1,2,3,4,1,2,3]
>>> shoeStock = Counter(shoeList)
>>> print(shoeStock)
Counter({1: 4, 2: 4, 3: 4, 4: 3, 5: 2})

 

defaultdict example

>>> from collections import defaultdict
>>> d = defaultdict(list)
>>> d['python'].append("awesome")
>>> d['something-else'].append("not relevant")
>>> d['python'].append("language")
>>> for i in d.items():
        print i
('python', ['awesome', 'language'])
('something-else', ['not relevant'])

explanation:
when d['python'] was first used, there was no item  in the dictionary with key='python', but because default factory was set as list, it will call list() to assign  empty list to d['python'] and then allow it to call the append method (which is available to list data type] to
add items to that list value.
 

OrderedDict example

from collections import OrderedDict
N = int(input())
inventory = OrderedDict()
for i in range(N) :
   itemlist = input().split()
   item_name = ' '.join(itemlist[:-1])
   price = int(itemlist[-1])
   try:
       inventory[item_name] += price
   except:
       inventory[item_name] = price

for item,price in inventory.items():
    print(item,price)

deque example

 

Go Top

 

csv
References: help('csv') / https://docs.python.org/3/library/csv.html

Functions

csv.reader(csvfile, dialect='excel', **fmtparams)

    Return a reader object which will iterate over lines in the given csvfile. csvfile can be any object which supports the iterator protocol and returns a string each time its __next__() method is called — file objects and list objects are both suitable


csv.writer(csvfile, dialect='excel', **fmtparams)

    Return a writer object responsible for converting the user’s data into delimited strings on the given file-like object. csvfile can be any object with a write() method


class csv.DictReader(f, fieldnames=None, restkey=None, restval=None, dialect='excel', *args, **kwds)

    Create an object that operates like a regular reader but maps the information in each row to a dict whose keys are given by the optional fieldnames parameter


class csv.DictWriter(f, fieldnames, restval='', extrasaction='raise', dialect='excel', *args, **kwds)

    Create an object which operates like a regular writer but maps dictionaries onto output rows

 

 


 

Example - csv.reader

>>> import csv
>>> with open('eggs.csv', newline='') as csvfile:
...     spamreader = csv.reader(csvfile, delimiter=' ', quotechar='|')
...     for row in spamreader:
...         print(', '.join(row))
Spam, Spam, Spam, Spam, Spam, Baked Beans
Spam, Lovely Spam, Wonderful Spam

Example - csv.writer

import csv
with open('eggs.csv', 'w', newline='') as csvfile:
    spamwriter = csv.writer(csvfile, delimiter=' ',
    quotechar='|', quoting=csv.QUOTE_MINIMAL)
    spamwriter.writerow(['Spam'] * 5 + ['Baked Beans'])
    spamwriter.writerow(['Spam', 'Lovely Spam', 'Wonderful Spam'])

Example - csv.DictReader

>>> import csv
>>> with open('names.csv', newline='') as csvfile:
...     reader = csv.DictReader(csvfile)
...     for row in reader:
...         print(row['first_name'], row['last_name'])
...
Eric Idle
John Cleese

>>> print(row)
{'first_name': 'John', 'last_name': 'Cleese'}

Example - csv.DictWriter

import csv

with open('names.csv', 'w', newline='') as csvfile:
    fieldnames = ['first_name', 'last_name']
    writer = csv.DictWriter(csvfile, fieldnames=fieldnames)

    writer.writeheader()
    writer.writerow({'first_name': 'Baked', 'last_name': 'Beans'})
    writer.writerow({'first_name': 'Lovely', 'last_name': 'Spam'})

    writer.writerow({'first_name': 'Wonderful', 'last_name': 'Spam'})

 

Writer Objects  
   
   

 

 

ctypes
References: help('csv') / https://docs.python.org/3/library/ctypes.html

   
(FOR WINDOWS environment)

ctypes.windll.shell32.CommandLineToArgvW   
ctypes.windll.shell32.IsUserAnAdmin   
ctypes.windll.shell32.SetCurrentProcessExplicitAppUserModelID   
ctypes.windll.shell32.SHGetFolderPathW   
ctypes.windll.shell32.ShellExecuteW
ctypes.windll.Shell32.DllGetVersion   
ctypes.windll.shell32.SHFileOperationW   
ctypes.windll.shell32.ShellExecuteExW   

see https://programtalk.com/python-examples/ctypes.windll.shell32./
 
 

 

 

datetime
References: help('datetime') or https://docs.python.org/3/library/datetime.html

 

Classes

class datetime.date

class datetime.time

class datetime.datetime

class datetime.timedelta

class datetime.tzinfo

class datetime.timezone

 

 

 

 

 

Go Top

Class date  

Methods:

date(year,month,day)
- create an object of type date with the specified day,month,year

today()
- creates an object of type date with day,month,year taken from current date.

fromtimestamp(timestamp)
- creates an object of type date from the timestamp value

Attributes:

year
returns year from the date object

month
returns the month from the date object
   
day
returns the day from the date object

Instance methods:

replace(year,month,day)
update any or all of the 3 attributes e.g. d=d.replace(month=3,day=1)

timetuple()
returns a tuple of the attributes for the current local time
         

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Go Top

Class date  
Class datetime Exampes

Methods

datetime(year, month, day[, hour[, minute[, second[, microsecond[,tzinfo]]]]])

astimezone()

ctime()

date()

dst()

isoformat()

replace()

time()

timestamp()

timetuple()

timetz()

tzname()

utcoffset()

utctimetuple()

combine()

fromisoformat()

fromtimestamp()

now(tz=None)

strptime(string, format)

utcfromtimestamp()

utcnow()

Class Attributes

.min

 The earliest representable datetime, datetime(MINYEAR, 1, 1, tzinfo=None).

.max

 The latest representable datetime, datetime(MAXYEAR, 12, 31, 23, 59, 59, 999999, tzinfo=None).

.resolution

 The smallest possible difference between non-equal datetime objects, timedelta(microseconds=1).

Instance attributes (read-only):
 

.year

 Between MINYEAR and MAXYEAR inclusive.

.month

 Between 1 and 12 inclusive.

.day

 Between 1 and the number of days in the given month of the given year.

.hour

 In range(24).

.minute

 In range(60).

.second

 In range(60).

.microsecond

 In range(1000000).

.tzinfo

The object passed as the tzinfo argument to the datetime constructor, or None if none was passed.

.fold

In [0, 1]. Used to disambiguate wall times during a repeated interval.

 

 

Class timedelta

 

Class tzinfo

 

 
Class timezone  

format code for datetime module

Directive

Meaning Example
%a Abbreviated weekday name. Sun, Mon, ...
%A Full weekday name. Sunday, Monday, ...
%w Weekday as a decimal number. 0, 1, ..., 6
%d Day of the month as a zero-padded decimal. 01, 02, ..., 31
%-d Day of the month as a decimal number. 1, 2, ..., 30
%b Abbreviated month name. Jan, Feb, ..., Dec
%B Full month name. January, February, ...
%m Month as a zero-padded decimal number. 01, 02, ..., 12
%-m Month as a decimal number. 1, 2, ..., 12
%y Year without century as a zero-padded decimal number. 00, 01, ..., 99
%-y Year without century as a decimal number. 0, 1, ..., 99
%Y Year with century as a decimal number. 2013, 2019 etc.
%H Hour (24-hour clock) as a zero-padded decimal number. 00, 01, ..., 23
%-H Hour (24-hour clock) as a decimal number. 0, 1, ..., 23
%I Hour (12-hour clock) as a zero-padded decimal number. 01, 02, ..., 12
%-I Hour (12-hour clock) as a decimal number. 1, 2, ... 12
%p Locale’s AM or PM. AM, PM
%M Minute as a zero-padded decimal number. 00, 01, ..., 59
%-M Minute as a decimal number. 0, 1, ..., 59
%S Second as a zero-padded decimal number. 00, 01, ..., 59
%-S Second as a decimal number. 0, 1, ..., 59
%f Microsecond as a decimal number, zero-padded on the left. 000000 - 999999
%z UTC offset in the form +HHMM or -HHMM.  
%Z Time zone name.  
%j Day of the year as a zero-padded decimal number. 001, 002, ..., 366
%-j Day of the year as a decimal number. 1, 2, ..., 366
%U Week number of the year (Sunday as the first day of the week). All days in a new year preceding the first Sunday are considered to be in week 0. 00, 01, ..., 53
%W Week number of the year (Monday as the first day of the week). All days in a new year preceding the first Monday are considered to be in week 0. 00, 01, ..., 53
%c Locale’s appropriate date and time representation. Mon Sep 30 07:06:05 2013
%x Locale’s appropriate date representation. 09/30/13
%X Locale’s appropriate time representation. 07:06:05
%% A literal '%' character. %

 

Go Top

 

 


   

 

html.parser
References:help('html.parser') / https://docs.python.org/3/library/html.parser.html

 
Methods Examples

HTMLParser.feed(data)

HTMLParser.close()

HTMLParser.reset()

HTMLParser.getpos()

HTMLParser.get_starttag_text()

HTMLParser.handle_starttag(tag, attrs)

HTMLParser.handle_endtag(tag)

HTMLParser.handle_startendtag(tag, attrs)

HTMLParser.handle_data(data)

HTMLParser.handle_entityref(name)

HTMLParser.handle_charref(name)

HTMLParser.handle_comment(data)

HTMLParser.handle_decl(decl)

HTMLParser.handle_pi(data)

HTMLParser.unknown_decl(data)


 

from html.parser import HTMLParser
from html.entities import name2codepoint

class MyHTMLParser(HTMLParser):
    def handle_starttag(self, tag, attrs):
        print("Start tag:", tag)
        for attr in attrs:
            print("     attr:", attr)

    def handle_endtag(self, tag):
        print("End tag  :", tag)

    def handle_data(self, data):
        print("Data     :", data)

    def handle_comment(self, data):
        print("Comment  :", data)

    def handle_entityref(self, name):
        c = chr(name2codepoint[name])
        print("Named ent:", c)

    def handle_charref(self, name):
        if name.startswith('x'):
            c = chr(int(name[1:], 16))
        else:
            c = chr(int(name))
        print("Num ent  :", c)

    def handle_decl(self, data):
        print("Decl     :", data)

parser = MyHTMLParser()

 

Go Top

   

 

glob
References: help('glob') / https://docs.python.org/3/library/glob.html
 
Functions  
glob.glob (pathname, *, recursive=False)

Return a possibly-empty list of path names that match pathname, which must be a string containing a path specification. pathname can be either absolute (like /usr/src/Python-1.5/Makefile) or relative (like ../../Tools/*/*.gif), and can contain shell-style wildcards. Broken symlinks are included in the results (as in the shell).

If recursive is true, the pattern “**” will match any files and zero or more directories and subdirectories. If the pattern is followed by an os.sep, only directories and subdirectories match.

Note

Using the “**” pattern in large directory trees may consume an inordinate amount of time.

Changed in version 3.5: Support for recursive globs using “**”.

glob.iglob (pathname, *, recursive=False)

Return an iterator which yields the same values as glob() without actually storing them all simultaneously.

glob.escape (pathname)

Escape all special characters ('?', '*' and '['). This is useful if you want to match an arbitrary literal string that may have special characters in it. Special characters in drive/UNC sharepoints are not escaped, e.g. on Windows escape('//?/c:/Quo vadis?.txt') returns '//?/c:/Quo vadis[?].txt'.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Go Top

 

itertools
References: help('itertools') / https://docs.python.org/3/library/itertools.html

 
Functions Examples

count(start [, step])

Return a count object whose .__next__() method returns consecutive values.

cycle(p)

Return elements from the iterable until it is exhausted. Then repeat the sequence indefinitely.

repeat(elem [,n])

.

accumulate(p[,func])

.

chain(p,q, ....)

.

compress(data, selectors)

.

dropwhile(pred,seq)

.

filterfalse(pred,seq)

.

groupby(iterable[,key])

.

islice(seq, [start,] stop [,step])

.

starmap(func,seq)

.

takewhile(pred,seq)

.

tee(iterator,n)

.

zip_longest(p, q, ..)

.

product(p,q ....[,repeat=1])

.

permutations(p[, r])

.

combinations(p, r)

.

combinations_with_replacement(p,r)

Example(s) for itertools.accumulate

>>> for i in itertools.accumulate([1,2,3,4]):
...    print(i)
...
1
3
6
10

Example for itertools.cycle

>>> for i in itertools.cycle('ABC'):
>>>     print(i)
A
B
C
A
....

Example for itertools.groupby

eg. count the number of each sequence of characters in a string

>>> from itertools import groupby
>>> for key,grp in groupby(input()):
...    print("({0}, {1})".format(key,len(list(grp))))
...
aabbbccccdee
(a, 2)
(b, 3)
(c, 4)
(d, 1)
(e, 2)
>>>

 

Example for itertools.starmap

eg. for each pair of arguments in the iterable, apply the power function (which takes 2 args)

>>> list(itertools.starmap(pow,[(2,5),(3,3)]))
[32, 27]

Example(s) for itertools.product

- enter K rows of List of Integers of variable length
- find all posible combination of K integers taking 1 item each from each list

import itertools
K = int(input())
LoL = []
for i in range(K):   
   Li = list(map(int,input().split()))   
   LoL.append(Li) print(LoL)
for C in itertools.product(*LoL):  
   print("combination: {0}".format(C))

 

Sample input

5 4
7 8 9
5 7 8 9 10

Sample output

combination: (5, 7, 5)
combination: (5, 7, 7)
combination: (5, 7, 8)
combination: (5, 7, 9)
combination: (5, 7, 10)
combination: (5, 8, 5)
combination: (5, 8, 7)
combination: (5, 8, 8)
combination: (5, 8, 9)
combination: (5, 8, 10)
combination: (5, 9, 5)
combination: (5, 9, 7)
combination: (5, 9, 8)
combination: (5, 9, 9)
combination: (5, 9, 10)
combination: (4, 7, 5)
combination: (4, 7, 7)
combination: (4, 7, 8)
combination: (4, 7, 9)
combination: (4, 7, 10)
combination: (4, 8, 5)
combination: (4, 8, 7)
combination: (4, 8, 8)
combination: (4, 8, 9)
combination: (4, 8, 10)
combination: (4, 9, 5)
combination: (4, 9, 7)
combination: (4, 9, 8)
combination: (4, 9, 9)
combination: (4, 9, 10)

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Go Top

 

 

json
Reference: help('math') / https://docs.python.org/3/library/json.html#module-json 

 
   

 

 

math
Reference: help('math') / https://docs.python.org/3/library/math.html#module-math

 
Methods Examples

acos(x, /)

acosh(x, /)

asin(x, /)

asinh(x, /)

atan(x, /)

atan2(y, x, /)

atanh(x, /)

ceil(x, /)

copysign(x, y, /)

cos(x, /)

cosh(x, /)

degrees(x, /)
        Convert angle x from radians to degrees.

    erf(x, /)
        Error function at x.

    erfc(x, /)
        Complementary error function at x.

    exp(x, /)
        Return e raised to the power of x.

    expm1(x, /)
        Return exp(x)-1.

    fabs(x, /)
        Return the absolute value of the float x.

    factorial(x, /)
        Find x!.

        Raise a ValueError if x is negative or non-integral.

    floor(x, /)
        Return the floor of x as an Integral.

        This is the largest integer <= x.

    fmod(x, y, /)
        Return fmod(x, y), according to platform C.

        x % y may differ.

    frexp(x, /)
        Return the mantissa and exponent of x, as pair (m, e).

        m is a float and e is an int, such that x = m * 2.**e.
        If x is 0, m and e are both 0.  Else 0.5 <= abs(m) < 1.0.

    fsum(seq, /)
        Return an accurate floating point sum of values in the iterable seq.

        Assumes IEEE-754 floating point arithmetic.

    gamma(x, /)
        Gamma function at x.

    gcd(x, y, /)
        greatest common divisor of x and y

    hypot(x, y, /)
        Return the Euclidean distance, sqrt(x*x + y*y).

    isclose(a, b, *, rel_tol=1e-09, abs_tol=0.0)
        Determine whether two floating point numbers are close in value.

          rel_tol
            maximum difference for being considered "close", relative to the
            magnitude of the input values
          abs_tol
            maximum difference for being considered "close", regardless of the
            magnitude of the input values

        Return True if a is close in value to b, and False otherwise.

        For the values to be considered close, the difference between them
        must be smaller than at least one of the tolerances.

        -inf, inf and NaN behave similarly to the IEEE 754 Standard.  That
        is, NaN is not close to anything, even itself.  inf and -inf are
        only close to themselves.

    isfinite(x, /)
        Return True if x is neither an infinity nor a NaN, and False otherwise.

    isinf(x, /)
        Return True if x is a positive or negative infinity, and False otherwise.

    isnan(x, /)
        Return True if x is a NaN (not a number), and False otherwise.

    ldexp(x, i, /)
        Return x * (2**i).

        This is essentially the inverse of frexp().

    lgamma(x, /)
        Natural logarithm of absolute value of Gamma function at x.

    log(...)
        log(x, [base=math.e])
        Return the logarithm of x to the given base.

        If the base not specified, returns the natural logarithm (base e) of x.

    log10(x, /)
        Return the base 10 logarithm of x.

    log1p(x, /)
        Return the natural logarithm of 1+x (base e).

        The result is computed in a way which is accurate for x near zero.

    log2(x, /)
        Return the base 2 logarithm of x.

    modf(x, /)
        Return the fractional and integer parts of x.

        Both results carry the sign of x and are floats.

    pow(x, y, /)
        Return x**y (x to the power of y).

    radians(x, /)
        Convert angle x from degrees to radians.

    remainder(x, y, /)
        Difference between x and the closest integer multiple of y.

        Return x - n*y where n*y is the closest integer multiple of y.
        In the case where x is exactly halfway between two multiples of
        y, the nearest even value of n is used. The result is always exact.

    sin(x, /)
        Return the sine of x (measured in radians).

    sinh(x, /)
        Return the hyperbolic sine of x.

    sqrt(x, /)
        Return the square root of x.

    tan(x, /)
        Return the tangent of x (measured in radians).

    tanh(x, /)
        Return the hyperbolic tangent of x.

    trunc(x, /)
        Truncates the Real x to the nearest Integral toward 0.


 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Go Top

 

 

pillow
References:help(pillow) / https://pypi.org/project/Pillow/
 A fork of the Python Image Library (PIL) for python3
   

 

 

PyYAML
References:help(yaml) / https://pypi.org/project/PyYAML
Install : pip3 install PyYAML

PyYAML is a YAML parser and emitter for Python
   

 

 

operator
References:help('operator') / https://docs.python.org/3/library/operator.html

 
Methods Examples
operator.lt(a, b)

operator.le(a, b)

operator.eq(a, b)

operator.ne(a, b)

operator.ge(a, b)

operator.gt(a, b)

operator.not_(obj)


 

 

 

Go Top 

 

 


 

os
References:help('os') / https://docs.python.org/3/library/os.html

 
Methods Examples
Some of the commonly used methods :-

 

os.chdir(dirpath)
# change directory

 

os.environ()
# use OS environment variables

 

os.getcwd()
# get current directory

 

os.listdir(path)
# lists contents of directory.  if no path specified, lists current working dir

 

os.mkdir(dirname)
# create directory

 

os.path.abspath(path)
# returns the absolute path of the given path

os.path.basename(path)
# returns the basename i.e. filename part of the given path

os.path.dirname(path)
# returns the dirname i.e. directory part of the given path

os.path.exists(path)
# check if file/directory exists

os.path.join(directory,filename)
# joins the directory and file with the appropriate dir separator for the OS

 

os.remove(filename)
# delete file

 

 

os.rename(current_filename,new_filename)
# rename file

 

os.rmdir(dirname)
# remove the directory

 

os.system(command)
# run Operating System command

 



Eg. - os.listdir :  list files and then rename to specific format

find files that match "p (n).jpg" and rename to "p00n.jpg"

import os
import re
for f in os.listdir():
   m = re.match(r'(p) [(]([0-9]+)[)](.jpg)',f)
   if m is not None:
      nf = "{}{:03d}{}".format(m.group(1),int(m.group(2)),m.group(3))
      os.rename(f,nf)

 

Eg. - os.environ : Append a path to existing PATH env variable

>>> import os
>>> import subprocess
>>> my_env = os.environ.copy()
>>> print(my_env["PATH"])
/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/var/scripts:/home/centos/.local/bin:/home/centos/bin
>>> my_env["PATH"] = os.pathsep.join(["/opt/myapp", my_env["PATH"]])
>>> print(my_env["PATH"])
/opt/myapp:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/var/scripts:/home/centos/.local/bin:/home/centos/bin

 

Go Top


 

   

 

random
References:help('random') / https://docs.python.org/3/library/random.html

 
Methods Exampes

random.randint(fromInteger,toInteger)
    Generate random integer N where fromInteger <= N <= toInteger

 
random.choice(seq)
    Return a random element from the non-empty sequence seq. If seq is empty, raises IndexError

 

 

Go Top

 

re (Regular Expressions)

References:

special characters:

 .              Matches any character except a newline.
 ^              Matches the start of the string.
 $              Matches the end of the string or just before the newline at
                the end of the string.
 *              Matches 0 or more (greedy) repetitions of the preceding RE.
 *?             Matches 0 or more (non-greedy) repetitions of the preceding RE.
 +              Matches 1 or more (greedy) repetitions of the preceding RE.
 +?             Matches 1 or more (non-greedy) repetitions of the preceding RE.
 ?              Matches 0 or 1 (greedy) of the preceding RE.
 ??             Matches 0 or 1 (non-greedy) of the preceding RE.
 {m,n}          Matches from m to n repetitions (greedy) of the preceding RE.
 {m,n}?         Matches from m to n repetitions (non-greedy) of the preceding RE.

                Greedy means that it will match as many repetitions as possible.
                Non-Greedy means that it will match as few repetitions as possible.

 \              Either escapes special characters or signals a special sequence.
 []             Indicates a set of characters.  A "^" as the first character
                after "[" indicates a complementing set.
 |              A|B, creates an RE that will match either A or B.
 ()             Matches the RE inside the parentheses.
                The contents can be retrieved or matched later in the string.
 (?aiLmsux)     Set the A, I, L, M, S, U, or X flag for the RE (see below).
 (?:...)        Non-grouping/capturing version of regular parentheses.

                e.g. (?:apple|orange|banana) will match either apple or orange or banana but will not capture
                the string into the capture group.

  (?P<name>...)  The substring matched by the group is accessible by name.
 (?P=name)      Matches the text matched earlier by the group named name.
 (?#...)        A comment; ignored.

                   Lookarounds - lookahead and lookbehind
(?=...)            Matches if ... matches next, but doesn't consume the string. (Positive lookahead)
                   >>> re.findall(r'(1)(?=2)','123456')     
                   ['1']
                   >>> # finds 3 digit sequence where 2nd digit IS 2
                   >>> re.findall(r'(\d(?=2)\d\d)','121456')
                   ['121']

(?!...)            Matches if ... doesn't match next. (Negative lookahead)
                  
>>> # find 3 digit sequence where 2nd digit is NOT 2
                         >>> re.findall(r'(1)(?!2)','123456')
                   []

(?<=...)
           Matches if preceded by ... (must be fixed length). (Positive lookbehind)

(?<!...)           Matches if not preceded by ... (must be fixed length). (Negative lookbehind)

(?(id/name)yes|no) Matches yes pattern if the group with id/name matched,
                   the (optional) no pattern otherwise.

Go Top

Special sequences:

\A       Matches only at the start of the string.
\Z       Matches only at the end of the string.
\b       Matches the empty string, but only at the start or end of a word.
\B       Matches the empty string, but not at the start or end of a word.
\d       Matches any decimal digit; equivalent to the set [0-9] in
         bytes patterns or string patterns with the ASCII flag.
         In string patterns without the ASCII flag, it will match the whole
         range of Unicode digits.
\D       Matches any non-digit character; equivalent to [^\d].
\s       Matches any whitespace character; equivalent to [ \t\n\r\f\v] in
         bytes patterns or string patterns with the ASCII flag.
         In string patterns without the ASCII flag, it will match the whole
         range of Unicode whitespace characters.
\S       Matches any non-whitespace character; equivalent to [^\s].
\w       Matches any alphanumeric character; equivalent to [a-zA-Z0-9_]
         in bytes patterns or string patterns with the ASCII flag.
         In string patterns without the ASCII flag, it will match the
         range of Unicode alphanumeric characters (letters plus digits
         plus underscore).
         With LOCALE, it will match the set [0-9_] plus characters defined
         as letters for the current locale.
\W       Matches the complement of \w.
\        Matches a literal backslash.

 

Go Top

FUNCTIONS

re.compile(pattern, flags=0)

Compile a regular expression pattern string, returning a Pattern object.

example

import re
regexes = [ re.compile(p) for p in ['[a-zA-Z]+','[0-9]+','[!@#$%&-+]+'] ]
text = "This contain alphabets and numbers 123 but no symbols"
for regex in regexes:
   print("search for  {0:10s}\t{1}".format(regex.pattern,text))
   if regex.search(text):
       print("=> Match") 
   else:
       print("=> No match")

sample run

search for  [a-zA-Z]+   This contain alphabets and numbers 123 but no symbols
=> Match
search for  [0-9]+      This contain alphabets and numbers 123 but no symbols
=> Match
search for  [!@#$%&-+]+ This contain alphabets and numbers 123 but no symbols
=> No match

 

re.escape(pattern)

Escape special characters in a string.

example

>>> import re
>>> print(re.escape('http://www.python.org'))
http://www\.python\.org


re.findall(pattern, string, flags=0)

Return a list of all non-overlapping matches in the string.  If one or more capturing groups are present in the pattern, return a list of groups; this will be a list of tuples if the pattern has more than one group.  Empty matches are included in the result.

re.findall examples - extract repeating character or number sequences

>>> import re
>>> print([x[0] for x in re.findall(r'((.)*)', 'abbcccddddcccbba')])
['a', 'bb', 'ccc', 'dddd', 'ccc', 'bb', 'a']

>>> print([x[0] for x in re.findall(r'((\d)*)', '1233-4444-5543-2121')])
['1', '2', '33', '4444', '55', '4', '3', '2', '1', '2', '1']

>>> print(re.findall(r'((\d)*)', '1233-4444-5543-2121'))
[('1', '1'), ('2', '2'), ('33', '3'), ('4444', '4'), ('55', '5'), ('4', '4'), ('3', '3'), ('2', '2'), ('1', '1'), ('2', '2'), ('1', '1')]

re.finditer(pattern, string, flags=0)

Return an iterator over all non-overlapping matches in the string.  For each match, the iterator returns a Match object.  Empty matches are included in the result.

example

>>> text = "He was carefully disguised but captured quickly by police."
>>> for m in re.finditer(r"\w+ly", text):
...     print('%02d-%02d: %s' % (m.start(), m.end(), m.group(0)))
07-16: carefully
40-47: quickly


 

re.fullmatch(pattern, string, flags=0)

Try to apply the pattern to all of the string, returning a Match object, or None if no match was found.

example

>>> S="This is my ip address 123.123.123.123"
>>> T="This is my ip address 123.123.123.123 and other stuff"
>>> if re.fullmatch('This.*123',S) is not None:
...    print("Matched")
... else:
...    print("Not matched")
...
Matched
>>> if re.fullmatch('This.*123',T) is not None:
...    print("Matched")
... else:
...    print("Not matched")
...
Not matched

re.match(pattern, string, flags=0)

Try to apply the pattern at the start of the string, returning a Match object, or None if no match was found.

example 1

>>> S="This is my ip address 123.123.123.123"
>>> T="This is my ip address 123.123.123.123 and other stuff"
>>> if re.match("This.*123",S) is not None:
...    print("Matched")
... else:
...    print("Not matched")
...
Matched
>>> if re.match('This.*123',T) is not None:
...    print("Matched")
... else:
...    print("Not matched")
...
Matched

Note: both strings matched, because re.match checks for the pattern only at the start of the string

re.purge()

Clear the regular expression caches

 

re.search(pattern, string, flags=0)

Scan through string looking for a match to the pattern, returning a Match object, or None if no match was found.

example 1

>>> import re
>>> m = re.search(r'(\w+)@(\w+)\.(\w+)','username@example.com')
>>> m.groups()
('username', 'example', 'com')

example 2

>>> import re
>>> m = re.search(r'(?P<user>\w+)@(?P<website>\w+)\.(?P<extension>\w+)','myname@example.com')
>>> m.groupdict()
{'website': 'example', 'user': 'myname', 'extension': 'com'}

example 3

>>> import re 
>>> m = re.search(r'(\w+)@(\w+)\.(\w+)','username@example.com')
>>> m.group(0) # The entire match
'username@example.com'
>>> m.group(1) # The first parenthesized subgroup.
'username'
>>> m.group(2) # The second parenthesized subgroup.
'example'
>>> m.group(3) # The third parenthesized subgroup.
'com'
>>> m.group(1,2,3) # Multiple arguments give us a tuple.
('username', 'example', 'com')

re.split(pattern, string, maxsplit=0, flags=0)

Split the source string by the occurrences of the pattern, returning a list containing the resulting substrings.  If capturing parentheses are used in pattern, then the text of all groups in the pattern are also returned as part of the resulting list.  If maxsplit is nonzero, at most maxsplit splits occur, and the remainder of the string is returned as the final element of the list.

examples
- following example will split by ',' or '.'

import re
print("\n".join(re.split('[.,]',input())))


or

import re
regex_pattern = r"[,.]"   
print("\n".join(re.split(regex_pattern, input())))

 

re.sub(pattern, repl, string, count=0, flags=0)

Return the string obtained by replacing the leftmost non-overlapping occurrences of the pattern in string by the replacement repl.  repl can be either a string
or a callable; if a string, backslash escapes in it are processed.  If it is a callable, it's passed the Match object and must return a replacement string to be used.

example(s)

>>> import re
>>> S='1 two 3'
>>> R = re.sub(' two ','2',S)
>>> R
'123'

>>> import re
>>> S=' SeP 2018 (ABC).docx'
>>> R = re.sub('^\s+[Ss][Ee][Pp]\s+\d+\s','',S)
>>> R
'(ABC).docx'

# replace all commas (,) within double quotes
>>> import re
>>> s = 'abc,def,"ghi,jkl,mno",pqr,stu'
>>> re.sub(r'"[^"]+"', lambda x: x.group().replace(',', '|'), s)
'abc,def,"ghi|jkl|mno",pqr,stu'

re.subn(pattern, repl, string, count=0, flags=0)

Return a 2-tuple containing (new_string, number).  new_string is the string obtained by replacing the leftmost non-overlapping occurrences of the pattern in the source string by the replacement repl.  number is the number of substitutions that were made. repl can be either a string or a callable; if a string, backslash escapes in it are processed.  If it is a callable, it's passed the Match object and must return a replacement string to be used.

example

>>> import re
>>> S="yearly, monthly, weekly, daily"
>>> T=re.subn("ly","LY",S)
>>> T
('yearLY, monthLY, weekLY, daiLY', 4)

re.template(pattern, flags=0)

Compile a template pattern, returning a Pattern object

 

 

 

Go Top

FLAGS
re.A|re.ASCII
Make \w, \W, \b, \B, \d, \D, \s and \S perform ASCII-only matching instead of full Unicode matching. This is only meaningful for Unicode patterns, and is ignored for byte patterns. Corresponds to the inline flag (?a)

re.I|re.IGNORECASE
Perform case-insensitive matching; expressions like [A-Z] will also match lowercase letters. Full Unicode matching (such as Ü matching ü) also works unless the re.ASCII flag is used to disable non-ASCII matches. The current locale does not change the effect of this flag unless the re.LOCALE flag is also used. Corresponds to the inline flag (?i)

re.L|re.LOCALE
Make \w, \W, \b, \B and case-insensitive matching dependent on the current locale. This flag can be used only with bytes patterns. The use of this flag is discouraged as the locale mechanism is very unreliable, it only handles one “culture” at a time, and it only works with 8-bit locales. Unicode matching is already enabled by default in Python 3 for Unicode (str) patterns, and it is able to handle different locales/languages. Corresponds to the inline flag (?L)

re.M|re.MULTILINE
When specified, the pattern character '^' matches at the beginning of the string and at the beginning of each line (immediately following each newline); and the pattern character '$' matches at the end of the string and at the end of each line (immediately preceding each newline). By default, '^' matches only at the beginning of the string, and '$' only at the end of the string and immediately before the newline (if any) at the end of the string. Corresponds to the inline flag (?m)

re.S|re.DOTALL
Make the '.' special character match any character at all, including a newline; without this flag, '.' will match anything except a newline. Corresponds to the inline flag (?s)

re.X|re.VERBOSE
This flag allows you to write regular expressions that look nicer and are more readable by allowing you to visually separate logical sections of the pattern and add comments. Whitespace within the pattern is ignored, except when in a character class, or when preceded by an unescaped backslash, or within tokens like *?, (?: or (?P<...>. When a line contains a # that is not in a character class and is not preceded by an unescaped backslash, all characters from the leftmost such # through the end of the line are ignored






## e.g. use re.IGNORECASE to enable case Insensitive search
>>> import re
>>> string="PYTHon"
>>> m=re.search(r"thon",string,re.IGNORECASE)
>>> m
<re.Match object; span=(2, 6), match='THon'>

 

 

requests
References: help(request) / https://pypi.org/project/requests/
Module for sending HTTP requests
   

 

 

subprocess
References:help('subprocess) / https://docs.python.org/3/library/subprocess.html

 
Methods Examples

subprocess.run(args, *, stdin=None, input=None, stdout=None, stderr=None, capture_output=False, shell=False, cwd=None, timeout=None, check=False, encoding=None, errors=None, text=None, env=None, universal_newlines=None, **other_popen_kwargs)

The function (method) returns an instance of the subprocess.completedProcess class.

 

## Eg. - subprocess.run with capture_output set to True

>>> import subprocess
>>> subprocess.run(["host","8.8.8.8"],capture_output=True)
CompletedProcess(args=['host', '8.8.8.8'], returncode=0, stdout=b'8.8.8.8.in-addr.arpa domain name pointer dns.google.\n', stderr=b'')
>>> print(result.returncode)
0
>>> print(result.stdout)
b'8.8.8.8.in-addr.arpa domain name pointer dns.google.\n'
>>> print(result.stdout.decode().split())
['8.8.8.8.in-addr.arpa', 'domain', 'name', 'pointer', 'dns.google.']


## Eg. - subprocess.run with Environment variable

>>> import os,subprocess
>>> my_env = os.environ.copy()
... # can edit or change my_env here

>>> subprocess.run(["env"],env=my_env)

Classes  

subprocess.completedProcess

args

The arguments used to launch the process. This may be a list or a string.

returncode

Exit status of the child process. Typically, an exit status of 0 indicates that it ran successfully.
A negative value -N indicates that the child was terminated by signal N (POSIX only).

stdout

Captured stdout from the child process. A bytes sequence, or a string if run() was called with an encoding, errors, or text=True. None if stdout was not captured.

If you ran the process with stderr=subprocess.STDOUT, stdout and stderr will be combined in this attribute, and stderr will be None.

stderr

Captured stderr from the child process. A bytes sequence, or a string if run() was called with an encoding, errors, or text=True. None if stderr was not captured.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Go Top

 

 

tqdm
References:help(yaml) / https://pypi.org/project/tqdm
Install : pip3 install tqdm

tqdm is a module for showing progress meter for your loops
   

 

 

unittest
References:help('unittest') / https://docs.python.org/3/library/unittest.html

 
Methods Examples

setUp()

    Method called to prepare the test fixture.


tearDown()

    Method called immediately after the test method has been called and the result recorded.


setUpClass()

    A class method called before tests in an individual class are run.


tearDownClass()

    A class method called after tests in an individual class have run.


run(result=None)

    Run the test, collecting the result into the TestResult object passed as result.



skipTest(reason)

    Calling this during a test method or setUp() skips the current test.


debug()

    Run the test without collecting the result. This allows exceptions raised by the test to be propagated to the caller, and can be used to support running tests under a debugger.
import unittest

class TestStringMethods(unittest.TestCase):

    def test_upper(self):
        self.assertEqual('foo'.upper(), 'FOO')

    def test_isupper(self):
        self.assertTrue('FOO'.isupper())
        self.assertFalse('Foo'.isupper())

    def test_split(self):
        s = 'hello world'
        self.assertEqual(s.split(), ['hello', 'world'])
        # check that s.split fails when the separator is not a string
        with self.assertRaises(TypeError):
            s.split(2)

if __name__ == '__main__':
    unittest.main()
Classes  

unittest.TestCase

base class that you can use to create subclasses for more specific tests

assert methods in this class:

 

Method

Checks that

assertEqual(a, b)

a == b

assertNotEqual(a, b)

a != b

assertTrue(x)

bool(x) is True

assertFalse(x)

bool(x) is False

assertIs(a, b)

a is b

assertIsNot(a, b)

a is not b

assertIsNone(x)

x is None

assertIsNotNone(x)

x is not None

assertIn(a, b)

a in b

assertNotIn(a, b)

a not in b

assertIsInstance(a, b)

isinstance(a, b)

assertNotIsInstance(a, b)

not isinstance(a, b)

 

 

 

 

Go Top

 

 

zipfile
References:help('zipfile') / https://docs.python.org/3/library/zipfile.html

 
Methods Examples
ZipFile.write() from zipfile import ZipFile
zip=ZipFile("/var/tmp/myzipfile.zip",'w')
zip.write("/var/tmp/file1.txt")
zip.write("/var/tmp/file2.log")
zip.close()

 
ZipFile.close()

from zipfile import ZipFile
zip=ZipFile("/var/tmp/myzipfile.zip",'w')
zip.write("/var/tmp/file1.txt")
zip.write("/var/tmp/file2.log")
zip.close()

 

Go Top

 

 

 References

Modules reference : help('modules') /  https://docs.python.org/3/py-modindex.html