Intro to NumPy

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

1. DataTypes

• NumPy's main datatype is ndarray
  • This type applies to numpy arrays of any dimension

Ndarray Attributes

• ndim – dimension count
• shape – num elements in each sub-dimension
• size – element count
• dtype – type of data stored

arr_1d = np.array([1, 2, 3]) # Shape = (1,3)
arr_1d

array([1, 2, 3])

arr_2d = np.array([[1, 2.0, 3.3],
                   [4, 5, 6.5]]) # Shape = (2,3)

arr_3d = np.array([[[1, 2, 3],
                    [4, 5, 6],
                    [7, 8, 9],
                    [1, 2, 3]],
                   [[10, 11, 12],
                    [13, 14, 15],
                    [16, 17, 18],
                    [10, 11, 12]]]) # Shape = (2,4,3)

type(arr_1d), type(arr_3d)

(numpy.ndarray, numpy.ndarray)

arr_1d.shape

(3,)

arr_3d.shape

(2, 4, 3)

arr_1d.ndim, arr_2d.ndim

(1, 2)

arr_1d.size, arr_2d.size, arr_3d.size

(3, 6, 24)

arr_2d.dtype, arr_3d.dtype

(dtype('float64'), dtype('int32'))

2. Creating Arrays

Basics

np.ones(shape, dtype=None)

• Return a new array of given shape & type, filled with ones

np.zeroes(shape, dtype=None)

• Return a new array of given shape & type, filled with zeroes

np.arange([start,] stop[, step,], dtype=None)

• Return evenly spaced values within a given interval
• stop is exclusive

ones = np.ones((3), dtype='int32')
ones

array([1, 1, 1])

type(ones), ones.dtype

(numpy.ndarray, dtype('int32'))

arr_range = np.arange(0, 10, 2)
arr_range

array([0, 2, 4, 6, 8])

Random Numbers

np.random.randint(low, high=None, size=None, dtype='l')

• Return random integers from low (inc) to high (exc)
• size means shape
• low param is optional, default 0

np.random.random(size=None)

• Return random floats in the half-open interval [0.0, 0.1)

np.random.rand(d0, d1, ..., dn)

• Return random values in a given shape

arr_rand = np.random.randint(0, 10, size=(3, 5))
arr_rand

array([[8, 8, 8, 8, 5],
       [6, 9, 3, 9, 9],
       [1, 2, 9, 9, 9]])

"Elements: " + str(arr_rand.size), "Shape: " + str(arr_rand.shape)

('Elements: 15', 'Shape: (3, 5)')

arr_rand2 = np.random.random((4,2))
arr_rand2

array([[0.77476891, 0.59285154],
       [0.37936252, 0.22070625],
       [0.85267978, 0.82480569],
       [0.15193611, 0.9084882 ]])

arr_rand3 = np.random.rand(4,2)
arr_rand3

array([[0.70077868, 0.38643371],
       [0.5086054 , 0.45856735],
       [0.76164805, 0.01281061],
       [0.07035427, 0.40050648]])

Random Seed

np.random.seed(seed=None)

• A seed allows reproducible randomness, useful when sharing the notebook
• A defined seed only applies to the cell its declared in

np.random.seed(42)
arr_rand4 = np.random.randint(21, size=(2,10))
arr_rand4

array([[ 6, 19, 14, 10,  7, 20,  6, 18, 10, 10],
       [20,  3,  7,  2, 20,  1, 11,  5,  1, 20]])

3. Viewing Arrays & Matrices

Matrix – an array with 2+ dimensions

Access matrix elements by index via square brackets, e.g. arr[index_d0, index_d1]

np.unique(ndarray)

• Return the unique elements as a (sorted?) array
• Works on ndarray of any shape

np.unique(arr_rand4)

array([ 1,  2,  3,  5,  6,  7, 10, 11, 14, 18, 19, 20])

arr_3d, arr_3d[0,2,1]

(array([[[ 1,  2,  3],
         [ 4,  5,  6],
         [ 7,  8,  9],
         [ 1,  2,  3]],
 
        [[10, 11, 12],
         [13, 14, 15],
         [16, 17, 18],
         [10, 11, 12]]]),
 8)

arr_rand4[1], arr_rand4[1,4]

(array([20,  3,  7,  2, 20,  1, 11,  5,  1, 20]), 20)

arr_3d[:2, 1:3, 1:] # Slicing also works on ndarrays

array([[[ 5,  6],
        [ 8,  9]],

       [[14, 15],
        [17, 18]]])

arr_rand5 = np.random.randint(10, size=(2,3,4,5))
arr_rand5

array([[[[0, 9, 5, 8, 0],
         [9, 2, 6, 3, 8],
         [2, 4, 2, 6, 4],
         [8, 6, 1, 3, 8]],

        [[1, 9, 8, 9, 4],
         [1, 3, 6, 7, 2],
         [0, 3, 1, 7, 3],
         [1, 5, 5, 9, 3]],

        [[5, 1, 9, 1, 9],
         [3, 7, 6, 8, 7],
         [4, 1, 4, 7, 9],
         [8, 8, 0, 8, 6]]],


       [[[8, 7, 0, 7, 7],
         [2, 0, 7, 2, 2],
         [0, 4, 9, 6, 9],
         [8, 6, 8, 7, 1]],

        [[0, 6, 6, 7, 4],
         [2, 7, 5, 2, 0],
         [2, 4, 2, 0, 4],
         [9, 6, 6, 8, 9]],

        [[9, 2, 6, 0, 3],
         [3, 4, 6, 6, 3],
         [6, 2, 5, 1, 9],
         [8, 4, 5, 3, 9]]]])

How would we get the center 3 numbers of the innermost arrays?

arr_rand5[:, :, :, 1:4]

array([[[[9, 5, 8],
         [2, 6, 3],
         [4, 2, 6],
         [6, 1, 3]],

        [[9, 8, 9],
         [3, 6, 7],
         [3, 1, 7],
         [5, 5, 9]],

        [[1, 9, 1],
         [7, 6, 8],
         [1, 4, 7],
         [8, 0, 8]]],


       [[[7, 0, 7],
         [0, 7, 2],
         [4, 9, 6],
         [6, 8, 7]],

        [[6, 6, 7],
         [7, 5, 2],
         [4, 2, 0],
         [6, 6, 8]],

        [[2, 6, 0],
         [4, 6, 6],
         [2, 5, 1],
         [4, 5, 3]]]])

4. Manipulating & Comparing Arrays

Broadcasting Rules

When operating on two arrays, NumPy compares their shapes element-wise. It starts with the trailing (i.e. rightmost) dimensions and works its way left. Two dimensions are compatible when:

• they are equal, or
• one of them is 1

NumPy Docs

• Broadcasting
• Reshaping Data

Arithmetic

arr_sm1 = np.array([9, 4, 6, 5])
arr_sm2 = np.array([2, 2, 5, 3])

arr_sm1 + arr_sm2

array([11,  6, 11,  8])

arr_sm1 - arr_sm2

array([7, 2, 1, 2])

arr_sm1 * arr_sm2

array([18,  8, 30, 15])

np.random.seed(8)
arr_md = np.random.randint(1, 10, size=(2,4))
arr_md

array([[4, 5, 2, 6],
       [9, 4, 9, 1]])

arr_sm1 * arr_md

array([[36, 20, 12, 30],
       [81, 16, 54,  5]])

arr_sm1 / arr_md

array([[2.25      , 0.8       , 3.        , 0.83333333],
       [1.        , 1.        , 0.66666667, 5.        ]])

arr_sm1 // arr_md # Double slash means floor division

array([[2, 0, 3, 0],
       [1, 1, 0, 5]], dtype=int32)

arr_md ** 2 # Double asterisk means 'power of'

array([[16, 25,  4, 36],
       [81, 16, 81,  1]], dtype=int32)

arr_md % 3

array([[1, 2, 2, 0],
       [0, 1, 0, 1]], dtype=int32)

Aggregation

Aggregation – performing a collective operation on a group of numbers/items

NumPy Aggregation Functions

• np.sum()
• np.mean()
• np.min()
• np.max()

list_py = [2, 1, 3, 4]
sum(list_py)

10

# arr_sm1 = np.array([9, 4, 6, 5])

np.sum(arr_sm1)

24

arr_massive = np.random.random(100000)
arr_massive[:10]

array([0.61203333, 0.76606293, 0.40192511, 0.87208858, 0.9264384 ,
       0.14652468, 0.70152791, 0.40267737, 0.81826598, 0.02128132])

%timeit sum(arr_massive) # Python's sum()
%timeit np.sum(arr_massive) # NumPy's np.sum()

30.5 ms ± 4.82 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
83.6 µs ± 2.68 µs per loop (mean ± std. dev. of 7 runs, 10000 loops each)

Standard Deviation & Variance

• Variance – the average of the squared differences from the Mean
• Standard Deviation (σ) – a measure of how spread out numbers are
  • σ = √(Variance)

NumPy Functions

• np.var()
• np.std()

Explanation of Standard Deviation

4a. Reshaping & Transposing

• np.reshape()
• arr.reshape()
• arr.T
  • Transposing switches the axes of the array

NumPy Docs: Reshape

arr_2d

array([[1. , 2. , 3.3],
       [4. , 5. , 6.5]])

arr_2d.reshape(2,3,1)

array([[[1. ],
        [2. ],
        [3.3]],

       [[4. ],
        [5. ],
        [6.5]]])

arr_2d.T

array([[1. , 4. ],
       [2. , 5. ],
       [3.3, 6.5]])

arr_3d # shape(2,4,3)

array([[[ 1,  2,  3],
        [ 4,  5,  6],
        [ 7,  8,  9],
        [ 1,  2,  3]],

       [[10, 11, 12],
        [13, 14, 15],
        [16, 17, 18],
        [10, 11, 12]]])

arr_3d.T # shape(3,4,2)

array([[[ 1, 10],
        [ 4, 13],
        [ 7, 16],
        [ 1, 10]],

       [[ 2, 11],
        [ 5, 14],
        [ 8, 17],
        [ 2, 11]],

       [[ 3, 12],
        [ 6, 15],
        [ 9, 18],
        [ 3, 12]]])

4b. Element Wise Ops vs. Dot Product

• Element-wise (Hadamard) Product
• Dot Product

Resources

• Matrix Multiplication Explained
• Matrix Multiplication Waterfall Method

np.random.seed(42)

mtrx_sm1 = np.random.randint(10, size=(5,3))
mtrx_sm2 = np.random.randint(10, size=(5,3))

mtrx_sm1

array([[6, 3, 7],
       [4, 6, 9],
       [2, 6, 7],
       [4, 3, 7],
       [7, 2, 5]])

mtrx_sm2

array([[4, 1, 7],
       [5, 1, 4],
       [0, 9, 5],
       [8, 0, 9],
       [2, 6, 3]])

# Element-wise
mtrx_sm1 * mtrx_sm2

array([[24,  3, 49],
       [20,  6, 36],
       [ 0, 54, 35],
       [32,  0, 63],
       [14, 12, 15]])

mtrx_sm1dp2 = np.dot(mtrx_sm1, mtrx_sm2.T)

mtrx_sm1dp2

array([[ 76,  61,  62, 111,  51],
       [ 85,  62,  99, 113,  71],
       [ 63,  44,  89,  79,  61],
       [ 68,  51,  62,  95,  47],
       [ 65,  57,  43, 101,  41]])

4c. Comparison Operators

NumPy Docs: Logic Functions

arr_1d, arr_2d

(array([1, 2, 3]),
 array([[1. , 2. , 3.3],
        [4. , 5. , 6.5]]))

arr_1d < arr_2d

array([[False, False,  True],
       [ True,  True,  True]])

arr_1d == arr_2d

array([[ True,  True, False],
       [False, False, False]])

np.not_equal(arr_1d, arr_2d)

array([[False, False,  True],
       [ True,  True,  True]])

arr_2d < 4.5

array([[ True,  True,  True],
       [ True, False, False]])

5. Sorting Arrays

• np.sort()
• np.argsort()

np.random.seed(50)

arr_rand6 = np.random.randint(50, size=(3,5))
arr_rand6

array([[48, 32, 11, 45, 33],
       [30,  4,  6, 37,  6],
       [22, 45,  5,  2, 31]])

np.sort(arr_rand6)

array([[11, 32, 33, 45, 48],
       [ 4,  6,  6, 30, 37],
       [ 2,  5, 22, 31, 45]])

arr_rand6

array([[48, 32, 11, 45, 33],
       [30,  4,  6, 37,  6],
       [22, 45,  5,  2, 31]])

np.argsort(arr_rand6)

array([[2, 1, 4, 3, 0],
       [1, 2, 4, 0, 3],
       [3, 2, 0, 4, 1]], dtype=int64)

Argsort is bit tricky at first. Be careful of the Einstellung trap here!

The indices along the sorted axis are NOT telling you the sorted position of the corresponding element in the unsorted array.

So how does Argsort actually work?

The indices themselves are in sorted order. E.g. looking at row 1 above:

• index 2 holds the smallest element of the row, i.e. 11
• index 1 holds the next largest, i.e. 32
• ...
• index 0 holds the largest element, i.e. 48

arr_rand6 # Shape: (3,5)

array([[48, 32, 11, 45, 33],
       [30,  4,  6, 37,  6],
       [22, 45,  5,  2, 31]])

np.argmin(arr_rand6)

13

np.argmax(arr_rand6)

0

np.argmax(arr_rand6, axis=1)

array([0, 3, 1], dtype=int64)

np.argmin(arr_rand6, axis=0)

array([2, 1, 2, 2, 1], dtype=int64)

6. Images → NumPy Arrays

from matplotlib.image import imread

panda = imread("../images/hello-numpy/panda.png")

type(panda)

numpy.ndarray

panda.shape, panda.size

((2330, 3500, 3), 24465000)