We use python's Lists pretty much everywhere. Since we don't declare it's size and we just add items, Ever wondered how and when the memory allocation and consumption is done under the hood?

Lets analyze the same now, for that we need...

1. sys package to get the list size
2. matplotlib for plotting graphs
Lets start writing code by importing the packages.

import sys
from matplotlib import pyplot as plt

Lets print and see the length and memory of our list for 10 items.


l = [] # our list
size = 10

# here we are just adding some values to our list

for counter in range(size):
    l.append(counter)
    print("length {}, memory in bytes {}".format(
        length[counter], memory[counter]))

We can see the following in console.

length 1, memory in bytes 96
length 2, memory in bytes 96
length 3, memory in bytes 96
length 4, memory in bytes 96
length 5, memory in bytes 128
length 6, memory in bytes 128
length 7, memory in bytes 128
length 8, memory in bytes 128
length 9, memory in bytes 192
length 10, memory in bytes 192

We can see that after about 4 iterations each time, the memory is increased.
Now lets increase the loop counter to 1000 and graph it out...


size = 1000 # increasing the size
length = [] # to hold lengths at each iteration
memory = [] # to hold memory values at each iteration

And then change the for loop


for counter in range(size):
    l.append(counter)
    # capture and store the length
    length.append(len(l)) 
    # Capture and store the memory in bytes
    memory.append(sys.getsizeof(l))

Now lets plot the graph using the captured values and see what is shows.


plt.plot(length, memory)
plt.title("Python List length - Memory comparison")
plt.xlabel("Length")
plt.ylabel("Memory in bytes")
plt.show()



We can see that when ever list is running out of size, its getting added with additional memory.

Now, Lets change the code to capture the difference in memory only when there is a change.


# declaring two more lists
counter_at_which_memory_changed = []
changed_memory_difference = []

and changing the for loop as follows


for counter in range(size):
    l.append(counter)
    # capture and store the length
    length.append(len(l))
    # Capture and store the memory in bytes
    memory.append(sys.getsizeof(l))

    if memory[counter] != memory[counter - 1]:
        counter_at_which_memory_changed.append(counter)
        changed_memory_difference.append(
            memory[counter] - memory[counter - 1])

# Lets try checking the first 10 values of length and difference in memory

for counter in range(10):
    print("at list length {}, memory increase in bytes {}".format(
        counter_at_which_memory_changed[counter],
        changed_memory_difference[counter]))

The first 10 values of memory changes is as follows.

at list length 4, memory increase in bytes 32
at list length 8, memory increase in bytes 64
at list length 16, memory increase in bytes 72
at list length 25, memory increase in bytes 80
at list length 35, memory increase in bytes 88
at list length 46, memory increase in bytes 96
at list length 58, memory increase in bytes 112
at list length 72, memory increase in bytes 128
at list length 88, memory increase in bytes 144
at list length 106, memory increase in bytes 160

# And the graph for index - memory difference is as follows
plt.plot(counter_at_which_memory_changed, changed_memory_difference)
plt.title("Counter - Memory difference")
plt.xlabel("Counter")
plt.ylabel("Memory difference")
plt.show()



So, by checking the difference we can see that the memory increase is follows..
8, 8, 8, 8, 16, 16, 16, 16, 16, 24, 24, 32, 32, 32, 40, 48, 48, 56, 64, 72, 88, 88, 104, 112.

Thanks for your time, this is my first post. Comments are welcome :)