Python tips: unpack data gracefully

v_it_aly

Vitaly Shchurov

Posted on December 5, 2020

Python tips: unpack data gracefully

Do you know how to easily assign data to variables from lists of various sizes? How to get a 'head' and 'tail' from a tuple without using clumsy indexes? How to use star expressions? Or simplify your code with a throaway variable? If not, get comfy :)

As you may know, we can assign values from sequences or iterables to variables. In short, a sequence is an ordered collection of items that supports accessing elements by indexing (some_list[0]). An iterable is an object that we can loop over (e.g., set is an iterable: for num in {1, 2, 3}: print(num)).

This assignment process is pretty easy, let's try assigning a list of values concerning some company's CEO:

name, experience, job = ['John Doe', 30, 'CEO']
name, experience, job
('John Doe', 30, 'CEO')
Enter fullscreen mode Exit fullscreen mode

Why not try a tuple?

name, experience, job = ('John Doe', 30, 'CEO')
name, experience, job
('John Doe', 30, 'CEO')
Enter fullscreen mode Exit fullscreen mode

Or a string:

a, b, c, = '123'
a, b, c
('1', '2', '3')
Enter fullscreen mode Exit fullscreen mode

Since the object returned by a range() function produces a sequence, we can easily assign its contents to a bunch of variables:

d, e, f = range(3) 
d, e, f
(0, 1, 2)
Enter fullscreen mode Exit fullscreen mode

If you're not very experienced with this, you may want to try playing with other iterable data structures (say dictionaries, sets).

So, what if you need to process data structures with arbitrary amounts of data? For example, you're expecting a server response with a tuple containing the information you need, but you may get more items than you expected, what then? Trying to unpack the same way will fail:

>>> name, experience, job = ('John Doe', 30, 'CEO', (2012, 1, 24))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: too many values to unpack (expected 3)
Enter fullscreen mode Exit fullscreen mode

Adjusting your code to any possible mismatch wouldn't benefit your realtime application and can easily result in some ugly code. This is where starred expressions come into play. They're called so because you use a * sign before a variable name like that: *var.

Let's handle our last error:

>>> name, experience, job, *other = ('John Doe', 30, 'CEO', (2012, 1, 24), 100000)
>>> name, experience, job
('John Doe', 30, 'CEO')
>>> other
[(2012, 1, 24), 100000]
Enter fullscreen mode Exit fullscreen mode

Here, we assigned all unnecessary information to a special variable. Python takes all remaining values and stores them in a list (always in a list).

In case you don't need all this other data at all, you can make your code even cleaner and use a special throwaway variable _:

>>> name, experience, job, *_ = ('John Doe', 30, 'CEO', (2012, 1, 24), 100000)
Enter fullscreen mode Exit fullscreen mode

The data will still be assigned to it, but other developers will understand that this information won't be used or processed. By the way, you don't have to use it with starred expressions, it can be just _: a, _, b, _ = [1, 2, 3, 4].

You're welcome to use a starred expression in absolutely any part of your assignment:

>>> first, *middle, last = [1, 2, 3, 4]
>>> first
1
>>> middle
[2, 3]
>>> last
4
Enter fullscreen mode Exit fullscreen mode

Or like that:

>>> *first, middle, last = 1, 2, 3, 4
>>> first
[1, 2]
>>> middle
3
>>> last
4
Enter fullscreen mode Exit fullscreen mode

Have you noticed that we assigned variables to 1, 2, 3, 4? It's not a sequence, why didn't Python trow an exception? This is because we type 1, 2, 3, 4, but Python sees it as a tuple. In the examples at the beginning you may have seen that the interpreter always printed tuples as a result.

Another wonderful thing about star expressions is that they handle the situations where there're no extra values to be stored in them. See how we get an empty list in our John Doe example.

>>> name, experience, job, *other = ('John Doe', 30, 'CEO')
>>> name, experience, job
('John Doe', 30, 'CEO')
>>> other
[]
Enter fullscreen mode Exit fullscreen mode

Now, let's try something more spicy :) Don't forget that we can unpack both sequences and iterables with starred expressions. So, let's unpack an iterator, it should be fun:

>>> seq = range(10)
>>> it = iter(seq)
>>> first, *rest = list(it)
>>> first, rest  # notice that the result is printed as a tuple:
(0, [1, 2, 3, 4, 5, 6, 7, 8, 9])
Enter fullscreen mode Exit fullscreen mode

Can we get even more creative and use a couple of starred expressions in our assignment?

Actually, no. When you use a starred expression, you basically tell Python to assign certain values to certain variables, and store the rest of the data in a special starred variable.

So, if you use more than one *var during a single assignment, Python just won't know what to do. So, for one line of code, use only one star. Otherwise, you'll get an exception.

Bonus. Guess, can we use a starred expression alone? I'll publish the answer in the comments in case you'd like to guess it yourself ;)

So, this is how unpacking with starred expressions works in Python. If you'd like to see more practical examples, read my follow-up post.

Feeling curious to read some other tips? Check out my previous Python tips post about elegant exception handling.

💖 💪 🙅 🚩
v_it_aly
Vitaly Shchurov

Posted on December 5, 2020

Join Our Newsletter. No Spam, Only the good stuff.

Sign up to receive the latest update from our blog.

Related

Python tips: unpack data gracefully
python Python tips: unpack data gracefully

December 5, 2020