I recently tried to figure out a subtle bug in our code. The piece of code I was debugging has one sole purpose: get list of items and remove the duplicates preserving the order of the original list.
After implementing testing, it turned out that the order wasn't always preserved as expected.
The way the code was implemented for stripping out the duplicates from the list was very simple:
1>>> my_list = [1, 5, 5, 2, 6, 6]
2>>> my_unique_list = list(set(my_list))
3>>> print(my_unique_list)
4[1, 2, 5, 6]
This looked fine, but the order or the original list was not preserved. After trying multiple different approaches, I settled on:
1>>> from collections import OrderedDict
2>>> my_list = [1, 5, 5, 2, 6, 6]
3>>> my_unique_list = list(OrderedDict.fromkeys(my_list))
4>>> print(my_unique_list)
5[1, 5, 2, 6]
As of Python 3.6, you can change this to:
1>>> my_list = [1, 5, 5, 2, 6, 6]
2>>> my_unique_list = list(dict.fromkeys(my_list))
3>>> print(my_unique_list)
4[1, 5, 2, 6]
Just be aware that this approach has one drawback: it's slower than using the set
approach.
If this post was enjoyable or useful for you, please share it! If you have comments, questions, or feedback, you can email my personal email. To get new posts, subscribe use the RSS feed.