Bash (Scripting) - Removing duplicate values from a list

by
Jeremy Canfield |
Updated: June 13 2024
| Bash (Scripting) articles
Let's say you have an list named fruit that contains the following values. Notice apple is listed twice. A for loop can be used to iterate over each item in the list, like this.
#!/bin/bash
fruits=(banana apple orange apple grapes)
for fruit in ${fruits[@]}; do
echo $fruit
done
Which should return the following. Notice apple is still listed twice.
banana
apple
orange
apple
grape
Here is how you can remove duplicate values from the list while preserving the original order of the list (no sort).
#!/bin/bash
fruits=(banana apple orange apple grapes)
fruits=($(echo ${fruits[@]} | tr [:space:] '\n' | awk '!a[$0]++'))
for fruit in ${fruits[@]}; do
echo $fruit
done
Here is how you can sort the list alphabetically and then only return unique values.
IFS=$'\n'
fruit=($(sort --uniq <<<"${fruit[*]}"))
unset IFS
A for loop can be used to iterate over each item in the list.
for myFruit in ${sorted[@]}; do
echo $myFruit
done
Which should return the following. Notice that apple is now only listed once.
apple
banana
grape
orange
Did you find this article helpful?
If so, consider buying me a coffee over at