Bootstrap FreeKB - Bash (Scripting) - join lines in a variable
Bash (Scripting) - join lines in a variable

Updated:   |  Bash (Scripting) articles

Let's consider a scenario where you need to merges lines of data in a variable. In this example, the $foo variable contains two lines of data that will be merged into one line of data.

foo="
Employee,name=John Doe,occupation=Engineer
Employee,name=John Doe,department=IT Infrastructure
Employee,name=Jane Doe,occupation=Sales Manager
Employee,name=Jane Doe,department=Sales Department
"

 

The following script demonstrates the logic.

#!/bin/bash

foo="
Employee,name=John Doe,occupation=Engineer
Employee,name=John Doe,department=IT Infrastructure
Employee,name=Jane Doe,occupation=Sales Manager
Employee,name=Jane Doe,department=Sales Department
"

# Tell bash to split at newline
IFS=$'\n'

# Create an empty variable that will contain the merged data
merged=""

# Loop through the variable that contains the lines that need to be merged ($foo in this example)
for line in $foo
do

  # Create unique variable for each field of data from each line in the $foo variable
  field2=$(echo $line | awk -F',' '{print $2}') # name
  field3=$(echo $line | awk -F',' '{print $3}') # occupation or department

  echo "Evaluating '$field2, $field3'"

  # if $merged does not yet contain a line with the common field (name in this example), then execute the logic inside of the if statement
  if [[ $merged != *"$field2"* ]]; then

    echo "\$merged does not contain *$field2* - Adding '$field2 $field3' to the \$merged variable"

    # Append the data to the $merged variable
    merged+="$field2, $field3"

    # Add a new line
    merged+=$'\n'

  # if $merged contain a line with the common field (name in this example) but the line does not yet contain the data in the $field3 variable, then execute the logic inside this elif statement
  elif [[ $merged = *"$field2"* && $merged != *"$field3"* ]]; then

    echo "\$merged already contains '*$field2*' - Appending '$field3' to the line that contains '$field2' in \$merged"

    # Store the data that already exists in $merged in a variable
    for data in $merged
    do
      existingData=$(echo "$data" | grep "$field2")
    done

    # Append the data to the $merged variable
    merged="${merged/$existingData/$existingData, $field3}"


  # if $merged already contains the data, do nothing
  else [[ $merged = *"$field2"* && $merged = *"$field3"* ]]

    echo "\$merged already contains '$field2' and '$field3' - skipping this record and moving on"

  fi

  echo

done 

echo
echo "-----FINAL RESULTS-----"

# Loop through each line
for line in $merged
do
  echo $line
done

# Cleanup
unset IFS

 

Running this script will produce a variable that contains the merged data.

name=John Doe, occupation=Engineer, department=IT Infrastructure
name=Jane Doe, occupation=Sales Manager, department=Sales Department

 




Did you find this article helpful?

If so, consider buying me a coffee over at Buy Me A Coffee



Comments


Add a Comment


Please enter 5b9162 in the box below so that we can be sure you are a human.