String Manipulation in Bash (Without sed or awk)

Built-In Is Faster

People reach for sed, awk, or tr for every string operation. Each spawns a process.

Bash 4+ has native string functions. One process instead of five. Faster and more readable.

Substring Extraction

Extract characters at a position:

str="hello world"

# Extract 5 characters starting at position 0
echo "${str:0:5}"  # hello

# Extract from position 6 onwards
echo "${str:6}"    # world

# Extract from the end
echo "${str: -5}"  # world (note the space before the -)

${string:start:length} is the syntax. Negative indices count from the end.

Real use case: extract part of a filename:

filename="backup_2025-04-04.tar.gz"

# Remove everything before the underscore
date="${filename#*_}"
echo "$date"  # 2025-04-04.tar.gz

# Remove the .tar.gz
base="${date%.tar.gz}"
echo "$base"  # 2025-04-04

${var#pattern} removes the shortest match from the start. ${var##pattern} removes the longest. ${var%pattern} removes from the end.

Remove Prefix or Suffix

Strip protocol from a URL:

url="https://example.com/path"

# Remove https://
noprotocol="${url#https://}"
echo "$noprotocol"  # example.com/path

Strip extension from a filename:

filename="backup.tar.gz"

# Remove .gz
nogz="${filename%.gz}"
echo "$nogz"  # backup.tar

# Remove all extensions
noext="${filename%%.*}"
echo "$noext"  # backup

# removes from the start. % removes from the end. Double (## or %%) is greedy.

Find and Replace

Replace all occurrences:

text="docker docker docker"
echo "${text//docker/podman}"  # podman podman podman

${var//find/replace} replaces all. Single slash /var/find/replace replaces only the first.

Replace only at the start:

path="/usr/bin/python"
echo "${path/#\/usr/\/opt}"  # /opt/bin/python

/# anchors to the start. /% anchors to the end.

Case Conversion (Bash 4+)

Convert to uppercase:

name="alice"
echo "${name^^}"  # ALICE

Convert to lowercase:

name="ALICE"
echo "${name,,}"  # alice

Single caret/comma converts only the first character:

name="alice"
echo "${name^}"   # Alice

name="ALICE"
echo "${name,}"   # aLICE

Real example: normalize user input:

read -p "Enter env (dev/prod): " env
env="${env,,}"  # Force lowercase

if [[ "$env" == "prod" ]]; then
  echo "Production mode"
fi

Default Values and Fallbacks

Use a variable, or a default if it’s unset:

echo "${var:-default}"   # Use default if var is unset or empty
echo "${var-default}"    # Use default only if var is unset (not empty)

Assign a default if unset:

echo "${var:=default}"   # Use default AND assign it to var

Remove characters that match a pattern:

# Not string manipulation per se, but useful
ip="192.168.1.1"
nodots="${ip//.}"
echo "$nodots"  # 19216811

Length

Get string length:

str="hello"
echo "${#str}"  # 5

Also works for arrays:

arr=("one" "two" "three")
echo "${#arr[@]}"  # 3

Trimming Whitespace

Bash doesn’t have a built-in trim, but here’s a one-liner:

# Trim leading and trailing whitespace
str="  hello world  "
trimmed="${str#"${str%%[![:space:]]*}"}"    # Remove leading
trimmed="${trimmed%"${trimmed##*[![:space:]]}"}"  # Remove trailing
echo "'$trimmed'"  # 'hello world'

That’s complex. Better approach: use a function:

trim() {
  local var="$*"
  # Remove leading whitespace
  var="${var#"${var%%[![:space:]]*}"}"
  # Remove trailing whitespace
  var="${var%"${var##*[![:space:]]}"}"
  echo "$var"
}

result=$(trim "  hello world  ")
echo "'$result'"  # 'hello world'

Or just use xargs:

echo "  hello world  " | xargs
# Outputs: hello world

Less elegant, but 5 seconds to write.

Splitting Strings

Split on a delimiter:

csv="alice,bob,charlie"
IFS=',' read -ra names <<< "$csv"

for name in "${names[@]}"; do
  echo "User: $name"
done

# Output:
# User: alice
# User: bob
# User: charlie

IFS is the field separator. <<< is a here-string. read -ra array reads into an array.

Real Example: Parse a Config Line

You have a config file:

username=alice
password=secret123
home=/home/alice

Parse it:

#!/bin/bash

while IFS='=' read -r key value; do
  [[ "$key" == \#* ]] && continue  # Skip comments
  [[ -z "$key" ]] && continue      # Skip empty lines

  # Trim whitespace
  key="${key// }"
  value="${value// }"

  echo "Key: $key, Value: $value"
done < config.txt

Output:

Key: username, Value: alice
Key: password, Value: secret123
Key: home, Value: /home/alice

Real Example: Extract Version from Binary

#!/bin/bash

output=$( myapp --version )
# Output: "MyApp version 1.2.3, built 2025-04-04"

# Extract just the version number
version="${output##*version }"     # Remove everything up to and including "version "
version="${version%%,*}"            # Remove everything from the comma onwards
echo "Version: $version"            # Version: 1.2.3

When to Reach for sed or awk

Bash string operations are fast for:

Substring extraction
Find-replace
Prefix/suffix removal
Case conversion

Use sed/awk when:

Regex is complex
You’re processing line-by-line in a loop (then awk is the tool, not bash)
You need to maintain state across lines

For 80% of sysadmin string work, bash built-ins are enough and faster.