How to Write to Files in Bash Scripting: A Comprehensive Guide

As a Linux system administrator or developer, you‘ll often need to write data to files from a Bash script. Whether it‘s logging debug messages, saving output, or manipulating configuration files, writing files is a common task.

In this comprehensive guide, I‘ll cover several methods for writing to files in Bash, including when and how to use each one.

Overview of Writing to Files in Bash

Here are some of the main ways you can write to files from Bash scripts:

• Redirect output (> and >>) – Redirect command output to a file with >, appending output with >>. Simple but limited.
• tee command – Write command output to a file and also print to stdout. Useful for logging.
• heredocs – Supply multi-line input to commands like cat and write to files. Great for scripts.
• printf – Format and write output exactly how you want it.

I‘ll provide examples of each method below, along with explanations of how they work and when to use them.

But first, let‘s cover some Bash scripting basics for working with files.

Bash File Basics

Before diving into the different write methods, understanding some core Bash file concepts will be helpful:

File Descriptors

When a Bash script opens a file, it is assigned a file descriptor. By default, these are:

• 0 – Standard input (stdin)
• 1 – Standard output (stdout)
• 2 – Standard error (stderr)

So if you don‘t redirect output to a different file, it will go to stdout.

File Permissions

Linux files have read, write, and execute permissions for the owner, group, and all users. Before writing to files, your script will need write permissions.

Use chmod to modify permissions. For example, to allow the owner to read and write:

chmod 600 file.txt

Checking for Errors

When writing scripts that modify files, add error checking with:

• $? – Contains exit code of last command
• if/else statements – Check exit codes for errors
• bash -x script – Debug scripts

This will help identify and handle issues when working with files.

Okay, now that the basics are covered, let‘s look at some methods for writing files!

Method 1: Redirect Output (> and >>)

The simplest way to write to files is redirecting output using > and >>.

Redirect To File (>)

The > redirect sends output from a command to a file. For example:

$ echo "Linux is cool" > file.txt

This takes the output from the echo command and writes it to file.txt, creating the file if it doesn‘t exist yet.

Important: Using the single > will overwrite the existing file contents!

Append To File (>>)

To append instead of overwriting, use >> like:

$ echo "Learning Bash is fun" >> file.txt

Now file.txt will contain both lines:

Linux is cool
Learning Bash is fun

So >> adds the output to the end of the file.

When To Use

The redirection operators are best for simple cases where you just want to dump output from a single command to a file.

You have minimal control over formatting compared to other methods – it writes the output as-is.

Method 2: The tee Command

The appropriately named tee command lets you redirect output to a file and stdout simultaneously. This makes it great for logging.

Basic tee syntax:

$ command | tee file

Let‘s see an example:

$ echo "Logging test" | tee test.log
Logging test

This pipes the output from echo to tee, which:

1. Writes "Logging test" to test.log
2. Prints "Logging test" to stdout

We can append instead of overwrite using the -a flag:

$ echo "Another log test" | tee -a test.log
Another log test

Now test.log contains both log lines.

Root priviliges

Normally tee won‘t be able to write files you don‘t have write permissions for.

But using sudo tee lets you bypass permissions and log root-only resources:

$ sudo service nginx status | tee -a /var/log/services.log

When To Use

tee is great when you need to write log output somewhere while also seeing it in the terminal standard output.

This helps when debugging issues with commands that run in background processes or cron jobs.

Method 3: Heredocs

Heredocs provide a way to supply multi-line input to text processing commands like cat, awk, tr, etc.

This makes them very useful for writing more complex, formatted output to files in Bash scripts.

Heredoc Syntax

Here is the syntax:

command <<DELIMITER
[ Input lines ]
DELIMITER

It works like this:

1. Specify a delimiter string after <<
2. Input lines until the delimiter is encountered again
3. The input lines get fed to the command

For example:

cat <<EOT >> file.txt
Hello there! 
This is being written 
using a heredoc.
Pretty cool, huh?
EOT

This will append those lines to file.txt.

Some things to note about heredocs:

• Delimiters like EOT, END, or EOF are commonly used
• No spaces around << or the delimiter
• Indenting lines is optional but makes scripts more readable

Next I‘ll show some more useful examples.

Writing Multi-line Script Output

Here‘s a script that uses heredoc and cat to write some formatted output:

#!/bin/bash

cat <<EOF > /home/user/status.txt 
System Status Report
======================= 

$(uname -a)

Disk usage: $(df -h | grep /dev/sda1)  

Memory usage: $(free -m | grep Mem)
EOF

When run it will save nicely formatted system stat output to status.txt.

Self-documenting Scripts

Because heredocs preserve all whitespace and formatting, they are awesome for writing help text and documentation within scripts.

For example:

#!/bin/bash

display_help(){
cat <<HELP 
$(tput bold)Site Backup Script$(tput sgr0)

Description:
  Automatically backs up website files nightly

Usage: 
  $0 [options]

Options:
  -h         Show this help
  -f <dir>   Backup from specified directory
  -t <dir>   Backup to specified directory 
HELP
}

# Check args
if [[ $1 == "-h" ]]; then
   display_help
   exit 0
fi

# Actual script logic goes here  

This prints nice formatted help with some color.

When To Use

Use heredocs when you need to:

• Output multiple lines at once
• Format text nicely for files or stdout
• Document scripts with long help text
• Dynamically generate config files

The multi-line and formatting abilities provide more control than standard output redirection.

Method 4: The printf Command

While heredocs allow formatting over multiple lines, printf gives precise control when writing a single line.

It works similarly to the C printf() function.

Basic printf Syntax

Here‘s the syntax:

printf "format string" arguments 

For example:

printf "My name is %s and I‘m %d years old\n" Chris 33

Would write the formatted string: "My name is Chris and I‘m 33 years old" to stdout.

You specify strings and variables in the format string, and provide corresponding arguments. Common formatting includes:

• %s – String
• %d – Integers
• %f – Floating point

There‘s also syntax for width, padding, precision, etc.

Now let‘s see how to write printf output to files.

Writing printf To Files

To redirect printf output to files, choose from:

printf "%s %s" foo bar > file
printf "%s" "test" | tee file 
printf "lines" >> file

And so on. The same output redirection and piping rules apply.

Here‘s an example script that writes some printf output to a file:

#!/bin/bash 

# Website stats 
bytes=$(cat /var/log/nginx/access.log | awk ‘{s+=$10} END {print s}‘)
hits=$(cat /var/log/nginx/access.log | wc -l)

printf "Total Hits: %d\nTotal Bytes Transferred: %d\n" $hits $bytes >> /var/www/reports/stats.txt 

The file will contain nice formatted website usage statistics.

When To Use

Reasons to use printf include:

• Format output lines just as you want them
• Easily pad/truncate strings, control precision
• Script reporting and logging stats
• Matches C-style printf formatting
• Alternative to fmt or col for text formatting

So turn to printf when you need formatted lines written to files or stdout.

Choosing the Right Write Method

With several options available for writing to files in Bash, how do you choose?

Consider which method best fits your specific needs.

I suggest:

• Redirection – Basic append/overwrite of command output
• tee – Simultaneous write and stdout output for logging
• heredocs – Multi-line input and formatting when writing files
• printf – Precise control over line output formatting

The above scenarios summarize when to use each method.

But feel free to combine them as needed – for example, using printf with tee or append redirection.

The important thing is that you have versatile, powerful options for writing files from Bash scripts.

Now go let your scripts loose reading and writing files!

Reference List:

1. "Bash Redirections". GNU Bash Manual. https://www.gnu.org/software/bash/manual/html_node/Redirections.html

2. "Linux tee Command for Redirection". GeeksForGeeks. https://www.geeksforgeeks.org/tee-command-linux-redirection/

3. "Bash Here Document". Bash Hackers Wiki. https://wiki.bash-hackers.org/syntax/redirection/here-documents

4. "Bash printf Command". Linuxize. https://linuxize.com/post/bash-printf-command/