Tips-And-Tricks/Computer/Linux/Shell-Commands

Linux is a growing and popular operating system, especially among us citizen mycologists. Unknown to a lot of first-time users, it comes with an extensive set of tools that can do things that feel like black magic on Windows, or even to the casual Mac user.

As a macOS user, you have access to most of these commands too (and surprisingly, without paying an exorbitant fee for them). macOS descends from BSD (the Berkeley Software Distribution), an offshoot of the original AT&T Unix, so most of the core shell commands are at your disposal.

Pipes, Redirection, and Sub-shells

Standard streams

Every process in a *nix shell starts with three open streams, called file descriptors: one for input and two for output. Windows implements the same idea.

Standard file descriptors

Name	FD	Description
stdin	0	Standard input: what the command reads, either typed at the terminal or fed from another command.
stdout	1	Standard output: the command's normal output. This is what normally gets printed to the terminal.
stderr	2	Standard error: error messages and anything the program wants the user to see even when its output is being redirected or piped.

Pipes

The pipe symbol | (usually Shift + backslash) sends the stdout of the command on the left into the stdin of the command on the right. You can chain as many as you like, which makes for very powerful text processing.

Example: list any files from December

ls -l | grep -iw Dec

Example: count how many of those there are

ls -l | grep -iw Dec | wc -l

Redirection

Redirection connects a stream to a file instead of the terminal.

Syntax	What it does
command > file	Write stdout to file, replacing whatever was in it
command >> file	Append stdout to the end of file
command < file	Read stdin from file
command 2> file	Write stderr to file
command > file 2>&1	Write both stdout and stderr to file
command > /dev/null	Throw the output away

Example: save a list of every photo in your grow log folder

ls ~/GrowLog/*.jpg > photos.txt

Note the order in > file 2>&1: it means "send stdout to the file, then send stderr wherever stdout is going." Written the other way around, stderr still goes to the terminal.

Sub-shells

A sub-shell runs commands in a separate copy of the shell.

Command substitution runs a command and drops its output into another command:

echo "Today is $(date +%A)"

mkdir "grow-$(date +%Y-%m-%d)" creates a folder named for today's date.

You may also see the older backtick form, `date`; $( ) is easier to read and can be nested.

Grouping with parentheses runs a set of commands together in a sub-shell. Changes such as cd don't affect your current shell:

(cd ~/GrowLog && ls) lists that folder and leaves you where you started.

You can also redirect a whole group at once:

(date; ls ~/GrowLog) > log.txt

File Compression and Extraction

There are two main utilities for packing a whole directory into a single file, so you can move it to another computer at home or over the internet and unpack it there. Which one to use mostly depends on the computers on each end.

• zip/unzip: built into Microsoft Windows since Windows XP, as well as macOS and most Linux systems
• tar: standard on Linux and every Unix-like system, including macOS. Recent Windows 10 and 11 releases include it too.

zip/unzip

Compression

The zip command packs files into a zip archive, which can be opened almost anywhere. The general format is

zip [flags] destination[.zip] file list

The .zip extension is optional. To zip a list of files in the same directory:

zip filename.zip file1 file2 file3 ...

To zip an entire directory structure, use the -r (recursive) flag. It descends through the whole structure, adding everything it finds:

zip -r docs.zip Documents

Extraction

The unzip command does the opposite and extracts files. It has flags for special situations, but there's really only one you need day to day:

-l lists the contents instead of extracting.

unzip -l zipfile.zip lists all of the files in the archive.

unzip zipfile.zip extracts the files into the current directory.

tar

The tar command works much like zip but supports several kinds of compression. The name stands for "Tape ARchive," from the early days when computers literally used reel-to-reel tape.

tar [flags] archive [file1 file2 file3]

The most commonly used flags are:

• c create an archive
• x extract an archive
• t list the contents (add v for details)
• v verbose: show each file as it's added or extracted
• f filename the archive file to create or read
• z compress with gzip (fast, decent compression)
• j compress with bzip2 (medium speed, better compression)
• J compress with xz (slower, best compression)

Examples:

tar czvf growlog.tar.gz GrowLog creates a gzip-compressed archive of the GrowLog folder.

tar xzvf growlog.tar.gz extracts it.

tar tzvf growlog.tar.gz lists what's inside.

Always include a compression flag when creating an archive. Plain tar doesn't compress at all; it just bundles files together, another holdover from its tape origins.

Working with a single file

Sometimes you only need to compress one file to shrink it, for example to send a video over the internet. As a rule of thumb, the better the compression, the longer it takes. Each tool has its own options, but the basic form is tool filename, which replaces the file with a compressed version ending in the tool's extension.

Common compression utilities

Name	Compress	Decompress	Speed	Compression	Extension
gzip	gzip	gunzip	Fast	Moderate	.gz
bzip2	bzip2	bunzip2	Slow	High	.bz2
xz (LZMA)	xz	unxz	Slowest	Highest	.xz