Chapter Four

Text Processing

The six tools that turn plain text into structured data — paging, matching, editing, and slicing files and streams without ever opening an editor.

6 topics

On a server, almost everything you care about arrives as text: logs, config files, command output, CSV exports, the contents of /proc. The Unix toolset treats that text as a stream you can filter, transform, and reshape one line at a time, and the skill that separates a confident operator from someone who reaches for a GUI is fluency with the handful of tools that do it.

This chapter covers those tools in the order you reach for them: paging through a file to read it, grep and regular expressions to find lines, sed to edit them in place, awk to treat lines as fields and do real computation, find and xargs to run all of this across thousands of files, and the small field tools that handle columns and sorting. Together they are the language of every log-grep one-liner and shell pipeline you will ever write.

The data pipeline

grep ERRORfilter lines

→

awk '{print $5}'extract a field

→

sortorder

→

uniq -ccount

→

sort -rnrank

Topics in This Chapter

Viewing and Paging Files

cat, less, head, and tail — when to dump a file whole and when to page it. Why less beats more on a 2 GB log, and how tail -f follows a file as it grows.

grep and Regular Expressions

The pattern-matching workhorse and the regex syntax behind it. Basic versus extended expressions, anchors and character classes, and why grep -F is faster when you only need a literal string.

The stream editor for non-interactive find-and-replace across files and pipelines. Substitution with capture groups, line addressing, and the in-place edit (-i) that rewrites a config file without an editor.

A full pattern-action language for column-oriented text. Splitting lines into fields, filtering on conditions, and running sums and counts — the right tool the moment a grep-and-cut pipeline starts needing arithmetic.

FieldsScripting

Walking a directory tree by name, size, age, or permission, then feeding the results into another command. Why xargs and find -exec exist, and how -print0 survives filenames with spaces and newlines.

FilesystemBatch

cut, sort, uniq, tr, paste, and join — the small filters that slice columns, order lines, and collapse duplicates. The glue that holds most one-line pipelines together.

ColumnsPipelines