www.delorie.com/gnu/docs/textutils/coreutils_29.html

search

	Buy GNU books!

GNU Core-utils

7.2 uniq: Uniquify files

uniq writes the unique lines in the given `input', or standard input if nothing is given or for an input name of `-'. Synopsis:

uniq [option]... [input [output]]

By default, uniq prints the unique lines in a sorted file, i.e., discards all but one of identical successive lines. Optionally, it can instead show only lines that appear exactly once, or lines that appear more than once.

The input need not be sorted, but duplicate input lines are detected only if they are adjacent. If you want to discard non-adjacent duplicate lines, perhaps you want to use sort -u.

If no output file is specified, uniq writes to standard output.

The program accepts the following options. Also see 2. Common options.

`-f n'

`--skip-fields=n'

Skip n fields on each line before checking for uniqueness. Fields are sequences of non-space non-tab characters that are separated from each other by at least one space or tab.

On older systems, uniq supports an obsolete option `-n'. POSIX 1003.1-2001 (see section 2.5 Standards conformance) does not allow this; use `-f n' instead.

`-s n'

`--skip-chars=n'

Skip n characters before checking for uniqueness. If you use both the field and character skipping options, fields are skipped over first.

On older systems, uniq supports an obsolete option `+n'. POSIX 1003.1-2001 (see section 2.5 Standards conformance) does not allow this; use `-s n' instead.

`-c'

`--count'

Print the number of times each line occurred along with the line.

`-i'

`--ignore-case'

Ignore differences in case when comparing lines.

`-d'

`--repeated'

Print one copy of each duplicate line.

`-D'

`--all-repeated[=delimit-method]'

Print all copies of each duplicate line. This option is useful mainly in conjunction with other options e.g., to ignore case or to compare only selected fields. The optional delimit-method tells how to delimit groups of duplicate lines, and must be one of the following:

`none'

Do not delimit groups of duplicate lines. This is equivalent to `--all-repeated' (`-D').

`prepend'

Output a newline before each group of duplicate lines.

`separate'

Separate groups of duplicate lines with a single newline. This is the same as using `prepend', except that there is no newline before the first group, and hence may be better suited for output direct to users.

Note that when groups are delimited and the input stream contains two or more consecutive blank lines, then the output is ambiguous. To avoid that, filter the input through `tr -s '\n'' to replace each sequence of consecutive newlines with a single newline.

This is a GNU extension.

`-u'

`--unique'

Print non-duplicate lines.

`-w n'

`--check-chars=n'

Compare n characters on each line (after skipping any specified fields and characters). By default the entire rest of the lines are compared.