cat

concatenates files and prints the result on the stdout.

cat [options] file1 file2 ..

cat -n app.cpp | more

display app.cpp with line numbers;

cat file02.txt >> my_file.txt

add contents of file02.txt to my_file.txt;

cat *txt > all_info.txt

copy contents of several files [in curr dir] into a single file;

cat info04 1> mydata 2> myerr

redirect stdout to mydata, redirect stderr to myerr; if info04 exists, then mydata will be its exact copy;

cat info05 1> mydata 2>&1

if info05 does not exist, then error msg will be written to mydata, otherwise, contents of info05 will be copied to mydata; in fact, this cmd concatenates stdout and stderr and writes their contents to mydata file;

cat /etc/*-release

Linux name and version numbers; you can also use:

lsb-release -a

Getting misc system info

cat /proc/cpuinfo

show info about CPU installed in your computer;

cat /proc/cpuinfo|grep processor|wc -l

get the number of processors (cores, if CPU is multicore);

cat /proc/cmdline

show command line params received by kernel at the start;

cat /proc/devices

show currently configured block & character devices;

cat /proc/asound/cards

show info about sound cards installed;

cat /proc/filesystems

show fs types supported by the running kernel;

cat /proc/meminfo

show RAM & virtual memory info;

cat /proc/mounts

show mount points;

cat /proc/partitions

show info about disk partitions;

cat /proc/swaps

show info about swap partitions;

cat /proc/version

show kernel & C-compiler versions;

check IP forwarding status (0 means OFF, 1 means ON);

cat /proc/net/if_inet6

check if IPv6 is enabled (if disabled - file does not exist, if enabled, the output usually includes eth0 and lo); see also sysctl;

cat /proc/sys/vm/swappiness

show current swappiness param: default is usually 60, but for SSD you should have 1 (set vm.swappiness=1 in /etc/sysctl.conf);

cat /sys/block/sda/queue/scheduler

show current I/O scheduler; the best for SSD (and probably HDD too) is thought to be deadline (default in Ubuntu and Mint); the above cmd usually outputs "noop [deadline] cfq" which means deadline is active, but noop and cfq are also supported; to change default scheduler, you have to modify /etc/default/grub;

Options

--help    --version

chattr

sets/clears file attributes on an ext2/ext3/ext4 filesystems. Though ~ belongs to e2fsprogs pkg, it supports (at least partially) XFS, ReiserFS, JFS, OCFS2, BTRFS. Note that only root (or a process with CAP_LINUX_IMMUTABLE capability) can set/clear immutable attrib (i). Also, only root (or a process with CAP_SYS_RESOURCE) can set/clear journaling (j).

Take into account that c, s, and u attribs may be not honored by ext2/ext3/ext4 filesystems (depends on kernel version).

chattr =s mesg.txt

set secure deletion attribute for mesg.txt (all prev attribs will be removed);

chattr +i setmode.sh

make this file (setmode.sh) immutable (nobody can change, delete, rename, etc);

chattr -i setmode.sh

remove immutable attrib of the specified file;

chattr [options] [mode] files ...

Options

In mode specification + causes the selected attributes to be added to existing, - causes them to be removed, = means "remove all existing, add the specified".

Attributes

chgrp

changes the group of each file to grp.

chgrp dba control.dbf

change the group of the specified file (control.dbf) to dba;

chgrp -R root ./arc01

change group of subdir arc01 (including all its contents) in the current directory to root;

chgrp [options] grp file

Options

--help    --version    --verbose    -v

-f, --silent, --quiet

suppress most error messages;

-h, --no-dereference

affect symlinks instead of referenced files (useful only on systems that can change the ownership of a symlink);

--dereference

change files referenced by symlinks;

--no-preserve-root

do not treat root dir specially (default);

--preserve-root

fail to operate recursively on root dir;

--reference=file

use file's group rather than specifying grp;

chkconfig

(outdated, related to system init proc) updates and queries runlevel info for system services, maintains links in /etc/rc.d dir hierarchy.

chkconfig --list [name]

chkconfig --add name

chkconfig --del name

chkconfig [--level n] name [on | off | reset]

name is a service name (like httpd, sshd, named). on and off cause service to be started or stopped, reset resets the startup info for the service to whatever is specified in its init script. By default, on and off affect levels 2,3,4,5, while reset affects all levels.

chkconfig cups

check if cups is configured to be started at the current runlevel; this cmd outputs nothing, just sets the exit status;

chkconfig --list

show status of all services [known to ~] at all runlevels;

chkconfig --list anacron

query a service managed by /etc/rc.d;

chkconfig --level 2345 db_ora on

turn on db_ora at runlevels 2, 3, 4, 5 (start Oracle database at normal system startup);

chkconfig --level 06 db_ora off

turn off db_ora in runlevels 0, 6 (shutdown Oracle database when system goes down or restarts);

Options

-h    --help    --version

--add name

add a new service for management by ~;

--del name

remove service from ~ management, and remove corresponding symlinks in /etc/rc.d;

--level n

specify the runlevels to be affected;

--list [name]

show info about the status of the specified service at all runlevels;

Ubuntu (since 10.xx ?) does not install ~ by default. But you can:

sudo apt-get install chkconfig

A note for init script writers

Each service to be managed by ~, needs two or more lines added to its /etc/init.d script file:

# chkconfig: 2345 80 20
# description: Oracle Database

The first line tells ~ in what runlevels (2345) the service should be started up by default, and its start/stop priority levels (80/20 - typical for non-privileged third party services). If the service should not be started by default in any runlevels, put '-' in place of the runlevels list. The 2nd line is a description for the service (add '\' at the EOL to make it multiline).

chmod

sets/changes the access permissions of the specified file/directory.

chmod [options] mode file

It never changes permissions of the symbolic links (in fact, symlink permissions are never used), instead it changes permissions of the pointed-to files. An important thing to remember: chmod ignores symbolic links encountered during recursive directory traversals!

Linux uses symbolic and numeric notations for permissions. See Examples.

Thus, the mode arg is either symbolic representation of changes to be done, or an octal number representing the bit pattern for the new permissions. Note, that '+' at the end of the symbolic notation means that ACL is in effect for that file (see ls for details).

The file arg can be filename, dirname or name pattern (e.g., *.html).

chmod 644 index.html

set -rw-r--r-- permissions on index.html (everybody can read, owner can modify);

chmod 644 V*

set rw-r--r-- perms on all files beginning with V (in the current dir) (everybody can read, owner can write, delete, etc);

chmod 755 VS_VIDEO

set rwxr-xr-x perms on dir VS_VIDEO;

chmod 700 clean_arc.sh

set rwx------ perms on a bash script; it means only owner can run this script; other users cannot even look inside;

chmod a+r readme.doc

add (give) read perm on readme.doc to all users;

chmod u+rwx rlist.pl

give read/write/exec on rlist.pl to owner;

chmod g-w infodata

revoke write perm on infodata from group;

chmod o-rwx rlist.pl

revoke read/write/exec on rlist.pl from others;

chmod u+s /usr/sbin/pppd

set user ID on execution;

Symbolic mode args

Symbolic notation

The first char in ls output is the file type. Linux/POSIX file types:

Then go three permission triads (see Examples):

• first triad - the rights of the owner;
• second triad - the rights of the group;
• third triad - the rights of all other users;

Each char in each triad designates a specific permission:

• - - (any position in triad) no permission;
• r - (first position in triad) read (file is readable);
• w - (second position in triad) write (file is writable);
• x - (third position in triad) execute (file is executable); the third char can also be:
  • s - file is executable with setuid/setgid (1st and 2nd triads only);
  • S - file is NOT executable with setuid/setgid (1st and 2nd triads);
  • t - file is executable with sticky bit (3rd triad only);
  • T - file is NOT executable with sticky bit (3rd triad only);

There is some difference in meaning of read/write/execute permissions when they apply to files and when they apply to directories.

Numeric notation

consists of 4 octal digits (0..7). If it's shorter, then missing digits are assumed to be leading zeros. Each of these digits is the sum of its component bits in the binary numeral system.

The first digit (leftmost) means:

The second, third, fourth digits set permissions on file/dir for owner, group, others (it looks like others does not mean everybody, i.e., file owner and group members are excluded from others). The meaning is like this:

For files:

For directories:

Examples of symbolic and numeric notation

Setuid, setgid on files

The setuid (set user ID) and setgid (set group ID), when it's set on executable file, allow users to run an executable with the permissions of the executable's owner (or group). Usually it means root.

This feature is used to allow users to run programs with temporarily elevated privileges in order to perform specific potentially not safe tasks.

For example, to change password, user must be able to modify /etc/shadow file. But ordinary users cannot even read this file (-rw-r-----). However, /usr/bin/passwd program has setuid bit (-rwsr-xr-x), i.e., user runs this prog as root, and root can modify /etc/shadow.

This feature may not work with shell scripts, and even if it works, you should not set this permission on scripts, or your own binary executables, or any other binary executables unless you are 100% sure the app is perfectly safe, unbreakable, etc. Shortly speaking, don't use it at all, leave it to OS maintainers.

The setuid bit s is displayed at the same location as x permission for owner. It's assumed that x is also present, otherwise setuid bit is displayed as S.

Setuid, setgid on dirs

The setuid permission set on a directory is ignored on most systems.

The setgid permission on a directory (like chmod g+s) causes new files and subdirs created within this dir to inherit group ID of this dir, rather than the primary group ID of the user who created the file. There are following subtle moments:

• the owner ID is never affected, only group ID;
• newly created subdirs inherit setgid (and behavior of the parent dir);
• the group ID of the files that were created elsewhere and moved to this dir is not afftected;
• setting this permission does not affect files and subdirs that existed in this dir before;

The setgid bit s is displayed at the same location as x perm for group owner. It's assumed that x is also present, otherwise setgid bit is displayed as S.

Sticky bit

The Linux kernel ignores the sticky bit on files.

When the sticky bit is set on a directory, the system treats the files in this directory in a special way so that only file's owner, or directory's owner, or root can rename or delete those files. In other words, sticky bit prevents users from removing files that they do not own.

The sticky bit t is displayed at the same location as the x (executable permission) for others. It's assumed that x is also set, otherwise the sticky bit is displayed as T.

Typically sticky bit is set on /tmp and other world-writable dirs to prevent ordinary users from deleting files that belong to other users.

Options

--help    --version    --verbose    -v

-c, --change

report only when a change is made;

-f, --silent, --quiet

suppress most error messages;

-R, --recursive

operate on files/dirs recursively;

--no-preserve-root

do not treat root dir specially (default);

--preserve-root

fail to operate recursively on root dir;

--reference=file

use file's mode instead of mode;

About umask

Though umask (bash built-in cmd) is not directly related to chmod, it's related to permissions.

When a new file (or dir) is created by some app, its permissons are defined by two params: default perms and umask. Default (aka base) perms are usually 666 for files and 777 for dirs; umask is usually 022 (unless changed by user/admin). In fact, if you run umask it outputs 0022. However, the first digit is not essential, because sticky bit, setgid, setuid are never set during file/dir creation. Umask is what it sounds - a mask that turns off perm bits set by default (base) perms. The rough and convenient (though absolutely incorrect) method of calculating real perms is to subtract umask from default perms. With umask = 022 new files will be created with perm 644, new dirs will be created with perms 755. Exec bit is never set for a newly created file, you must set it explicitly afterwards.

There are some diffs between distros concerning umask. For example, Ubuntu Linux may have option

USERGROUPS_ENAB yes

in /etc/login.defs. It means each user has his/her personal (private) group. In this case umask for non-root users is 002, so files will be created with 664, dirs with 775. For details see manpage for pam_umask PAM module.

chown

changes file's owner and group.

chown [options] owner:group file

Dot (.) can be used instead of column (:).

chown pro08 install.doc

make pro08 the owner of install.doc;

chown -R root:root /u02

set owner/group = root/root for the whole dir tree /u02;

chown -R root.root ./install

set owner/group = root/root for the whole dir tree install;

chown alex:users *.png

set ownership of all PNG files in the current dir to user/group = alex/users;

Options

--help    --version    --verbose    -v

-c, --change

report only when a change is made;

-f, --silent, --quiet

suppress most error messages;

-h, --no-dereference

affect symlinks, do not touch files referenced by them;

-R, --recursive

operate on files and dirs recursively;

--dereference

change files referenced by symbolic links;

--from=curr_owner:curr_grp

change the owner and/or group of each file only if its current owner/group match those specified here; either may be omitted, in which case a match is not required for the omitted attribute;

--no-preserve-root

do not treat root dir specially (default);

--preserve-root

fail to operate recursively on root dir;

--reference=file

change owner/group of each file to that of file;

cksum

outputs CRC checksum and byte count of the specified file (multiple files can be selected using a space-separated list or wildcards).

cksum my_notes.tar.gz

count and display CRC checksum and size of the specified file;

cksum notes201?.html

count and display checksums and sizes of all files in the current dir matching the wildcard pattern;

cksum rpt01.html rpt02.html

count and display checksums and sizes of these two files;

cksum *.gz

count and display checksums and sizes of all gzip archives matching the specified wildcard pattern in the current dir;

The only options are --help and --version.

cmp

compares two files of any type and writes the results to the stdout. If filename is - or missing, reads stdin.

cmp [options] file1 [file2 [skip1 [skip2]]]

By default ~ prints nothing if files are the same, otherwise byte and line number at which the first difference occurred are shown. Bytes and lines numbering starts from 1. The optional args skip1, skip2 are the byte offsets from the beginning of file1 and file2, respectively, where the comparison starts. By default the offset is decimal, but can be hex/octal if prefixed with 0x or 0 (zero). Multiplicative suffixes: kB (1000), K (1024), MB (1,000,000), M (1,048,576), etc.

cmp rpt20090504 rpt20090507

compare 2 files; silent termination means that files are identical, otherwise only the first difference is shown;

cmp -s rpt20090504 rpt20090507

compare 2 files silently: nothing is displayed, the only outcome is the exit status (echo $?);

cmp -l --bytes=1024 users.dbf users2.dbf

compare only first 1024 bytes of each file, show each differing byte within that range;

Options

--help    --version    -v

-b, --print-bytes

print differing bytes;

-i n, --ignore-initial=n

skip first n bytes of input;

-i n:m, --ignore-initial=n:m

skip first n bytes of file1 and first m bytes of file2;

-l, --verbose

output byte numbers (decimal) and values (octal) of all differing bytes;

-n m, --bytes=m

compare at most m bytes;

comm

compares 2 sorted files line by line.

comm [options] file1 file2

Without options ~ outputs 3 columns: first contains lines unique to file1, second contains lines unique to file2, third contains lines common to both files.

comm info01.txt info02.txt

compare two text files;

comm -1 -3 index1.html index2.html > upd.html

create file containing only those lines from the second file that are missing from the first file (last additions);

Options

--help    --version

cp

copies files and directories.

cp [options] src dst

If dst is an existing dir, ~ copies each specified file into a file with the same name in that dir. It's an error if the last arg is not a dir and more than two files are given. By default, ~ does not copy dirs. Default backup suffix is "~".

cp index.html index2.html

make a copy of an existing file (index.html);

cp ./tmp4/news.html .

copy news.html from the subdir tmp4 to the current directory;

cp -u ../alex/last_proj/*.cpp ./last_proj

copy brand new files and overwrite (update) existing ones only if source file is newer then destination file;

cp -i ../alex/last_proj/*.cpp ./last_proj

copy files interactively, prompt before overwriting;

cp -a ../alex/last_proj ./last_proj

copy dir recursively, preserve attribs, don't dereference links;

cp -R --preserve=all alex /mnt

copy dir alex recursively to a disk partition mounted on /mnt, preserve all attribs, don't dereference links;

Note!

In some systems, when you invoke this cmd as cp, an alias cp -i is used instead, and you are always asked about overwriting an existing file; to eliminate this implicit -i, either remove the corresponding alias (usually it's in .bashrc) or use /bin/cp;

/bin/cp --backup=none /opt/db/exp.tar.bz2 /var/ftp/pub/

copy exp.tar.bz2 to /var/fpt/pub dir, overwrite the existing file (if ...) without prompts and backup;

Options

--help    --version    --verbose    -v

-a, --archive

same as -cdpR;

-f, --force

if an existing dest file cannot be opened, remove it and try again;

-i, --interactive

prompt before overwrite;

-L, --dereference

always follow symlinks;

-l, --link

link files instead of copying;

-n, --no-clobber

do not overwrite an existing file;

-P, --no-dereference

never follow symlinks in src;

-R, -r, --recursive

copy directories recursively;

-S suffix, --suffix=suffix

override default bkp suffix ~;

-s, --symbolic-link

make symlinks instead of copying;

-T, --no-target-directory

treat dst as a normal file;

-t dir, --target-directory=dir

copy all src args into dir;

-u, --update

copy only when the src file is newer than the dest or when the dest is missing;

-x, --one-file-system

stay on this file system;

-Z ctx, --context=ctx

set security context of the copy to ctx;

--backup[=ctrl]

make a backup of each existing dest file; version control (ctrl) can be:

--copy-contents

copy contents of special files when recursive;

--no-preserve=attr

don't preserve the specified attributes;

--parents

use full src filename under dest directory;

--preserve[=attr]

preserve the specified attributes (comma-separated list, by default mode,ownership,timestamps); other attribs: context (SELinux), links (preserve symlinks as they are in the src), xattr (extended attribs), all (all available attribs);

--remove-destination

remove each existing dest file before attempting to open it;

--sparse=when

control creation of sparse files (when : auto, always, never);

--strip-trailing-slashes

remove any trailing slashes from each src arg;

cpio

copies files to and from archives. In copy-out mode (cpio -o) reads a list of filenames (one per line) from stdin and writes archive to stdout. In copy-in mode (cpio -i) copies files from archive or lists archive contents, reading archive from stdin. In copy-pass mode (cpio -p) copies files from one directory tree to another without using an archive.

cpio -iudv -I /dev/st0 -C 10240

restore all files from a tape archive;

cpio -iv -I /dev/rstp1 -C 10240 "bin/sort" "bin/xbackup"

restore from a tape archive specified files only;

cpio -itv -I /devrstp1 -C 10240

check integrity of a tape archive;

cpio -idmv < ship.lnx32.cpio

unpack cpio archive creating required directories and preserving file modification times;

find . -depth -print -mount | cpio -ov -O /dev/st0 -C 10240

archive to tape all files from the current dir (including subdirs);

find /usr/local -depth -print | cpio -ov | gzip -c > usr_loc.cpio.gz

archive to a file everything in /usr/local;

Options

--help    --usage    --version    --verbose    -v

in : the option is valid for copy-in mode only;
out : the option is valid for copy-out mode only;
pass : the option is valid for copy-pass mode only;

-0, --null

(out/pass) a list of filenames is terminated by a null char instead of a nl;

-a, --reset-access-time

(out/pass) reset the access times of files after reading them;

-A, --append

(out) append to an existing archive;

-b, --swap

(in) swap both halfwords of words and bytes of halfwords in the data; equivalent to -sS;

-C n, --io-size=n

set I/O block size to n bytes, default is 512;

-d, --make-directories

(in/pass) create leading directories when needed;

-E file, --pattern-file=file

(in) read additional patterns specifying filenames to extract or list from file;

-F [[user@]host:]fname, --file=[[user@]host:]fname

use the specified filename instead of stdin or stdout; optional user and host specify the user and host names in case of a remote archive;

-f, --nonmatching

only copy files that do not match any of the given patterns;

-H format, --format=format

use specified archive format (copy-in autodetects, copy-out uses bin by default):

-I [[user@]host:]fname

(out/pass) set archive filename to use instead of stdin; optional user and host specify the user and host names in case of a remote archive;

-i, --extract

(in) extract files from an archive;

-L, --dereference

(out/pass) dereference symlinks;

-l, --link

(pass) link files instead of copying when possible;

-M str, --message=str

output str when the end of a volume of the backup media is reached;

-m, --preserve-modification-time

(in/pass) retain prev file modification times when creating files;

-n, --numeric-uid-gid

in the verbose table of contents listing, show numeric UID and GID;

-O [[user@]host:]fname

(out) set archive filename to use instead of stdout; optional user and host specify the user and host names in case of a remote archive;

-o, --create

create archive (run in copy-out mode);

-p, --pass-through

run in copy-pass mode;

-R [user][:.][group], --owner=[user][:.][group]

(out/pass) set the ownership of all created files to the specified user/group;

-r, --rename

(in) interactively rename files;

-s, --swap-bytes

(in) swap bytes of each halfword in files;

-S, --swap-halfwords

(in) swap halfwords of each word (4 bytes) in files;

-t, --list

print a table of contents of the input;

-u, --unconditional

(in/pass) replace all files unconditionally;

-V, --dot

print a "." for each file processed;

--absolute-filenames

(in) do not strip leading filename components that contain ".." and slashes;

--block-size=n

set I/O block size to n * 512 bytes;

--force-local

archive file is local, even if its name contains colons;

--no-absolute-filenames

(in) create all files relative to the current dir;

--no-preserve-owner

(in/pass) do not change the ownership of the files;

--only-verify-crc

(in) only verify the CRC's of each file in the archive;

--quiet

do not show the number of blocks copied;

--rsh-command=cmd

use remote cmd instead of rsh;

--sparse

(out/pass) write files with large blocks of zeros as sparse files;

--to-stdout

(in) extract files to standard output;

crontab

Probably it will be deprecated in future versions, and maybe you should start learning systemd timers already!

The cron daemon runs in the background and continuoulsy checks /etc/crontab, /etc/cron.*/ dirs, /var/spool/cron/ dir, and executes scheduled cmds. crontab installs, deinstalls, lists the tables used to drive the cron daemon.

crontab [-u user] { -l | -r | -e }

crontab [-u user] file

The last form is used to set crontab from an existing file or stdin. In last case use - instead of filename.

Each user can have his/her own crontab (see in /var/spool/cron/...). If /etc/cron.allow exists, you must be listed therein to use ~. Otherwise, if /etc/cron.deny exists you must not be listed in it. If both are missing, then only root can use ~ (in some systems - all).

You can also use the following special strings:

Examples of crontab entries

45 22 * * fri /usr/local/bin/arc00.sh

run arc00.sh every friday at 22:45;

05 18 * * mon-fri /usr/local/bin/rpt2.pl

run rpt2.pl every day (except Sat and Sun) at 18:05;

1 * * * * /root/bin/save_cnt.sh 1> /dev/null 2> /dev/null

run save_cnt.sh at 1 min after each hour; discard msgs;

0,15,30,45 8-18 * * * /usr/local/bin/chk >/dev/null 2>&1

run chk every 15 min, every day from 8:00 to 18:00, discard msgs;

@hourly /usr/sbin/ntpdate ru.pool.ntp.org >/dev/null 2>&1

sync system time once per hour, discard msgs;

Names are first three letters of a particular day or month, case doesn't matter. Ranges or lists of names are not allowed, but ranges (1-5) and lists (2,5,7) of nums are supported. Asterisk * describes the whole range of values. Ranges can be combined with steps (8-20/2 means 8,10,12,14,..). The part of the cmd line up to a newline or % sign will be executed by /bin/sh or by the shell specifed in the SHELL var in the crontab file. The % sign (unless escaped by '\') means nl, all data after the first % will be sent to the cmd as std input. By default the cmd output is mailed to the crontab owner. You can redirect it using MAILTO=userid in crontab file. cron examines crontab entries once a minute.

Options

-u user

specifies the user whose crontab is to be tweaked; by default this is the user running this cmd; always use this option inside of su;

curl

can be used to transfer resources specified as URLs from/to a server using internet protocols like HTTP, HTTPS, FTP, FTPS, LDAP, SMB, SMTP, and many other. It also provides proxy support, user authentication, FTP upload, HTTP post, SSL connections, cookies, etc.

curl http://localhost

retrieves a default (index) HTML page of a web-server running on localhost (if there is one, of course) and displays it in the terminal (stdout);

curl -v http://localhost

like previous, but outputs more information - mostly HTML header details;

curl http://localhost -v

like prev; you can put options before or after; you can also use --verbose instead of -v;

curl file://localhost/home/alex/public_html/dict/phrasal_verbs.html

outputs contents of a local HMTL file;

curl file:///home/alex/public_html/dict/phrasal_verbs.html

like prev; note the triple slash! double slash is only OK with localhost or 127.0.0.1 or blank (which turns it into a triple slash);

curl http://www.example.com

retrieves a default (home, index) HTML page of the specified web-server and displays it in the terminal;

curl http://www.example.com -i

retrieves a web page and displays its header;

curl http://www.example.com -o page1.html

retrieves a web page and saves it to a file;

curl -x proxy.mydomain.com:3128 http://www.example.org/unix/

downloads via a proxy server;

curl -F password=@/etc/passwd www.mypasswords.com

sends your password file to the server; 'password' is the name of the form-field to which /etc/passwd will be the input;