Useful *NIX Shell Commands for Web Developers

So, why *NIX and why do you need to go to the console?

According to the statistics of W3Techs, Unix is
used by 68% of all the websites whose operating system they know.
This means that if you are web developer, your code is most probably running on a Linux server. And at least you need to know how to
configure and debug your code on Unix and Linux systems. Let's find
out what you need to know to feel comfortable in the command line.

The Basics

The basic *NIX command consists of three components:

  • command or program to run
  • options to alter or specify the behavior of the command
  • arguments or input data that is
    needed to run the command   

For example, if you need to get a list of
files in the directory /var/www, you need to run the command ls with the argument /var/www. To add the size of files to the output, you need to add the -s option, and the final command will look like this:

I/O Redirections and Pipelines

Many *NIX commands use text input and
output which you can operate with, and the great feature of this is
that you can send the output results of the command to a file using redirect,
or even pass the output of one command to the input of another command
using the pipelines. For example, we can output the command from the previous
example to a file:

This command will create or erase file
/var/www/files.txt and output a list of files in the /var/www directory.
Here is a list of standard I/O redirections and pipelines:

  • > Redirect output from a
    command to a file on disk. The file will be erased and overwritten.
  • >>
    The same redirect, but appending the output file.
  • < Get
    input to command from a file.
  • | Pass the
    output of one command to the input of another command.
  • tee Both
    redirect output to a file and pass it to the next command in the
    pipeline.

The Main Commands

To get manual pages for a command, run
man. Manual pages follow a common layout and may include name, synopsis,
description, and usage examples. This will display the documentation
for the chmod command:

To execute some commands like saving
configurations or rebooting processes, you need to run them as
the super user. To do this, you need to prepend sudo to your command:

If you need to execute a bunch of
commands as a super user, you can use su, or switch user command.

Note: To save the security layer and
avoid accidental execution of objectionable commands, do not use
sudo and su without any purpose.

Into the Real World

Basic Navigation

There are three main commands to
navigate in the file tree:

  • pwd to print the name of the current
    working directory
  • cd to change directory
  • ls to list directory contents

Here is an example of using these commands with the output of terminal:

Searching Files

There is the find command to search for
files in a directory hierarchy. This command is very powerful and can
search for files and directories by name, access permissions, date, and size.

Find all directories with the “logs”
name in the /var/www directory using the -type option:

To search PHP files in the current
directory, add the -name option:

Find files with defined permissions
using the -perm option:

Find all files which are greater than
500MB:

Of course, you can combine all of those
options in one command, and this is only the basics of the find command, which
is a very powerful tool for searching files. Use manual pages to get more
information.

Manipulating Files and Folders

There are five
main commands for manipulating files and folders in *NIX system:

  • touch is used to change timestamps on existing files and
    directories, but also this is the easiest way to create new file
  • mkdir to make
    directories
  • cp to copy
    files and directories
  • mv to move or
    rename files and directories
  • rm to delete
    files and folders

The next example will
create a file index.html, copy this file to the new directory in
/var/www, and remove the source file.

Another
great command is ln, which is designed to make links between files.
The command ln is often used to create a symbolic link for
enabling a virtual host:

Change Access Permissions

To change the file owner and group, use
chown. Don't forget to grant ownership to the apache user when you are
creating a new virtual host of your web application:

Sometimes cache or log directories of
your application must be writable for all users so you need to change
access modes to 777 with the chmod command. Add the -R option to add permission to all nested
files and folders.

If you just want to make a file
executable, use chmod with the +x option.

File Reading

To view files in the console, you can use the cat command. With cat, you can concatenate files' contents using extra
parameters, and you can also use mask in filenames.

But the cat command will get you
confused very fast, because it shows output in raw format without
any paging—so it is inconvenient to use with log output. To get
a filter for paging through text one screenful at a time, you should
use the more or less commands, which are much of a muchness.

Another useful command is tail, which is
created to output the last part of files. This command is perfect to look through log histories. By default, this tail command prints the last 10
lines, and you can change this number using the -n parameter.

But if you have, for example, a bunch
of log files, you need something more powerful to make a proper
search. Something like grep—a program that reads from standard
input, tests each line against a pattern, and writes to standard
output the lines that match this pattern. By using it in combination with cat
and pipelines, you will get what you want.

If you want to filter text-lines of
output, you can use the grep command:

As you can see, grep is good for using
in pipelines. In this example, the last command will output all lines
containing the “shutting down” string from log-files.

File Editing

If you want to edit text files in
console mode, you can use one of the three most popular text editors:

  • GNU nano, a small and friendly
    default text editor, which is a perfect choice for basic tasks
  • Vim, an improved programmers' text
    editor, which is most powerful, but complex for beginners
  • mcedit, a full-featured windowed
    editor from Midnight Commander, which is easy to use but not
    installed by default on *NIX systems

Compare them and make your choice:

Archiving

Sometimes you need to back up or
compress some data on your server.

The most common archiving utilities are tar
and zip. Notice that the zip command may not be installed on
your server by default.

You can create an archive with the following commands:

If you want just to see a list of files
in the archive, you can use the -l option for both tar and unzip:

Or extract some source files:

Schedule Tasks

If you want to schedule your scripts to
run periodically, you need to use the Cron utility, which is driven by a
cron table—a configuration file that specifies the shell commands to
run periodically on a given schedule. And the command to maintain
cron tables is crontab.

Calling crontab with option -l will
show your cron table.

Also, the -u option is provided to specify
the name of the user whose crontab is being used. If you are going
to run tasks of your web application, it is better to edit crontab
for user www-data.

In this output, you can take a look at an example of a cron table. As you can see, every line is scheduled by minute, hour,
day of month, month, and day of week. Every field may be an asterisk, which means every value of the field. Also you can use sets and ranges using
commas and hyphens. Following a range with a slash specifies skips of
the number's value through the range. In this example, the first command
will run every five minutes, and the second command will run from
Monday to Friday at 15:00.

To edit this list, run crontab with the -e
key instead of -l. The cron list will be opened in your default editor.
Use the -r option to clear the cron list.

Performance Monitoring

Command top shows system summary
information and provides a dynamic real-time view of running
system processes. Press Shift-M to sort processes by memory usage, or Shift-P to sort by CPU usage.

To display the amount of free and used memory
in the system, use the free command. Add the -h option to display output fields
in human-readable format.

Another useful command is df, which is a
command to report file system disk space usage. You can call it with the -a option to show all the file systems of your server. Also, don't
forget to add the -h option for human-readable format.

Command Line History

You can use the !! command to repeat the previous command, or use sudo !! if you forgot to run a command with
sudo.

If you forgot the syntax of commands or are feeling lazy about typing a large command query, you can use history to
display your command history. It is good to combine this command with
strings filter commands like grep, tail and others to find exactly what
you want.

Conclusion

Using the console is not rocket science.
Unix and Linux systems are easy to understand and use because of their simple design and good documentation. I hope this article will make
you pretty comfortable with the command line and bring you to the
next level of managing your web applications with the command line.

If you have any questions or you want
to share your favourite console commands, don't hesitate to leave a
comment below the article.

Further Reading

Source: Tuts Plus

About the Author