Bash

From AdminWiki

(Difference between revisions)
Jump to: navigation, search
(Error and Output Redirect)
(Digression for a better understanding of file descriptors)
Line 106: Line 106:
So, what do those numbers mean? If software allocates file descriptors (thats what it gets when it opens files for reading or writing) from the operating system, it will receive integer numbers. Thus, the [[PID]] plus the [[file descriptor]] will uniquely identify an open file system wide - a bit like an ip-address + port pair for a TCP connection.
So, what do those numbers mean? If software allocates file descriptors (thats what it gets when it opens files for reading or writing) from the operating system, it will receive integer numbers. Thus, the [[PID]] plus the [[file descriptor]] will uniquely identify an open file system wide - a bit like an ip-address + port pair for a TCP connection.
-
'''0''' is '''standard input'''
+
* '''0''' is '''standard input'''
-
'''1''' is '''standard output'''
+
* '''1''' is '''standard output'''
-
'''2''' is '''standard error'''
+
* '''2''' is '''standard error'''
-
'''everything else''' are filedescriptors received when the software opened other files, like configuration files, log files etc.
+
* '''everything else''' are filedescriptors received when the software opened other files, like configuration files, log files etc.
-
'''255''' is just there to mess with my explanation.
+
* '''255''' is just there to mess with my explanation.
The whole "standard X" metaphor is a bit to wrap ones head around, but after a while using them extensively and with deep understanding, one will be able to prevent a lot of shit from happening (processes dying under certain conditions, missing logfiles, mailbombs from cron etc) and finally lead a happy life.
The whole "standard X" metaphor is a bit to wrap ones head around, but after a while using them extensively and with deep understanding, one will be able to prevent a lot of shit from happening (processes dying under certain conditions, missing logfiles, mailbombs from cron etc) and finally lead a happy life.
Line 129: Line 129:
  strace echo Hello World 2>&1 |less
  strace echo Hello World 2>&1 |less
the latter will redirect the standard error output into the pipe of the standard output and pipe the resulting combined output through less.
the latter will redirect the standard error output into the pipe of the standard output and pipe the resulting combined output through less.
 +
 +
I encourage you to play with redirection a bit before using it for bigger purposes. At the end of the day. almost all of the bash behaviour for redirection makes sense in the greater picture of the "everything is a file" metaphor - you just need to figure it all out ;)
==== redirect output and error ====
==== redirect output and error ====

Revision as of 13:41, 10 April 2009

Contents

Bash Handling

Job Control

Interactive

Key Combinations

I will not use the ^X notation of key combos since those are a bit misleading imo - after all, we are mostly using lower case letters, also people who modified the ^ or meta keys know how to substitute correctly. And of course the terminal type must be right.

ATTENTION: Bash's key combo mode can be set to editor-alike behaviour, specifically it can be set to behaving like emacs or to behaving like vi. You do this with:

set -o emacs 

for the emacs mode, obviously, and

set -o vi

for the vi mode.

Most distributions of the GNU bash come with emacs mode preset.

For the purpose of readability, we will give the key kombinations as follows:

  • Ctrl-a/A means: "Ctrl-a in emacs mode, "a" in vi mode"
  • Ctrl-r/" means: "Ctrl-r in emacs mode, Ctrl-r in vi mode"
  • Ctrl-o/- means: "Ctrl-o in emacs mode, nothing identical in vi mode (or not known to the author ;)"

A word about the vi mode

The vi mode is only "something like" the vi you know. Per default, you are in insert mode, you get to command mode (the mode where key combos will work) by pressing Esc. If you use Ctrl-c, you will be returned to a clear insert mode Prompt again. By pressing "i", you get back to the insert mode. Be aware that some vi key combos do not really behave alike in bash.

Searching in bash history

Of course you know that bash will pull lines from the history into the buffer with the up and down cursor keys.

Ctrl-r/" allows you to recall-by-typing from the bash history.

Executing in bash history

Ctrl-o/-(emacs) alone does nothing, but while pressed when on the line of a history entry (pulled with Ctrl - r or cursor keys), it will execute the bash history entry and when it exists, put the next command in history into the shell buffer.

Example:

philip@dinky:~$ vi test.c
philip@dinky:~$ make test
cc     test.c   -o test
philip@dinky:~$ ./test
Hello World!
philip@dinky:~$ (press Ctrl-r vi here)
(reverse-i-search)`vi': vi test.c (press enter here)
philip@dinky:~$ vi test.c
philip@dinky:~$ (press Ctrl-r mak here)
(reverse-i-search)`mak': make test (press Ctrl-o here)
philip@dinky:~$ make test
cc     test.c   -o test
philip@dinky:~$ ./test

Moving about the prompt/command line

There are some slight differences between the emacs and the vi versions, mostly about whats considered to be in the copy buffer, where the cursor offset will be etc.

  • ctrl-a/0 move the cursor to the beginning of the bash prompt/line
  • ctrl-e/A move the cursor to the end of line
  • ctrl-w/- cut the word previous to the cursor (including intermittent whitespace)
  • ctrl-u/" cut everything from line beginning to, but not including the cursor position
  • ctrl-k/", /D cut everything from, and including the cursor position to end of line
  • ctrl-_/u undo
  • alt-b/- move the cursor back one word
  • alt-f/- move the cursor forward one word
  • ctrl-y/p paste the last thing cut
  • -/h move cursor left
  • -/l move cursor right
  • -/a start inserting after current cursor position
  • -/D cut everything from, and not including
  • -/dd delete line (save it for pasting)
  • ctrl-c/" reset input buffer (do not save for pasting)
  • arrow-up/j move up through history
  • arrow-down/k move down through history

Bash Programming

Bash is the most common shell form on linux, and most other unix systems.

Just a small reminder. Before you start writing a long bash script, think if you can do this in perl^W python.


Redirection

Standard Input, Output and Error

Digression for a better understanding of file descriptors

If you are on a system with a /proc filesystem (and may god have mercy on your soul if you are not), you will find that there are interesting symlinks in /proc/<pid>/fd. Let me elaborate.

Consider the following: you are logged on, have an interactive shell like bash, and you are the owner of your terminal (tty). Now, find out your PID:

philip@dinky:~$ w |head -3
 15:15:04 up 72 days, 17 min, 35 users,  load average: 0.15, 0.11, 0.09
USER     TTY      FROM              LOGIN@   IDLE   JCPU   PCPU WHAT
philip   tty2     -                11Feb09 58days  0.10s  0.00s /bin/login -- 
philip@dinky:~$ ps waux |grep bash |grep tty2
philip   22008  0.0  0.0  21396  1372 tty2     S    Feb11   0:00 -bash
philip@dinky:~$ 

now, with that information, let's have a peek into our filedescriptors:

philip@dinky:~$ ls -l /proc/22008/fd
total 0
lrwx------ 1 philip philip 64 2009-04-10 15:16 0 -> /dev/tty2
lrwx------ 1 philip philip 64 2009-04-10 15:16 1 -> /dev/tty2
lrwx------ 1 philip philip 64 2009-04-10 15:16 2 -> /dev/tty2
lrwx------ 1 philip philip 64 2009-04-10 15:16 255 -> /dev/tty2
philip@dinky:~$ 

there are, of course, redirections to your current interactive shell. This is why your software will crash receiving a SIGPIPE signal when you close your terminal while something is running.

So, what do those numbers mean? If software allocates file descriptors (thats what it gets when it opens files for reading or writing) from the operating system, it will receive integer numbers. Thus, the PID plus the file descriptor will uniquely identify an open file system wide - a bit like an ip-address + port pair for a TCP connection.

  • 0 is standard input
  • 1 is standard output
  • 2 is standard error
  • everything else are filedescriptors received when the software opened other files, like configuration files, log files etc.
  • 255 is just there to mess with my explanation.

The whole "standard X" metaphor is a bit to wrap ones head around, but after a while using them extensively and with deep understanding, one will be able to prevent a lot of shit from happening (processes dying under certain conditions, missing logfiles, mailbombs from cron etc) and finally lead a happy life.

The standard file descriptors are the fds where software will write and receive their user interaction by default. Yet, software does not necessarily need to do that; while libc provides for every running piece of software with those filedescriptors by standard, the software can close them willingly. Software can indeed decide to "unbind" itself from standard input and output by closing the respective file descriptor. Analog to that, the respective symlinks in the proc fs will disappear, and so will the depdendy on those files (like the interactive shell).

By using redirection, you can remap those standard file descriptors in your own way. Consider doing something like this on one terminal:

philip@dinky:~$ cat > /dev/null

and try finding the PID of this cat process, then do something like the following on another shell (without killing the cat of course):

philip@dinky:~$ ls /proc/10519/fd -l
total 0
lrwx------ 1 philip philip 64 2009-04-10 15:31 0 -> /dev/pts/17
l-wx------ 1 philip philip 64 2009-04-10 15:31 1 -> /dev/null
lrwx------ 1 philip philip 64 2009-04-10 15:31 2 -> /dev/pts/17

so, i hope now it starts making sense for you: by redirecting (which is a bash/shell feature by the way), you can remap the file descriptors to an endpoint you chose. This can be useful for logging errors to a file, like in

find . 2>nopermissions.txt

or to pipe all output through a filter, as in

strace echo Hello World 2>&1 |less

the latter will redirect the standard error output into the pipe of the standard output and pipe the resulting combined output through less.

I encourage you to play with redirection a bit before using it for bigger purposes. At the end of the day. almost all of the bash behaviour for redirection makes sense in the greater picture of the "everything is a file" metaphor - you just need to figure it all out ;)

redirect output and error

 ls -l 1>normal_out 2>error_out

if you want to have both in one file

 ls -l 1>normal_out 2>&1

Questions and Solutions

Below some examples for bash problems that might come up

Arrays in Bash:

 foo[0]=1;
 foo[1]=2;
 foo[2]=3;
 # loop
 for (( i=0; i<${#foo[@]}; i++ ));
 do
     echo ${foo[$i]};
 done;
 # more like a foreach
 for i in ${foo[@]};
 do
     echo $i;
 done;

Variable Variables

 foobar=5;
 bar="foobar";
 echo $foobar;
 echo $bar;
 echo ${!bar};

Variable Variables in Arrays

 foo[0]=1;
 foo[1]=2;
 foo[2]=3;
 foo_tcp[0]="AT";
 foo_upd[0]="AU";
 foo_tcp[1]="BT";
 foo_upd[1]="BU";
 foo_tcp[2]="CT";
 foo_upd[2]="CU";
 # loop
 for (( i=0; i<${#foo[@]}; i++ ));
 do
     echo ${foo[$i]};
     for k in upd tcp;
     do
         data=foo_${k}[$i];
         echo ${!data};
     done;
 done;

More Documentation

The best advanced guide for bash scripting: http://www.tldp.org/LDP/abs/html/

Personal tools