Unix Shell Scripting Basics

Unix Shell Scripting Basics — Presentation Transcript

• UNIX Shell-Scripting Basics
• Agenda
  • What is a shell? A shell script?
  • Introduction to bash
  • Running Commands
  • Applied Shell Programming
• What is a shell?
  • %
  ▌
• What is a shell?
  • /bin/bash
• What is a shell?
  • #!/bin/bash
• What is a shell? INPUT shell OUTPUT ERROR
• What is a shell?
  • Any Program
  • But there are a few popular shells…
• Bourne Shells
  • /bin/sh
  • /bin/bash “Bourne-Again Shell”
  Steve Bourne
• Other Common Shells
  • C Shell ( /bin/csh )
  • Turbo C Shell ( /bin/tcsh )
  • Korn Shell ( /bin/ksh )
• An aside: What do I mean by /bin ?
  • C Shell ( /bin/csh )
  • Turbo C Shell ( /bin/tcsh )
  • Korn Shell ( /bin/ksh )
• An aside: What do I mean by /bin ?
  • /bin, /usr/bin, /usr/local/bin
  • /sbin, /usr/sbin, /usr/local/sbin
  • /tmp
  • /dev
  • /home/borwicjh
• What is a Shell Script?
  • A Text File
  • With Instructions
  • Executable
• What is a Shell Script?
  • % cat > hello.sh <<MY_PROGRAM
  • #!/bin/sh
  • echo ‘Hello, world’
  • MY_PROGRAM
  • % chmod +x hello.sh
  • % ./hello.sh
  • Hello, world
• What is a Shell Script? A Text File
  • % cat > hello.sh <<MY_PROGRAM
  • #!/bin/sh
  • echo ‘Hello, world’
  • MY_PROGRAM
  • % chmod +x hello.sh
  • % ./hello.sh
  • Hello, world
• An aside: Redirection
  • cat > /tmp/myfile
  • cat >> /tmp/myfile
  • cat 2> /tmp/myerr
  • cat < /tmp/myinput
  • cat <<INPUT Some input INPUT
  • cat > /tmp/x 2>&1
  0 1 2 INPUT env OUTPUT ERROR
• What is a Shell Script? How To Run
  • % cat > hello.sh <<MY_PROGRAM
  • #!/bin/sh
  • echo ‘Hello, world’
  • MY_PROGRAM
  • % chmod +x hello.sh
  • % ./hello.sh
  • Hello, world
• What is a Shell Script? What To Do
  • % cat > hello.sh <<MY_PROGRAM
  • #!/bin/sh
  • echo ‘Hello, world’
  • MY_PROGRAM
  • % chmod +x hello.sh
  • % ./hello.sh
  • Hello, world
• What is a Shell Script? Executable
  • % cat > hello.sh <<MY_PROGRAM
  • #!/bin/sh
  • echo ‘Hello, world’
  • MY_PROGRAM
  • % chmod +x hello.sh
  • % ./hello.sh
  • Hello, world
• What is a Shell Script? Running it
  • % cat > hello.sh <<MY_PROGRAM
  • #!/bin/sh
  • echo ‘Hello, world’
  • MY_PROGRAM
  • % chmod +x hello.sh
  • % ./hello.sh
  • Hello, world
• Finding the program: PATH
  • % ./hello.sh
  • echo vs. /usr/bin/echo
  • % echo $PATH /bin:/usr/bin:/usr/local/bin: /home/borwicjh/bin
  • % which echo /usr/bin/echo
• Variables and the Environment
  • % hello.sh
  • bash: hello.sh: Command not found
  • % PATH=“$PATH:.”
  • % hello.sh
  • Hello, world
• An aside: Quoting
  • % echo ‘ $USER ’
  • $USER
  • % echo “ $USER ”
  • borwicjh
  • % echo “ ” ”
  • ”
  • % echo “deacnet sct”
  • deacnetsct
  • % echo ‘ ” ’
  • ”
• Variables and the Environment
  • % env
  • […variables passed to sub-programs…]
  • % NEW_VAR=“Yes”
  • % echo $NEW_VAR
  • Yes
  • % env
  • […PATH but not NEW_VAR…]
  • % export NEW_VAR
  • % env
  • […PATH and NEW_VAR…]
• Welcome to Shell Scripting! Shebang! The Environment PATH Input, Output, and Error chmod
• How to Learn
  • man
  • man bash
  • man cat
  • man man
  • man –k
  • man –k manual
  • Learning the Bash Shell , 2 nd Ed.
  • “ Bash Reference” Cards
  • http://www.tldp.org/LDP/abs/html/
• Introduction to bash
• Continuing Lines:
  • % echo This
  • Is
  • A
  • Very
  • Long
  • Command Line
  • This Is A Very Long Command Line
  • %
• Exit Status
  • $?
  • 0 is True
  • % ls /does/not/exist
  • % echo $?
  • 1
  • % echo $?
  • 0
• Exit Status: exit
  • % cat > test.sh <<_TEST_
  • exit 3
  • _TEST_
  • % chmod +x test.sh
  • % ./test.sh
  • % echo $?
  • 3
• Logic: test
  • % test 1 -lt 10
  • % echo $?
  • 0
  • % test 1 == 10
  • % echo $?
  • 1
• Logic: test
  • test
  • [ ]
  • [ 1 –lt 10 ]
  • [[ ]]
  • [[ “this string” =~ “this” ]]
  • (( ))
  • (( 1 < 10 ))
• Logic: test
  • [ -f /etc/passwd ]
  • [ ! –f /etc/passwd ]
  • [ -f /etc/passwd –a –f /etc/shadow ]
  • [ -f /etc/passwd –o –f /etc/shadow ]
• An aside: $(( )) for Math
  • % echo $(( 1 + 2 ))
  • 3
  • % echo $(( 2 * 3 ))
  • 6
  • % echo $(( 1 / 3 ))
  • 0
• Logic: if
  • if something
  • then
  • :
  • # “elif” a contraction of “else if”:
  • elif something-else
  • then
  • :
  • else
  • then
  • :
  • fi
• Logic: if
  • if [ $USER –eq “borwicjh” ]
  • then
  • :
  • # “elif” a contraction of “else if”:
  • elif ls /etc/oratab
  • then
  • :
  • else
  • then
  • :
  • fi
• Logic: if
  • # see if a file exists
  • if [ -e /etc/passwd ]
  • then
  • echo “/etc/passwd exists”
  • else
  • echo “/etc/passwd not found!”
  • fi
• Logic: for
  • for i in 1 2 3
  • do
  • echo $i
  • done
• Logic: for
  • for i in /*
  • do
  • echo “Listing $i:”
  • ls -l $i
  • read
  • done
• Logic: for
  • for i in /*
  • do
  • echo “Listing $i:”
  • ls -l $i
  • read
  • done
• Logic: for
  • for i in /*
  • do
  • echo “Listing $i:”
  • ls -l $i
  • read
  • done
• Logic: C-style for
  • for (( expr1 ;
  • expr2 ;
  • expr3 ))
  • do
  • list
  • done
• Logic: C-style for
  • LIMIT=10
  • for (( a=1 ;
  • a<=LIMIT ;
  • a++ ))
  • do
  • echo –n “$a ”
  • done
• Logic: while
  • while something
  • do
  • :
  • done
• Logic: while
  • a=0; LIMIT=10
  • while [ &quot;$a&quot; -lt &quot;$LIMIT&quot; ]
  • do
  • echo -n &quot;$a ”
  • a=$(( a + 1 ))
  • done
• Counters
  • COUNTER=0
  • while [ -e “$FILE.COUNTER” ]
  • do
  • COUNTER=$(( COUNTER + 1))
  • done
  • Note: race condition
• Reusing Code: “Sourcing”
  • % cat > /path/to/my/passwords <<_PW_
  • FTP_USER=“sct”
  • _PW_
  • % echo $FTP_USER
  • % . /path/to/my/passwords
  • % echo $FTP_USER
  • sct
  • %
• Variable Manipulation
  • % FILEPATH=/path/to/my/output.lis
  • % echo $FILEPATH
  • /path/to/my/output.lis
  • % echo ${FILEPATH %.lis }
  • /path/to/my/output
  • % echo ${FILEPATH #*/ }
  • path/to/my/output.lis
  • % echo ${FILEPATH ##*/ }
  • output.lis
• • It takes a long time to become a bash guru…
• Running Programs
• Reasons for Running Programs
  • Check Return Code
  • $?
  • Get Job Output
  • OUTPUT=`echo “Hello”`
  • OUTPUT=$(echo “Hello”)
  • Send Output Somewhere
  • Redirection: < , >
  • Pipes
• Pipes
  • Lots of Little Tools
  • echo “Hello” |
  • wc -c
  INPUT echo OUTPUT ERROR 0 1 2 INPUT wc OUTPUT ERROR 0 1 2 A Pipe!
• Email Notification
  • % echo “Message” |
  • mail –s “Here’s your message”
  • [email_address]
• Dates
  • % DATESTRING=`date +%Y%m%d`
  • % echo $DATESTRING
  • 20060125
  • % man date
• FTP the Hard Way
  • ftp –n –u server.wfu.edu <<_FTP_
  • user username password
  • put FILE
  • _FTP_
• FTP with wget
  • wget ftp://user:pass@server.wfu.edu/file
  • wget –r ftp://user:pass@server.wfu.edu/dir/
• FTP with curl
  • curl –T upload-file
  • -u username:password
  • ftp://server.wfu.edu/dir/file
• Searching: grep
  • % grep rayra /etc/passwd
  • % grep –r rayra /etc
  • % grep –r RAYRA /etc
  • % grep –ri RAYRA /etc
  • % grep –rli rayra /etc
• Searching: find
  • % find /home/borwicjh
  • -name ‘*.lis’
  • [all files matching *.lis]
  • % find /home/borwicjh
  • -mtime -1 –name ‘*.lis’
  • [*.lis, if modified within 24h]
  • % man find
• Searching: locate
  • % locate .lis
  • [files with .lis in path]
  • % locate log
  • [also finds “/var/log/messages”]
• Applied Shell Programming
• Make Your Life Easier
  • TAB completion
  • Control+R
  • history
  • cd -
  • Study a UNIX Editor
• pushd/popd
  • % cd /tmp
  • % pushd /var/log
  • /var/log /tmp
  • % cd ..
  • % pwd
  • /var
  • % popd
  • /tmp
• Monitoring processes
  • ps
  • ps –ef
  • ps –u oracle
  • ps –C sshd
  • man ps
• “DOS” Mode Files
  • #!/usr/bin/bash^M
  • FTP transfer in ASCII, or
  • dos2unix infile > outfile
• sqlplus
  • JOB=“ZZZTEST”
  • PARAMS=“ZZZTEST_PARAMS”
  • PARAMS_USER=“BORWICJH”
  • sqlplus $BANNER_USER/$BANNER_PW << _EOF_
  • set serveroutput on
  • set sqlprompt &quot;&quot;
  • EXECUTE WF_SATURN.FZ_Get_Parameters('$JOB', '$PARAMS', '$PARAMS_USER');
  • _EOF_
• sqlplus
  • sqlplus $USER/$PASS @$FILE_SQL
  • $ARG1 $ARG2 $ARG3
  • if [ $? –ne 0 ]
  • then
  • exit 1
  • fi
  • if [ -e /file/sql/should/create ]
  • then
  • […use SQL-created file…]
  • fi
  • Ask Amy Lamy! 
• Passing Arguments
  • % cat > test.sh <<_TEST_
  • echo “Your name is $1 $2 ”
  • _TEST_
  • % chmod +x test.sh
  • % ./test.sh John Borwick ignore-this
  • Your name is John Borwick
• INB Job Submission Template
  • $1 : user ID
  • $2 : password
  • $3 : one-up number
  • $4 : process name
  • $5 : printer name
  • % /path/to/your/script $UI $PW
  • $ONE_UP $JOB $PRNT
• Scheduling Jobs
  • % crontab -l
  • 0 0 * * * daily-midnight-job.sh
  • 0 * * * * hourly-job.sh
  • * * * * * every-minute.sh
  • 0 1 * * 0 1AM-on-sunday.sh
  • % EDITOR=vi crontab –e
  • % man 5 crontab
• It's Over!
• Other Questions?
  • Shells and Shell Scripts
  • bash
  • Running Commands
  • bash and Banner in Practice