Scripting 101

<datruth29> hey

<datruth29> is the a session today?

<hemanth> yes

<hemanth> 20 mins from now

<datruth29> okay, just wanted to make sure :)

<hemanth> :)

<datruth29> ohhhhhhhhhhh that's right, i thought it was at 11:00 xDD

<hemanth> using the class script ;)?

<datruth29> i just logged on through p2pu, i'm going to start it up now

<datruth29> oh wait

<datruth29> are you talking about limpid?

<datruth29> or another script?

<hemanth> no i had made on script that would tell when the next class is

<hemanth> brb

<datruth29> okay

<datruth29> brb

<hemanth> back

* SourRust (473b12d5@gateway/web/freenode/ip.71.59.18.213) has joined #p2pu-webcraft/scripting-101

<hemanth> SourRust, hey :)

<SourRust> hey

<hemanth> Lets wait for 3 more mins?

<SourRust> k

<hemanth> Pablo cant make it to class today it seems

<SourRust> are we learning ruby, perl redirects because I was looking at the chat log and it look like u only taught error handling

<hemanth> yes, but the std{in.out,err} can be handled with that

<SourRust> k

<hemanth> ./my.rb 2>&1 > log

<hemanth> normally could be done

<hemanth> but the same, when we catch errors could be written to a file

<hemanth> with a File object

<hemanth> so either ways the intention is redirection

<hemanth> looking for something particular?

<SourRust> quick example or both i guess

<SourRust> of*

<hemanth> If you want to do it withing the script dircetly with the std{out,err} you could do stdout.reopen("out.txt", "w")

<hemanth> Example 1: ./my.rb 2>&1 > log

<hemanth> ^ Where nothing is handled by the script, but by the shell

<hemanth> Example 2:

<hemanth> stdout.reopen("out.txt", "w")

<hemanth> stderr.reopen("err.log", "w")

<hemanth> warn 'this goes to err stderr'

<hemanth> puts 'this goes to err'

<hemanth> That was with ruby along with, the last class try rescue block

<SourRust> k thx

<hemanth> With Perl, the first method and the last class try catch works, along with it..

<hemanth> open (STDOUT, ">outfile") or die "Can't open STDOUT: $!";

<hemanth> open (STDERR, ">>err.log") or die "Can't open STDERR: $!";

<hemanth> Fine :) ?

<SourRust> yup

<hemanth> >& also apply the std > >> >& applies

* hemanth has changed the topic to: Pipes and Filters BASH

<hemanth> today we are dealing with something that is very interesting and an integral part of the shell

<hemanth> So, why do one use pipe in day to day life?

<hemanth> note, not the programming pipe

<hemanth> just the normal pipe

<hemanth> ...

<SourRust> to get things to and from places

<hemanth> yes :) one of the most useful use of pipes

<hemanth> "A set of processes chained by their standard streams, so that the output of each process (stdout) feeds directly as input (stdin) to the next one." => Pipes

<hemanth> lets see how to use them

<hemanth> So, in BASH we have used few separators like ; & ||

<hemanth> between commands

<hemanth> logical and ,or also

<hemanth> | is called as the pipe symbol

<hemanth> we can run many commands in a single line like

<hemanth> cmd1;cmd2;cmd3

<hemanth> thats runs in a seq

<hemanth> i guess few you would have done something like cmd&

<hemanth> have you?

<SourRust> i haven't

<datruth29> doesn't that run the command in the background or something?

<datruth29> i forgot o.

<hemanth> cmd& will put it to bg as background process.

<hemanth> datruth29, yes correct

<hemanth> When one have a log data crunching to do and he doesn want to wait till it runs, but he just want the o/p

<hemanth> he can put it to bg

<hemanth> 'jobs' is a bulletin in bash

<hemanth> just type jobs and hit enter

<hemanth> what do u see?

* hemanth paste the chat logs so far to Lakshmi

<hemanth> jobs => Lists the active jobs. JOBSPEC restricts output to that job. Without options, the status of all active jobs is displayed.

<datruth29> nothing right now

<hemanth> ok :)

<datruth29> but i bet if i run a job in the background, i would see it with that command

<datruth29> :P

<hemanth> yes

<hemanth> indeed :)

<hemanth> when u say cmd&

<hemanth> you will see

<hemanth> [1] 1256

<hemanth> then cmd1&

<hemanth> you will see something like [2] 1356

<Lakshmi> Thanks. Sorry I am late

<hemanth> wild guess what [number] number might be?

<hemanth> Lakshmi, its ok :)

<hemanth> no guesses ?

<datruth29> the order the jobs were called in?

<hemanth> almost close, but what is the second number

<hemanth> [number] might be consider as order in which they are called, but what might be the other number

<datruth29> i have no idea

<datruth29> the process id? o.o

<hemanth> bingo

* hemanth sends chocolates to datruth29

<hemanth> job number ([1]) and a process identification number

<datruth29> chocolate @_@ yum

<hemanth> :)

<hemanth> but why does the shell output them?

<hemanth> there must be some reason right

<hemanth> and the reason, is....

<hemanth> we can send each process to bg and fg on our choice

<hemanth> bg => background and fg=>foreground

<hemanth> Move jobs to the background, using 'bg'

* Lakshmi has quit (Quit: Page closed)

<hemanth> bg places the jobs identified by each JOB_SPEC in the background, as if they had been started with '&'

<hemanth> 'fg' does the reverse

<hemanth> fine?

<SourRust> yup

<hemanth> now in the pipe symbol, in your keyboards, do you all notice that its divided [[ a gap between them ]] ?

* hemanth magnifies it

<hemanth> |

<hemanth> |

<hemanth> notice that ?

<Lakshmi> Yes

<hemanth> why could that be so?

* hemanth don't think i'm doing some hardware course, but that is very meaningful symbol

* hemanth gives a clue : cmd1 | cmd2

* hemanth another clue, the answer is there in whatever i spoke till now

<hemanth> ??

<Lakshmi> The output of the first Cmd is input to the second one

<hemanth> yup :) the cmd1 output goes through the pipe to cmd2

* hemanth sends pipes to Lakshmi | | | | ...

<Lakshmi> :)

<hemanth> ls -lt | head

<hemanth> try that in the terminal

<SourRust> oh... I had to look at a different keyboard to see because my macbook doesn't have a split between the pipe symbol

<hemanth> SourRust, i was expecting this from a Mac boy

<SourRust> o_O

<hemanth> SourRust, did u think of apple jobs for jobs command? ;)

<hemanth> SourRust, at the end of the day mac's base is free BSD

<SourRust> ik

* hemanth sends apples to SourRust from now on ;)

<SourRust> but yea my keyboard does have the split so I was confuse for a second

<hemanth> So, what did ls -lt | head display

<SourRust> ls of folders/file with info attached to them

<hemanth> ls -lt | head

<SourRust> drwx------+ 6 Jeremy staff 204 Oct 16 03:06 Desktop <- but more

<hemanth> yes it displays the 10 newest files in the current directory.

<hemanth> :)

<hemanth> echo Hello HaCkErs | tr [:lower:] [:upper:]

<hemanth> try ^

<SourRust> all caps now

<hemanth> yup

<hemanth> echo Hello HaCkErs | tr [:upper:] [:lower:]

<Lakshmi> Yes

<hemanth> echo Hello | tr [a-z] [A-Z] can also be done

<hemanth> du | sort -nr

<hemanth> space consumption in sorted order

<Lakshmi> Yes

<hemanth> tr -dc A-Za-z0-9_ < /dev/urandom | head -c12 | xargs

<hemanth> my fav ^

<hemanth> create random passwords

<hemanth> here in these examples the second part of the pipe are called filters

<hemanth> as they do some filtering stuff for us

<hemanth> to make things easier and better

<hemanth> sort

<hemanth> uniq

<hemanth> grep

<hemanth> fmt

<hemanth> pr

<hemanth> head

<hemanth> tail

<hemanth> tr

<hemanth> sed

<hemanth> awk

<hemanth> are on of the most commonly used once

<hemanth> queries?

<SourRust> nope

<hemanth> So, pipes and filters are very useful in the shell

<hemanth> now, the challenge for you guys is

<hemanth> to do pipe and filter simulation with py,rb or pl

<hemanth> like the script takes in two or more commands

<hemanth> and does the piping

<hemanth> and also, you need to pick on of the filter and simulate it

<hemanth> say grep

<hemanth> grep => print lines matching a pattern

<hemanth> So, the choice is your; well and good if you can make them in all py,rb and pl

<hemanth> NOTE: Note using system calls

<hemanth> clear?

<Lakshmi> Ok

<hemanth> So, if no more doubts, this ends the pipes and filter with BASH :)

<hemanth> all clear?

<hemanth> SourRust, datruth29

<SourRust> yes pretty clear

<Lakshmi> Yes

<hemanth> do ping with a thread in the course with your doubts

<hemanth> as it will help others also

<datruth29> sorry, stepped away for a bit

<datruth29> but yes

<datruth29> pipes and filters

<datruth29> super powerful

<datruth29> and awesome

<datruth29> :D

<hemanth> :)

<hemanth> Just enjoy its power all

<hemanth> cya tomo

<SourRust> cya

<datruth29> later!

<hemanth> until then happy hacking

<Lakshmi> Cya thanks

<hemanth> logs will be up in mins

Course Material

Chat log Class 9