Week 7 workshop exercises

1. ping

Try using ping www.google.com to see if the host “www.google.com” is up and responding to ping requests.

Try using ping to see if your own computer is up and responding to ping requests. (Its host name should be localhost.)

(Challenge: Can you write a pipeline which will display just the time taken for each “ping”?)

2. netstat

netstat -t numeric-ports will list all TCP connections between your computer and other computers.

Suppose we want to print just the addresses and ports your computer is connecting to. Unfortunately, the columns are not separated by any single character, so we can’t straightforwadly use cut to extract the fifth column.

Here is an alternative way:

netstat -t --numeric-ports | while read line; do echo $line | cut -d' ' -f 5; done

Can you think of any others? (One possibility is to use the tool awk, which we will not cover in detail, but which can be very handy for parsing data.)

If we add the -p flag to netstat, we will also get a list of the programs making those connections:

netstat -t --numeric-ports -p

Can you construct a pipeline showing what programs on your computer are making connections elsewhere, and how many each one is making?

3. ssh and scp

Try using ssh to connect to the gp05.csse.uwa.edu.au server. Also try connecting to your computer, from itself (ssh localhost), and to demo@test.rebex.net (with the password “password”).

(Challenge exercise: what can you find out about the test.rebex.net host while logged into it?)

Try using scp to copy a file or directory between two computers. (The -r flag to scp will recursively copy a directory – the same as for the cp command.)

Can you copy the file readme.txt form test.rebex.net to your local computer?

(Tip: If you ever want practice on a Unix shell and don’t have easy access to one … try typing ssh new@sdf.org and follow the prompts, to create a free one. Please adhere to their terms and conditions, though!)

4. wget and curl

Try using both of wget and curl to fetch the contents of the CITS4407 home page http://teaching.csse.uwa.edu.au/units/CITS4407/.

Take a look at the man page for wget. Suppose you wanted to copy the whole content of the CITS4407 site – what arguments to wget would you need?

Try using curl to directly fetch the ENROLMENTS-2017 file from the week 4 workshop exercises page (http://teaching.csse.uwa.edu.au/units/CITS4407/workshops/workshop03-exercises.html) and pipe it to wc to count the number of lines.

5. Processing data from the web

Take a look at the listing of transcripts for the US Supreme Court: https://www.supremecourt.gov/oral_arguments/argument_transcript/2018. (Sadly, the Australian High Court website is harder to process, so we will not use it.)

First try using curl and less to display the content of the page, and look for the links to transcripts.

Then construct a bash pipeline which will filter lines containing links to transcript files (hint: these are PDFs), and display those to standard output.

Can you work out a way of processing those lines so as to display just the titles of the links?

(Challenge exercise: work out how to fetch a list of Australian High Court transcripts from http://www.austlii.edu.au/cgi-bin/viewdb/au/cases/cth/HCATrans/.)