Difference between pages "Birds of a Feather sessions (BoFs)" and "Tutorials/Tutorial: Packets don't lie: how can you use tcpdump/tshark (wireshark) to prove your point."

From LCA2016 Delegate wiki
(Difference between pages)
Jump to: navigation, search
 
 
Line 1: Line 1:
== Birds of a Feather (BoF) Sessions ==
+
===PREREQUISITES===
 +
Please note that this is a tutorial, not a talk.
 +
You should have tcpdump and  wireshark INSTALLED and do some captures BEFORE you come to the tutorial.
 +
To find wireshark packages do something like this:
 +
<pre>
 +
#archlinux
 +
pacman -Ss wireshark tcpdump
 +
#debian
 +
apt-cache search wireshark tcpdump
 +
#fedora
 +
yum search wireshark tcpdump
 +
</pre>
 +
Please install both GUI and CLI packages.
  
Although not an official social event of the Conference, Birds of a Feather - or BoFs as they are known - allow Delegates to meet around a particular topic or interest. BoFs usually occur during lunchtime, or after the main Conference presentations for the day.  
+
Please add the user you are going to run wireshark/tshark as to the 'wireshark' user group.  
 +
<pre>
 +
gpasswd -a james wireshark
 +
</pre>
 +
After this user 'james' will need to log out and log in again!
  
==== Instructions ====
 
Edit the table below to claim a BoF, and provide a way for people to contact you.
 
''Rooms will be added later.'' See also: [[mw:Help:Tables|Table MarkD2.193 up Help]].
 
  
{| class="wikitable" style="text-align:center"
+
If that still DOESN'T work,  you might want to add a capability. Do it ONLY if you are still unable to do capture.
<!-- Table Headers -->
+
<pre>
|+Birds of a Feather (BoF) Sessions
+
setcap cap_net_raw,cap_net_admin=eip /usr/bin/dumpcap
!
+
</pre>
!Monday 1st Feb
+
!Tuesday 2nd Feb
+
!Wednesday 3rd Feb
+
!Thursday 4th Feb
+
!Friday 5th Feb
+
  
|-
+
When finished, have a look around at what files came with the package:
! Early birds
+
<pre>
06:00
+
tcpdump --version
<!-- Mon -->
+
tshark --version
| [[Running BoF]]
+
</pre>
<!-- Tue -->
+
List binaries that came with the packge (example is for archlinux, for rmp-like do 'rpm -ql wireshark', for debian 'dpkg -L wireshark')
| [[Running BoF]]
+
<pre>
<!-- Wed -->
+
pacman -Ql wireshark-cli|grep bin
| [[Running BoF]]
+
wireshark-cli /usr/bin/androiddump
<!-- Thu -->
+
wireshark-cli /usr/bin/capinfos
| [[Running BoF]]
+
wireshark-cli /usr/bin/captype        #same as 'capinfos -t'
<!-- Fri -->
+
wireshark-cli /usr/bin/dftest        #display filter byte-code for debugging
| [[Running BoF]]
+
wireshark-cli /usr/bin/dumpcap        #can write files
|-
+
wireshark-cli /usr/bin/editcap        #snaplenth, or split into multiple based on time, number of packets, adjust time
! Not-as-Early birds
+
wireshark-cli /usr/bin/idl2wrs
07:00
+
wireshark-cli /usr/bin/mergecap
<!-- Mon -->
+
wireshark-cli /usr/bin/randpkt        #creates a legitimate EthernetII packet with the given Type field set
| [[Interval Training BoF]]
+
wireshark-cli /usr/bin/rawshark      #cannot write files, only to standard output
<!-- Tue -->
+
wireshark-cli /usr/bin/reordercap    #part of the functionality of the editcap
| [[Interval Training BoF]]
+
wireshark-cli /usr/bin/text2pcap      #hexdump -> pcap
<!-- Wed -->
+
wireshark-cli /usr/bin/tshark
| [[Interval Training BoF]]
+
wireshark-cli /usr/include/wireshark/epan/dissectors/packet-ypbind.h
<!-- Thu -->
+
wireshark-cli /usr/share/wireshark/radius/dictionary.bintec
| [[Interval Training BoF]]
+
<!-- Fri -->
+
| [[Interval Training BoF]]
+
  
<!-- Keynote Speakers -->
+
</pre>
|- style="background-color: #f2f2f2;text-align:center;"
+
! 09:00
+
| Opening
+
| scope="row" colspan="4" |Keynote Speakers
+
  
|-
+
=Capturing=
! Morning Tea
+
====browsing exercise ====
10:00—10:40
+
# start capturing
<!--Mon-->
+
# navigate your browser to linux.conf.au
|Session
+
# navigate your browser to google.com
<!--Tue-->
+
# navigate your browser to xxxxxxx (your choice)
|Session
+
<!--Wed-->
+
|Session
+
<!--Thu-->
+
|Session
+
<!--Fri-->
+
|Session
+
  
<!-- Regular Schedule -->
+
====have a look at the capture files that you generated====
|- style="background-color: #f2f2f2;text-align:center;"
+
<pre>
|scope="row" colspan="6" | Regular Schedule
+
capinfos -T *.pcap{,ng}
 +
</pre>
 +
==capture interfaces==
 +
<pre>
 +
tcpdump -D
 +
tshark -D
 +
# try with no interface
 +
tshark
 +
</pre>
  
|-
+
==capturing on the CLI==
!Lunch Break
+
12:20—13:20
+
<!--Mon-->
+
|[[Queer BoF]]
+
<!--Tue-->
+
|[[Perl BoF]] D2.193 <p> [[Autonomous Robots BoF]] D.211
+
<!--Wed-->
+
|[[Emacs BoF]] D2.193 <br/> [[Safer Payments BoF]] D.211
+
Exploding kittens bof- student lounge
+
<br/>Cards Against Humanity BoF D.211
+
<!--Thu-->
+
|[[Ladies' Lunch]]
+
[[Debian Lunch @ wool museum restaurant]]
+
<br/>Cards Against Humanity BoF - D.211
+
<!--Fri-->
+
|[[Freedom Conservancy supporters]]
+
  
<!-- Regular Schedule -->
+
tcpdump can autostop after certain number of packets has been captured. tshark has addinionally duration, filesize and number of files autostop conditions.
|- style="background-color: #f2f2f2;text-align:center;"
+
What if we want to have a permanent capture running and keep last N days of the logs?
|scope="row" colspan="6"|Regular Schedule
+
<pre>
 +
tshark -b duration:2 -n -wevery2sec.pcapng & watch -n 1 ls -l
 +
tshark -b duration:2 -n -wevery2sec.pcapng -a files:5  & watch -n1 ls -l
 +
tshark -b duration:3600 -n -weveryHour.pcapng -b files:24
 +
tshark -a filesize:1024 -n -w1MiB.pcapng
 +
</pre>
 +
=expert info=
 +
==GUI==
 +
-r <pcap file>
 +
-J  <jump filter>
 +
          After reading in a capture file using the -r flag, jump to the packet matching the filter (display
 +
          filter syntax).  If no exact match is found the first packet after that is selected.
  
|-
+
        capinfo
!Afternoon Tea
+
        lower bottom corner
15:00—15:40
+
                expert info
<!--Mon-->
+
                file name
|Session
+
                packets, etc
<!--Tue-->
+
        statistics -> protocol hierarchy
|Session
+
        statistics -> HTTP -> packet counter
<!--Wed-->
+
        statistics -> HTTP -> requests
|[[BlueHackers BoF]] D2.193
+
        extract objects
<!--Thu-->
+
        follow TCP stream
|Jobs BoF D.211
+
        coloring rules
<!--Fri-->
+
        Statistics -> IPv4 Statistics ->  Destinations and  Ports
|Session
+
fields
 +
    delta time
 +
   
 +
==CLI==
 +
<pre>
 +
! tshark -q -z ptype,tree
 +
! tshark -q -z io,stat,20,eth -q
 +
! tshark -q -z io,stat,20,http -q
 +
! tshark -q -z io,stat,20,,"BYTES()http" -q
 +
! tshark -q -z http,tree
 +
! tshark -q -z http_req,tree
 +
! tshark -q -z http_srv,tree
 +
create a capture file for icmp
 +
! tshark -q -z icmp,srt
 +
! tshark -q -z io,phs
 +
! tshark -q -z io,stat
 +
! tshark -q -z ip_hosts,tree
 +
! tshark -q -z plen,tree
 +
! tshark -q -z endpoints,eth
 +
! tshark -q -z endpoints,eth,
 +
! tshark -q -z endpoints,ip
 +
! tshark -q -z conv,eth
 +
! tshark -q -z conv,udp
 +
! tshark -q -z conv,tcp
 +
! tshark -q -z conv
 +
! tshark -q -z expert,error -q
 +
! tshark -q -z expert,note -q
 +
</pre>
  
<!-- Regular Schedule -->
+
=DECRYPTING SSL=
|- style="background-color: #f2f2f2;text-align:center;"
+
When a client (for example, a web browser) makes a connection to a web server requiring SSL/TLS encryption - the encrypted channel is setup using a '''symmetric''' session key. This key is a random string generated by the client and then encrypted and transmitted using the servers '''public''' key, known as the ''Pre-master Secret''. Once shared, the client and server use this shared key to encrypt and decrypt traffic.
|scope="row" colspan="5" |Regular Schedule
+
|[[Lightning_talks|LightningTalks]] & Closing
+
  
|-
+
MacOS users can do: launchctl setenv SSLKEYLOGFILE /tmp/SSLKEYLOGFILE.txt; open -a Applications/Firefox.app
!Evening
+
17:20—
+
<!--Mon-->
+
|[[Ingress BoF]] D.192
+
[[Libre Instant Messaging and Social Media BoF|Libre IM & Social BoF]] D2.211
+
<!--Tue-->
+
|[[Keysigning bof|Keysigning BoF]] D2.211
+
<!--Wed-->
+
|[[Kerbal BoF]] D2.193
+
<!--Thu-->
+
|[[Parallelism and Concurrency BoF]]
+
<!--Fri-->
+
|Session
+
  
<!-- Regular Schedule -->
+
====exercise on decyphering SSL====
|- style="background-color: #f2f2f2;text-align:center;"
+
# <pre>
!18:00—
+
SSLKEYLOGFILE=/tmp/SSLKEYLOGFILE.txt firefox &  tail -f /tmp/SSLKEYLOGFILE.txt
<!--Mon-->
+
</pre>
|[[LA AGM]] D.193
+
#Navigate to https://google.com
<!--Tue-->
+
#Do some searches
|[[Professional Delegates Networking Session (PDNS)|PDNS]]
+
#Start capture
<!--Wed-->
+
#Open a new tab and do more searches on google.com
|[[Penguin Dinner]]
+
#try https://facebook.com or some other web site.
<!--Thu-->
+
|[[Speakers' Dinner]]
+
<!--Fri-->
+
|[[EFA Drinks]]
+
|}
+
  
  
Unscheduled sessions:
+
SSLKEYLOGFILE variable  works for firefox, chromium and any program built with NSS library (Network Security Services).
* Exploding kittens
+
{{Template:Navigation}}
+
  
{{BoF}}
+
==display filter==
 +
<pre>
 +
sack
 +
http
 +
        ip.src==1.1.1.1 &&      tcp.analysis.retransmission or tcp.analysis.fast_retransmission
 +
        http.time >= 0.4
 +
        tcp.analysis.rto >= 0.050
 +
        http.request.uri == "https://www.wireshark.org/"
 +
        http.response.code == 500
 +
        tcp.port in {80 443 8080}
 +
        #the above is same as:
 +
        tcp.port == 80 || tcp.port == 443 || tcp.port == 8080
 +
        _ws.expert.severity >= warn
 +
                0x1      ok
 +
                0x100000 comment
 +
                0x200000 chat
 +
                0x400000 note
 +
                0x600000 warn
 +
                0x800000 error
  
[[Category:Events]]
+
tshark -r /var/tmp/aros.pcapng -e frame.number -e ip.src -e ip.dst -Tfields
 +
tshark -r /srv/http/TCP_SACK.cap  -Y frame.number==29 -V
 +
tshark -r TCP_SACK.cap -Y 'frame.number>=10' -Y 'frame.number<=15'
 +
</pre>
 +
==columns==
 +
<pre>
 +
tshark -r http.pcapng -z follow,tcp,hex,1
 +
tshark -e ip.addr -e tcp.window_size -Tfields
 +
tshark -r http.pcapng -z follow,tcp,hex,127.0.0.1:59544,127.0.0.1:80
 +
tshark -r /srv/http/TCP_SACK.cap  -Tfields -e frame.number -e frame.time_epoch -e ip.src -e ip.dst -e tcp.seq -e tcp.len -e tcp.nxtseq -e tcp.ack  -e tcp.analysis.ack_rtt
 +
tshark -r /srv/http/TCP_SACK.cap  -Tfields -e frame.number -e frame.time_epoch -e ip.src -e ip.dst -e tcp.seq -e tcp.len -e tcp.nxtseq -e tcp.ack  -e  tcp.options.sack_le -e tcp.options.sack_re
 +
</pre>
 +
 
 +
=extra=
 +
<pre>
 +
-d tcp.port==8888,http
 +
</pre>

Revision as of 14:34, 3 February 2016

PREREQUISITES

Please note that this is a tutorial, not a talk. You should have tcpdump and wireshark INSTALLED and do some captures BEFORE you come to the tutorial. To find wireshark packages do something like this:

#archlinux
pacman -Ss wireshark tcpdump
#debian
apt-cache search wireshark tcpdump
#fedora
yum search wireshark tcpdump

Please install both GUI and CLI packages.

Please add the user you are going to run wireshark/tshark as to the 'wireshark' user group.

gpasswd -a james wireshark

After this user 'james' will need to log out and log in again!


If that still DOESN'T work, you might want to add a capability. Do it ONLY if you are still unable to do capture.

setcap cap_net_raw,cap_net_admin=eip /usr/bin/dumpcap

When finished, have a look around at what files came with the package:

tcpdump --version
tshark --version

List binaries that came with the packge (example is for archlinux, for rmp-like do 'rpm -ql wireshark', for debian 'dpkg -L wireshark')

pacman -Ql wireshark-cli|grep bin
wireshark-cli /usr/bin/androiddump
wireshark-cli /usr/bin/capinfos
wireshark-cli /usr/bin/captype        #same as 'capinfos -t'
wireshark-cli /usr/bin/dftest         #display filter byte-code for debugging
wireshark-cli /usr/bin/dumpcap        #can write files
wireshark-cli /usr/bin/editcap        #snaplenth, or split into multiple based on time, number of packets, adjust time
wireshark-cli /usr/bin/idl2wrs
wireshark-cli /usr/bin/mergecap
wireshark-cli /usr/bin/randpkt        #creates a legitimate EthernetII packet with the given Type field set
wireshark-cli /usr/bin/rawshark       #cannot write files, only to standard output
wireshark-cli /usr/bin/reordercap     #part of the functionality of the editcap
wireshark-cli /usr/bin/text2pcap      #hexdump -> pcap
wireshark-cli /usr/bin/tshark
wireshark-cli /usr/include/wireshark/epan/dissectors/packet-ypbind.h
wireshark-cli /usr/share/wireshark/radius/dictionary.bintec

Capturing

browsing exercise

  1. start capturing
  2. navigate your browser to linux.conf.au
  3. navigate your browser to google.com
  4. navigate your browser to xxxxxxx (your choice)

have a look at the capture files that you generated

capinfos -T *.pcap{,ng}

capture interfaces

tcpdump -D
tshark -D
# try with no interface
tshark

capturing on the CLI

tcpdump can autostop after certain number of packets has been captured. tshark has addinionally duration, filesize and number of files autostop conditions. What if we want to have a permanent capture running and keep last N days of the logs?

tshark -b duration:2 -n -wevery2sec.pcapng & watch -n 1 ls -l
tshark -b duration:2 -n -wevery2sec.pcapng -a files:5  & watch -n1 ls -l
tshark -b duration:3600 -n -weveryHour.pcapng -b files:24
tshark -a filesize:1024 -n -w1MiB.pcapng

expert info

GUI

-r <pcap file> -J <jump filter>

          After reading in a capture file using the -r flag, jump to the packet matching the filter (display
          filter syntax).  If no exact match is found the first packet after that is selected.
       capinfo
       lower bottom corner
               expert info
               file name
               packets, etc
       statistics -> protocol hierarchy
       statistics -> HTTP -> packet counter
       statistics -> HTTP -> requests
       extract objects
       follow TCP stream
       coloring rules
       Statistics -> IPv4 Statistics ->  Destinations and  Ports

fields

   delta time
   

CLI

! tshark -q -z ptype,tree
! tshark -q -z io,stat,20,eth -q
! tshark -q -z io,stat,20,http -q
! tshark -q -z io,stat,20,,"BYTES()http" -q
! tshark -q -z http,tree
! tshark -q -z http_req,tree
! tshark -q -z http_srv,tree
create a capture file for icmp
! tshark -q -z icmp,srt
! tshark -q -z io,phs
! tshark -q -z io,stat
! tshark -q -z ip_hosts,tree
! tshark -q -z plen,tree
! tshark -q -z endpoints,eth
! tshark -q -z endpoints,eth,
! tshark -q -z endpoints,ip
! tshark -q -z conv,eth
! tshark -q -z conv,udp
! tshark -q -z conv,tcp
! tshark -q -z conv
! tshark -q -z expert,error -q
! tshark -q -z expert,note -q

DECRYPTING SSL

When a client (for example, a web browser) makes a connection to a web server requiring SSL/TLS encryption - the encrypted channel is setup using a symmetric session key. This key is a random string generated by the client and then encrypted and transmitted using the servers public key, known as the Pre-master Secret. Once shared, the client and server use this shared key to encrypt and decrypt traffic.

MacOS users can do: launchctl setenv SSLKEYLOGFILE /tmp/SSLKEYLOGFILE.txt; open -a Applications/Firefox.app

exercise on decyphering SSL

SSLKEYLOGFILE=/tmp/SSLKEYLOGFILE.txt firefox & tail -f /tmp/SSLKEYLOGFILE.txt

  1. Navigate to https://google.com
  2. Do some searches
  3. Start capture
  4. Open a new tab and do more searches on google.com
  5. try https://facebook.com or some other web site.


SSLKEYLOGFILE variable works for firefox, chromium and any program built with NSS library (Network Security Services).

display filter

sack 
http
        ip.src==1.1.1.1 &&      tcp.analysis.retransmission or tcp.analysis.fast_retransmission
        http.time >= 0.4
        tcp.analysis.rto >= 0.050
        http.request.uri == "https://www.wireshark.org/"
        http.response.code == 500
        tcp.port in {80 443 8080}
        #the above is same as:
        tcp.port == 80 || tcp.port == 443 || tcp.port == 8080
        _ws.expert.severity >= warn
                0x1      ok
                0x100000 comment
                0x200000 chat
                0x400000 note
                0x600000 warn
                0x800000 error

tshark -r /var/tmp/aros.pcapng -e frame.number -e ip.src -e ip.dst -Tfields
tshark -r /srv/http/TCP_SACK.cap   -Y frame.number==29 -V
tshark -r TCP_SACK.cap -Y 'frame.number>=10' -Y 'frame.number<=15'

columns

 tshark -r http.pcapng -z follow,tcp,hex,1
 tshark -e ip.addr -e tcp.window_size -Tfields
 tshark -r http.pcapng -z follow,tcp,hex,127.0.0.1:59544,127.0.0.1:80
 tshark -r /srv/http/TCP_SACK.cap   -Tfields -e frame.number -e frame.time_epoch -e ip.src -e ip.dst -e tcp.seq -e tcp.len -e tcp.nxtseq -e tcp.ack  -e tcp.analysis.ack_rtt
 tshark -r /srv/http/TCP_SACK.cap   -Tfields -e frame.number -e frame.time_epoch -e ip.src -e ip.dst -e tcp.seq -e tcp.len -e tcp.nxtseq -e tcp.ack  -e  tcp.options.sack_le -e tcp.options.sack_re

extra

-d tcp.port==8888,http