diff --git a/log-formats.rst b/log-formats.rst index a88417de4..2d5e41d40 100644 --- a/log-formats.rst +++ b/log-formats.rst @@ -108,10 +108,10 @@ We use the :program:`cat` command to show the contents of each log. #unset_field - #path conn #open 2020-06-05-14-48-32 - #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p proto service duration orig_bytes resp_bytes conn_state local_orig local_resp missed_bytes history orig_pkts orig_ip_bytes resp_pkts resp_ip_bytes tunnel_parents - #types time string addr port addr port enum string interval count count string bool bool count string count count count count set[string] - 1591367999.305988 CazOhH2qDUiJTWMCY 192.168.4.76 36844 192.168.4.1 53 udp dns 0.066852 62 141 SF - -0 Dd 2 118 2 197 - - 1591367999.430166 CLqEx41jYPOdfHF586 192.168.4.76 46378 31.3.245.133 80 tcp http 0.254115 77 295 SF - -0 ShADadFf 6 397 4 511 - + #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p proto service duration orig_bytes resp_bytes conn_state local_orig local_resp missed_bytes history orig_pkts orig_ip_bytes resp_pkts resp_ip_bytes tunnel_parents ip_proto + #types time string addr port addr port enum string interval count count string bool bool count string count count count count set[string] count + 1591367999.305988 CazOhH2qDUiJTWMCY 192.168.4.76 36844 192.168.4.1 53 udp dns 0.066852 62 141 SF - -0 Dd 2 118 2 197 - 17 + 1591367999.430166 CLqEx41jYPOdfHF586 192.168.4.76 46378 31.3.245.133 80 tcp http 0.254115 77 295 SF - -0 ShADadFf 6 397 4 511 - 6 #close 2020-06-05-14-48-32 Next we look at Zeek’s :file:`dns.log`. @@ -434,11 +434,17 @@ First we look at :file:`packet_filter.log`. :: {"ts":1591368442.854585,"node":"zeek","filter":"ip or not ip","init":true,"success":true} + +Next we look at :file:`conn.log` and :file:`dns.log`: + +.. code-block:: console + zeek@zeek:~/zeek-test/json$ cat conn.log - {"ts":1591367999.305988,"uid":"CMdzit1AMNsmfAIiQc","id.orig_h":"192.168.4.76","id.orig_p":36844,"id.resp_h":"192.168.4.1","id.resp_p":53,"proto":"udp","service":"dns","duration":0.06685185432434082,"orig_bytes":62,"resp_bytes":141,"conn_state":"SF","missed_bytes":0,"history":"Dd","orig_pkts":2,"orig_ip_bytes":118,"resp_pkts":2,"resp_ip_bytes":197} - {"ts":1591367999.430166,"uid":"C5bLoe2Mvxqhawzqqd","id.orig_h":"192.168.4.76","id.orig_p":46378,"id.resp_h":"31.3.245.133","id.resp_p":80,"proto":"tcp","service":"http","duration":0.25411510467529297,"orig_bytes":77,"resp_bytes":295,"conn_state":"SF","missed_bytes":0,"history":"ShADadFf","orig_pkts":6,"orig_ip_bytes":397,"resp_pkts":4,"resp_ip_bytes":511} -Next we look at :file:`dns.log`. +:: + + {"ts":1591367999.305988,"uid":"CMdzit1AMNsmfAIiQc","id.orig_h":"192.168.4.76","id.orig_p":36844,"id.resp_h":"192.168.4.1","id.resp_p":53,"proto":"udp","service":"dns","duration":0.06685185432434082,"orig_bytes":62,"resp_bytes":141,"conn_state":"SF","missed_bytes":0,"history":"Dd","orig_pkts":2,"orig_ip_bytes":118,"resp_pkts":2,"resp_ip_bytes":197,"ip_proto":17} + {"ts":1591367999.430166,"uid":"C5bLoe2Mvxqhawzqqd","id.orig_h":"192.168.4.76","id.orig_p":46378,"id.resp_h":"31.3.245.133","id.resp_p":80,"proto":"tcp","service":"http","duration":0.25411510467529297,"orig_bytes":77,"resp_bytes":295,"conn_state":"SF","missed_bytes":0,"history":"ShADadFf","orig_pkts":6,"orig_ip_bytes":397,"resp_pkts":4,"resp_ip_bytes":511,"ip_proto":6} .. code-block:: console diff --git a/logs/conn.rst b/logs/conn.rst index 302eb2448..692e5b3e0 100644 --- a/logs/conn.rst +++ b/logs/conn.rst @@ -34,9 +34,9 @@ stored traffic. We use the :program:`jq` utility to review the contents. :: - {"ts":1591367999.305988,"uid":"CMdzit1AMNsmfAIiQc","id.orig_h":"192.168.4.76","id.orig_p":36844,"id.resp_h":"192.168.4.1","id.resp_p":53,"proto":"udp","service":"dns","duration":0.06685185432434082,"orig_bytes":62,"resp_bytes":141,"conn_state":"SF","missed_bytes":0,"history":"Dd","orig_pkts":2,"orig_ip_bytes":118,"resp_pkts":2,"resp_ip_bytes":197} + {"ts":1591367999.305988,"uid":"CMdzit1AMNsmfAIiQc","id.orig_h":"192.168.4.76","id.orig_p":36844,"id.resp_h":"192.168.4.1","id.resp_p":53,"proto":"udp","service":"dns","duration":0.06685185432434082,"orig_bytes":62,"resp_bytes":141,"conn_state":"SF","missed_bytes":0,"history":"Dd","orig_pkts":2,"orig_ip_bytes":118,"resp_pkts":2,"resp_ip_bytes":197,"ip_proto":17} - {"ts":1591367999.430166,"uid":"C5bLoe2Mvxqhawzqqd","id.orig_h":"192.168.4.76","id.orig_p":46378,"id.resp_h":"31.3.245.133","id.resp_p":80,"proto":"tcp","service":"http","duration":0.25411510467529297,"orig_bytes":77,"resp_bytes":295,"conn_state":"SF","missed_bytes":0,"history":"ShADadFf","orig_pkts":6,"orig_ip_bytes":397,"resp_pkts":4,"resp_ip_bytes":511} + {"ts":1591367999.430166,"uid":"C5bLoe2Mvxqhawzqqd","id.orig_h":"192.168.4.76","id.orig_p":46378,"id.resp_h":"31.3.245.133","id.resp_p":80,"proto":"tcp","service":"http","duration":0.25411510467529297,"orig_bytes":77,"resp_bytes":295,"conn_state":"SF","missed_bytes":0,"history":"ShADadFf","orig_pkts":6,"orig_ip_bytes":397,"resp_pkts":4,"resp_ip_bytes":511,"ip_proto":6} Alternatively, we could see each field printed on its own line: @@ -64,7 +64,8 @@ Alternatively, we could see each field printed on its own line: "orig_pkts": 2, "orig_ip_bytes": 118, "resp_pkts": 2, - "resp_ip_bytes": 197 + "resp_ip_bytes": 197, + "ip_proto": 17 } { "ts": 1591367999.430166, @@ -84,7 +85,8 @@ Alternatively, we could see each field printed on its own line: "orig_pkts": 6, "orig_ip_bytes": 397, "resp_pkts": 4, - "resp_ip_bytes": 511 + "resp_ip_bytes": 511, + "ip_proto": 6 } What an analyst derives from any log is a function of the questions that he or @@ -119,7 +121,8 @@ the following: "orig_pkts": 6, "orig_ip_bytes": 397, "resp_pkts": 4, - "resp_ip_bytes": 511 + "resp_ip_bytes": 511, + "ip_proto": 6 } For the second log, ``192.168.4.76`` talked to ``31.3.245.133``. @@ -264,6 +267,10 @@ In the highlighted output, we see that :program:`tshark` notes 77 bytes of data carried by TCP from ``192.168.4.76``. I highlighted what that data was, beginning with a GET request. +The ``orig_pkts`` and ``resp_pkts`` fields report the number of IP packets +transferred in the respective directions. The ``orig_ip_bytes`` and +``resp_ip_bytes`` indicate the total IP packet-level byte counts, respectively. + Another way to look at this TCP segment is to dump the hex contents using a different :program:`tshark` option, as shown below. @@ -339,7 +346,8 @@ reference. "orig_pkts": 2, "orig_ip_bytes": 118, "resp_pkts": 2, - "resp_ip_bytes": 197 + "resp_ip_bytes": 197, + "ip_proto": 17 } For the first entry, ``192.168.4.76`` talked to ``192.168.4.1``. @@ -370,6 +378,52 @@ the conversations as “normal establishment and termination” of the Similarly, the ``history`` field is simply ``Dd``, indicating that each party to the conversation sent data to the other. +The ``ip_proto`` Field +====================== + +.. versionadded:: 7.1 + +The numeric ``ip_proto`` field reports the `IP protocol number +`_ of +the connection. It relates to the ``proto`` field, but while the former +represents a :zeek:type:`transport_proto` value that exclusively covers +*transport* protocols Zeek knows how to parse (and ties into Zeek's +:zeek:type:`port` type), the ``ip_proto`` field is always present, including for +non-transport IP packet flows such as IGMP or OSPF. For example, an OSPF flow +might look as follows: + +:: + + { + "ts": 1098361214.420459, + "uid": "C9EV8R4fN8bfSj08f", + "id.orig_h": "192.168.170.2", + "id.orig_p": 0, + "id.resp_h": "224.0.0.6", + "id.resp_p": 0, + "proto": "unknown_transport", + "duration": 6.437546968460083, + "orig_bytes": 0, + "resp_bytes": 0, + "conn_state": "OTH", + "local_orig": true, + "local_resp": false, + "missed_bytes": 0, + "orig_pkts": 4, + "orig_ip_bytes": 768, + "resp_pkts": 0, + "resp_ip_bytes": 0, + "ip_proto": 89 + } + +You can adapt this feature in several ways. Load the +:doc:`/scripts/policy/protocols/conn/ip-proto-name-logging.zeek` policy script +to add an ``ip_proto_name`` column with a string version of the ``ip_proto`` +value. Also, you may disable the whole feature by loading the +:doc:`/scripts/policy/protocols/conn/disable-unknown-ip-proto-support.zeek` +script, returning conn.log to its pre-7.1 state. Zeek's :ref:`logging framework +` supports additional customizations. + The ``uid`` and Other Fields ============================ diff --git a/logs/dhcp.rst b/logs/dhcp.rst index ae27b91d2..ab92971d7 100644 --- a/logs/dhcp.rst +++ b/logs/dhcp.rst @@ -346,7 +346,7 @@ source IP address, sending its search to the local network for a DHCP server. "orig_ip_bytes": 1323, "resp_pkts": 0, "resp_ip_bytes": 0, - "sensorname": "so16-enp0s8" + "ip_proto": 17 } Notice that Zeek has tracked 4 “orig packets” here, which does not strictly @@ -384,7 +384,7 @@ client, and ``192.168.4.1``, the DHCP server. "orig_ip_bytes": 0, "resp_pkts": 2, "resp_ip_bytes": 660, - "sensorname": "so16-enp0s8" + "ip_proto": 17 } Here the count of 2 ``resp_pkts`` is correct. diff --git a/logs/dpd.rst b/logs/dpd.rst index d63d9202a..3ee60609d 100644 --- a/logs/dpd.rst +++ b/logs/dpd.rst @@ -248,7 +248,8 @@ Here is the :file:`conn.log` that Zeek generated for this activity: "orig_pkts": 6, "orig_ip_bytes": 776, "resp_pkts": 5, - "resp_ip_bytes": 675 + "resp_ip_bytes": 675, + "ip_proto": 6 } The :file:`conn.log` entry is fairly normal. diff --git a/logs/ftp.rst b/logs/ftp.rst index a658d21af..5f6801e05 100644 --- a/logs/ftp.rst +++ b/logs/ftp.rst @@ -63,6 +63,7 @@ for easier viewing. "orig_ip_bytes": 1232, "resp_pkts": 17, "resp_ip_bytes": 1343, + "ip_proto": 6, "community_id": "1:lEESxqaSVYqFZvWNb4OccTa9sTs=" } { @@ -296,6 +297,7 @@ IP addresses too. "orig_ip_bytes": 216, "resp_pkts": 4, "resp_ip_bytes": 290, + "ip_proto": 6, "community_id": "1:DNwvGR6Ots6pISvsdXBUIaG8y3Q=" } diff --git a/logs/irc.rst b/logs/irc.rst index 35aa4e6f2..20627acb5 100644 --- a/logs/irc.rst +++ b/logs/irc.rst @@ -118,7 +118,8 @@ Zeek generated the following :file:`conn.log` entry for the example traffic. "orig_pkts": 41, "orig_ip_bytes": 1963, "resp_pkts": 185, - "resp_ip_bytes": 246742 + "resp_ip_bytes": 246742, + "ip_proto": 6 } We see that Zeek correctly identified this traffic as IRC. We can expect to see diff --git a/logs/pe.rst b/logs/pe.rst index ce04a0375..9c4f4090a 100644 --- a/logs/pe.rst +++ b/logs/pe.rst @@ -14,7 +14,7 @@ Starting with :file:`conn.log` ============================== This example starts with the :file:`conn.log`. It’s not strictly necessary to -explain the :file:`pe.log`, although I wanted to include a very recent example +explain the :file:`pe.log`, although I wanted to include a recent example of a modern application conducting activities via HTTP. .. literal-emph:: @@ -40,6 +40,7 @@ of a modern application conducting activities via HTTP. "orig_ip_bytes": 29041, "resp_pkts": 1367, "resp_ip_bytes": 2030409, + "ip_proto": 6 } This example shows a host, ``192.168.4.152``, conducting a HTTP session with diff --git a/logs/smb.rst b/logs/smb.rst index 519b9d286..198af0a9a 100644 --- a/logs/smb.rst +++ b/logs/smb.rst @@ -118,7 +118,8 @@ traffic. "orig_pkts": 78, "orig_ip_bytes": 19403, "resp_pkts": 77, - "resp_ip_bytes": 16812 + "resp_ip_bytes": 16812, + "ip_proto": 6 } We see that ``192.168.10.31`` initiated a connection to ``192.168.10.10``. The @@ -376,7 +377,8 @@ The :file:`conn.log` has two entries: "orig_pkts": 66, "orig_ip_bytes": 816703, "resp_pkts": 91, - "resp_ip_bytes": 15309 + "resp_ip_bytes": 15309, + "ip_proto": 6 } { "ts": 1507565425.183882, @@ -392,7 +394,8 @@ The :file:`conn.log` has two entries: "orig_pkts": 1, "orig_ip_bytes": 207, "resp_pkts": 0, - "resp_ip_bytes": 0 + "resp_ip_bytes": 0, + "ip_proto": 17 } The first entry shows a connection initiated by ``192.168.10.31`` to @@ -790,7 +793,8 @@ The :file:`conn.log` for this case has only one entry: "orig_pkts": 74, "orig_ip_bytes": 8734, "resp_pkts": 575, - "resp_ip_bytes": 835740 + "resp_ip_bytes": 835740, + "ip_proto": 6 } We see the same pattern: ``192.168.10.31`` initiated a connection to @@ -1028,7 +1032,8 @@ Looking at the :file:`conn.log`, we see one entry: "orig_pkts": 11, "orig_ip_bytes": 1595, "resp_pkts": 9, - "resp_ip_bytes": 1397 + "resp_ip_bytes": 1397, + "ip_proto": 6 } We see ``192.168.10.31`` initiated a connection to ``192.168.10.30``, port 445 diff --git a/logs/tunnel.rst b/logs/tunnel.rst index 530da50cf..5de6e2070 100644 --- a/logs/tunnel.rst +++ b/logs/tunnel.rst @@ -146,7 +146,8 @@ The :file:`conn.log` for this traffic contains the following: "orig_pkts": 1, "orig_ip_bytes": 89, "resp_pkts": 1, - "resp_ip_bytes": 137 + "resp_ip_bytes": 137, + "ip_proto": 17 } This first :file:`conn.log` entry addresses frames 2 and 3 in the original packet @@ -172,7 +173,8 @@ shortly. "orig_pkts": 1, "orig_ip_bytes": 89, "resp_pkts": 0, - "resp_ip_bytes": 0 + "resp_ip_bytes": 0, + "ip_proto": 17 } This second :file:`conn.log` entry refers to frame 1 in the packet capture. @@ -196,7 +198,8 @@ Note the ``uid`` field. It will appear again shortly as well. "resp_ip_bytes": 0, "tunnel_parents": [ **"CO9T0A3FPac5ig4hud"** - ] + ], + "ip_proto": 1 } Here Zeek has created a new :file:`conn.log` entry for the ICMPv6 traffic