Malcolm serves a web browser-based upload form for uploading PCAP files and Zeek logs at https://localhost/upload/ if connecting locally.
Additionally, there is a writable files directory on an SFTP server served on port 8022 (e.g., sftp://USERNAME@localhost:8022/files/ if connecting locally).
The types of files supported are:
- PCAP files (of mime type
application/vnd.tcpdump.pcaporapplication/x-pcapng)- PCAPNG files are partially supported: Zeek is able to process PCAPNG files, but not all of Arkime's packet examination features work correctly
- Zeek logs in archive files (
application/gzip,application/x-gzip,application/x-7z-compressed,application/x-bzip2,application/x-cpio,application/x-lzip,application/x-lzma,application/x-rar-compressed,application/x-tar,application/x-xz, orapplication/zip)- because log fields may differ depending on Zeek's configuration, users are recommended to use Zeek JSON format logs when generating Zeek logs outside of Malcolm to later be uploaded to Malcolm for procesing
- where the Zeek logs are found in the internal directory structure in the archive file does not matter
Files uploaded via these methods are monitored and moved automatically to other directories for processing, generally within 1 minute of completion of the upload.
In addition to being processed for uploading, Malcolm events will be tagged according to the components of the filenames of the PCAP files or Zeek log archives files from which the events were parsed. For example, records created from a PCAP file named ACME_Scada_VLAN10.pcap would be tagged with ACME, Scada, and VLAN10. Tags are extracted from filenames by splitting on the characters , (comma), - (dash), and _ (underscore). These tags are viewable and searchable (via the tags field) in Arkime and OpenSearch Dashboards. This behavior can be changed by modifying the AUTO_TAG environment variable in upload-common.env.
Tags may also be specified manually with the browser-based upload form.