Introduction to Network Traffic Analysis
Network Traffic Analysis (NTA) is the practice of monitoring, capturing, and analyzing network traffic to detect security threats, troubleshoot performance issues, and investigate incidents. Every digital interaction generates network traffic — and this traffic contains a wealth of information about what's happening in an environment.
For security professionals, network traffic analysis is indispensable. Attackers must communicate across networks to exfiltrate data, establish command and control, or move laterally. By analyzing network traffic, defenders can detect these activities even when endpoint defenses fail. NTA provides visibility that no single security control can offer.
1. The Network Analysis Stack
Network traffic analysis spans multiple layers of capability, from raw packet capture to behavioral analytics.
2. Packet Capture Fundamentals
Packet capture (PCAP) provides the most detailed view of network traffic — every byte of every packet.
Essential Capture Tools
- tcpdump: Command-line packet capture for Linux/Unix
- Wireshark: GUI packet analyzer with protocol decoding
- tshark: Command-line version of Wireshark
- WinPcap/Npcap: Windows packet capture drivers
# Basic tcpdump capture tcpdump -i eth0 -w capture.pcap # Capture with filters tcpdump -i eth0 host 192.168.1.100 -w capture.pcap tcpdump -i eth0 port 443 -w https_traffic.pcap tcpdump -i eth0 tcp and not port 22 -w traffic_except_ssh.pcap # Capture with rotation (100MB files, max 50 files) tcpdump -i eth0 -w capture_%Y%m%d_%H%M%S.pcap -C 100 -W 50 -G 3600 # Read capture file tcpdump -r capture.pcap tcpdump -r capture.pcap 'tcp port 80'
Display Filters vs Capture Filters
- Capture Filters (BPF): Applied during capture, reduce file size (tcpdump syntax)
- Display Filters: Applied during analysis, more flexible (Wireshark syntax)
# Wireshark Display Filters ip.addr == 192.168.1.1 # Filter by IP address tcp.port == 443 # Filter by TCP port http.request.method == "GET" # HTTP GET requests dns.qry.name contains "malware" # DNS queries containing "malware" tcp.flags.syn == 1 && tcp.flags.ack == 0 # SYN packets only frame.time_relative > 10 # Packets after 10 seconds
3. Flow Data and NetFlow
Flow data provides summary information about network conversations without capturing full packets — ideal for long-term storage and analysis.
- Source/Destination IP addresses
- Source/Destination ports
- Protocol (TCP, UDP, ICMP)
- Packet and byte counts
- Start and end timestamps
- TCP flags (SYN, ACK, RST, FIN)
# NetFlow export configuration (Cisco) interface GigabitEthernet0/0 ip flow ingress ip flow egress # NetFlow collector configuration ip flow-export destination 192.168.1.100 2055 ip flow-export version 9 # View NetFlow statistics show ip cache flow
4. Protocol Analysis Deep Dive
4.1 TCP Analysis
TCP is the dominant transport protocol. Understanding TCP flags and behavior is essential for detecting malicious activity.
TCP Flags and Security Relevance
- SYN: Connection initiation — high volume may indicate SYN flood attack
- ACK: Acknowledgment — normal traffic
- RST: Reset — unusual resets may indicate scanning or firewalls
- FIN: Graceful termination
- PSH: Push data immediately — may indicate data exfiltration
- URG: Urgent data — rarely used, may indicate malware
4.2 DNS Analysis
DNS is frequently abused by attackers for command and control, data exfiltration, and malware delivery.
# Detect DNS tunneling (long queries, high frequency) tshark -r capture.pcap -Y "dns.qry.name matches \".*\.(tk|ml|ga|cf|xyz)\"" # Detect DNS query with suspicious length (tunneling) tshark -r capture.pcap -Y "dns.qry.name len > 50" # Detect DNS responses from unusual TLDs tshark -r capture.pcap -Y "dns.resp.name matches \"\.(top|win|bid|loan)\"" # Detect DNS over HTTPS (DoH) - port 443 with SNI of DoH providers tshark -r capture.pcap -Y "tls.handshake.extensions_server_name contains \"cloudflare-dns.com\""
4.3 HTTP/HTTPS Analysis
Web traffic analysis reveals malicious activity, data exfiltration, and user behavior.
# Extract HTTP requests tshark -r capture.pcap -Y "http.request" -T fields -e http.request.method -e http.request.uri # Detect user agents associated with malware tshark -r capture.pcap -Y "http.user_agent contains \"python-requests\"" # Detect POST requests with large payloads (exfiltration) tshark -r capture.pcap -Y "http.request.method == POST && http.content_length > 100000" # Extract files from HTTP traffic tshark -r capture.pcap --export-objects http,extracted_files
5. Intrusion Detection and Prevention Systems
IDS/IPS systems analyze traffic and generate alerts based on signatures or behavioral analysis.
| System | Type | Description | Examples |
|---|---|---|---|
| Snort | IDS/IPS | Open-source, signature-based | Community rules, emerging threats |
| Suricata | IDS/IPS | Multi-threaded, GPU support | ET Open, ET Pro rules |
| Zeek (Bro) | Network Security Monitor | Protocol analysis, event generation | Custom scripts, policy frameworks |
| Security Onion | Platform | Complete NTA platform | Snort, Suricata, Zeek, Elasticsearch |
# Snort rule example
alert tcp $HOME_NET any -> $EXTERNAL_NET 443 (msg:"ET TROJAN Possible Cobalt Strike Beacon";
flow:established,to_server;
content:"|00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00|";
depth:16;
threshold:type both, track by_src, count 5, seconds 30;
classtype:trojan-activity;
sid:1000001; rev:1;)
# Suricata rule format
alert dns any any -> any any (msg:"DNS Query to Malware Domain";
dns.query; content:"malware-domain.com"; nocase;
classtype:malware-cnc;
sid:2000001;)
6. Security Information and Event Management (SIEM)
SIEM platforms aggregate logs, alerts, and network data for centralized analysis and correlation.
- Log aggregation from diverse sources (firewalls, servers, applications)
- Correlation rules to detect complex attacks across multiple systems
- Dashboards and visualizations for situational awareness
- Alerting and case management for incident response
- Compliance reporting (SOC 2, PCI DSS, HIPAA)
# Splunk SPL query for detecting brute force index=firewall sourcetype=linux_secure | rex "Failed password for (?\w+) from (? \d+\.\d+\.\d+\.\d+)" | stats count by src_ip, user | where count > 10 # Elasticsearch query for suspicious network activity GET /_search { "query": { "bool": { "must": [ { "term": { "event.type": "connection" } }, { "range": { "bytes_out": { "gt": 1000000 } } } ], "filter": [ { "terms": { "destination.port": [22, 3389, 445] } } ] } } }
7. Threat Hunting with Network Data
Threat hunting proactively searches for malicious activity that may evade automated detection.
Common Threat Hunting Queries
- Beaconing Detection: Periodic outbound connections to external hosts
- Data Exfiltration: Unusual volume of outbound traffic
- DNS Tunneling: Long DNS queries or high-frequency DNS requests
- Lateral Movement: RDP, SMB, or WinRM connections between unusual hosts
- Command and Control: Connections to known malicious IPs or domains
- Encrypted Traffic Analysis: TLS certificates, JA3 fingerprints
# Detect beaconing (periodic connections)
# Calculate standard deviation of connection intervals
SELECT src_ip, dst_ip, dst_port,
AVG(time_diff) as avg_interval,
STDDEV(time_diff) as stddev_interval
FROM (
SELECT src_ip, dst_ip, dst_port,
LAG(timestamp) OVER (PARTITION BY src_ip, dst_ip, dst_port
ORDER BY timestamp) as prev_time,
timestamp - prev_time as time_diff
FROM connections
) WHERE time_diff IS NOT NULL
GROUP BY src_ip, dst_ip, dst_port
HAVING avg_interval BETWEEN 5 AND 60
AND stddev_interval < avg_interval * 0.2
# Detect data exfiltration (large outbound transfers)
SELECT src_ip, dst_ip, SUM(bytes_out) as total_bytes, COUNT(*) as connections
FROM connections
WHERE bytes_out > 1000000
GROUP BY src_ip, dst_ip
ORDER BY total_bytes DESC
8. Network Forensics
Network forensics involves preserving, analyzing, and presenting network evidence for investigations.
Forensic Best Practices
- Capture traffic before remediation (don't shut down the network first!)
- Preserve chain of custody for captured evidence
- Use write-blockers when analyzing captured files
- Document all analysis steps and findings
- Maintain timestamps and time synchronization
# Create forensic-quality capture with time stamps tcpdump -i eth0 -w forensic_$(date +%Y%m%d_%H%M%S).pcap -s 0 -n # Verify integrity with hash md5sum forensic_20260322_143000.pcap > forensic_20260322_143000.md5 # Extract files from PCAP (foremost, NetworkMiner) foremost -i capture.pcap -o extracted_files # Reconstruct TCP streams tshark -r capture.pcap -q --export-objects "tcp,streams"
9. Encrypted Traffic Analysis
With increasing encryption, analysts must derive insights from encrypted traffic without decryption.
- TLS Fingerprinting (JA3/JA3S): Identify TLS clients by their ClientHello characteristics
- Certificate Analysis: Certificate issuers, validity, Subject Alternative Names
- Traffic Patterns: Packet sizes, timing, volume
- SNI (Server Name Indication): Domain names in TLS handshakes
# JA3 fingerprint calculation for TLS clients # JA3 = MD5(SSLVersion,CipherSuites,Extensions,EllipticCurves,EllipticCurvePointFormats) # Detect suspicious TLS certificates tshark -r capture.pcap -Y "tls.handshake.certificate" -T fields \ -e tls.handshake.extensions_server_name \ -e tls.handshake.certificate # Detect self-signed certificates tshark -r capture.pcap -Y "tls.handshake.certificate and !tls.handshake.certificate.issuer"
10. Network Security Monitoring Tools
| Tool | Category | Purpose | \ \ \ \ \ \ \ \
|---|