CovertMark.strategy.entropy_est module

class CovertMark.strategy.entropy_est.EntropyEstimationStrategy(pt_pcap, negative_pcap=None, debug=True)[source]

Bases: CovertMark.strategy.strategy.DetectionStrategy

Detecting high-entropy fully-encrypted based on estimation of sliding window entropy on TCP payloads in both directions.

DESCRIPTION = 'Detecting high-entropy PTs based on sliding window entropy estimation.'
FALSE_POSITIVE_SCORE_WEIGHT = 0.5
MIN_TEST_SIZES = [1024, 512, 256, 128]
NAME = 'Entropy Estimation Strategy'
RUN_CONFIG_DESCRIPTION = ['Window Size', 'Test Size', 'Percentile Threshold']
THRESHOLDS = [0.1, 0.5, 1, 5]
TLS_HTTP_INCLUSION_THRESHOLD = 0.1
WINDOW_SIZE = 64
WINDOW_SIZES = [32, 64, 96]
config_specific_penalisation(config_set)[source]

Byte block sizes and min test sizes for entropy uniformity and distribution tests will have already inversely proportionally affected the positive execution time. As the percentile threshold has no effect on the difficulty of strategy deployment, no strategy-specific penalisation is required.

interpret_config(config_set)[source]

Block size, p-value threshold, and criteria distinguish entropy distribution tests.

negative_run(**kwargs)[source]

Test an identical configuration on negative packets. Reporting falsely blocked IPs.

Parameters
  • window_size (int) – the size of blocks of payload bytes tested in KS and AD. Default is set in BLOCK_SIZE.

  • test_size (int) – the minimum number of bytes tested in each payload for testing, with default set in TEST_SIZES.

  • threshold (int) – the percentile threshold for the proportion of high entropy packets considered as positives.

positive_run(**kwargs)[source]

Results from these tests estimate the presence of fully encrypted payloads by counting the number of sliding windows with large numbers of unique bytes.

Parameters
  • window_size (int) – the size of blocks of payload bytes tested in KS and AD. Default is set in BLOCK_SIZE.

  • test_size (int) – the minimum number of bytes tested in each payload for testing, with default set in TEST_SIZES.

  • threshold (int) – the percentile threshold for the proportion of high entropy packets considered as positives.

report_blocked_ips()[source]

Return a Wireshark-compatible filter expression to allow viewing blocked packets in Wireshark. Useful for studying false positives.

Returns

a Wireshark-compatible filter expression string.

run_strategy(**kwargs)[source]

PT input filters should be given as data.constants.IP_SRC and data.constants.IP_DST, and changed around if testing for downstream rather than upstream direction. Negative input filters specifying innocent clients should be given as an data.constants.IP_SRC.

Parameters

protocol_min_length (int) – Optionally set the minimum handshake TCP payload length of packets in that direction, allowing disregard of short packets.

set_strategic_filter()[source]

The base strategy is to only observe TCP packets that do not have valid TLS records (as identified by dpkt) but do bear a non-blank payload.

test_validation_split(split_ratio)[source]

Not needed, as a fixed strategy is used.