CovertMark.strategy.entropy_dist module

class CovertMark.strategy.entropy_dist.EntropyStrategy(pt_pcap, negative_pcap=None, debug=True)[source]

Bases: CovertMark.strategy.strategy.DetectionStrategy

Detecting high-entropy encryption based on payload byte-uniformity and entropy-distribution tests on TCP payloads in both directions.

BLOCK_SIZE = 8
BLOCK_SIZES = [16, 32, 64]
CRITERIA = [3, 2, 1]
CRITERIA_DESCRIPTIONS = {1: 'sensitive', 2: 'majority voting', 3: 'conservative'}
DESCRIPTION = 'Detecting high-entropy PTs based on payload byte-uniformity and entropy-distribution.'
FALSE_POSITIVE_SCORE_WEIGHT = 0.5
MAX_CRITERION = 3
MIN_TEST_SIZES = [1024, 512, 256, 128]
NAME = 'Entropy Distribution Strategy'
P_THRESHOLD = 0.1
RUN_CONFIG_DESCRIPTION = ['Block Size', 'Test Size', 'Criterion']
TLS_HTTP_INCLUSION_THRESHOLD = 0.1
config_specific_penalisation(config_set)[source]

Byte block sizes and min test sizes for entropy uniformity and distribution tests will have already inversely proportionally affected the positive execution time. Therefore the only additional penalty is based on the number of statistical run needed as determined by the number of agreements required, with 10% penalty for each additional statistical tests beyond the minimum.

interpret_config(config_set)[source]

Block size, p-value threshold, and criteria distinguish entropy distribution tests.

negative_run(**kwargs)[source]

Test an identical configuration on negative packets. Reporting falsely blocked IPs.

Parameters:
  • block_size (int) – the size of blocks of payload bytes tested in KS and AD. Default is set in BLOCK_SIZE.
  • test_size (int) – the minimum number of bytes tested in each payload for testing, with default set in TEST_SIZES.
  • criterion (int) – the number of rejected hypothesis among all tests needed to reach a positive conclusion.
positive_run(**kwargs)[source]

Three different criteria of combining results from KS byte-uniformity and Entropy Distribution, and Anderson_Darling tests together, with variable min tested payload sizes and test block sizes. Results from these tests detect uniformly high entropy payloads that are likely results of pseudo-random encryptions.

Parameters:
  • block_size (int) – the size of blocks of payload bytes tested in KS and AD. Default is set in BLOCK_SIZE.
  • test_size (int) – the minimum number of bytes tested in each payload for testing, with default set in TEST_SIZES.
  • criterion (int) – the number of rejected hypothesis among all tests needed to reach a positive conclusion.
report_blocked_ips()[source]

Return a Wireshark-compatible filter expression to allow viewing blocked packets in Wireshark. Useful for studying false positives.

Returns:a Wireshark-compatible filter expression string.
run_strategy(**kwargs)[source]

PT input filters should be given as data.constants.IP_SRC and data.constants.IP_DST, and changed around if testing for downstream rather than upstream direction. Negative input filters specifying innocent clients should be given as an data.constants.IP_SRC.

Parameters:protocol_min_length (int) – Optionally set the minimum handshake TCP payload length of packets in that direction, allowing disregard of short packets.
set_strategic_filter()[source]

The base strategy is to only observe TCP packets that do not have valid TLS records (as identified by dpkt) but do bear a non-blank payload.

test_validation_split(split_ratio)[source]

Not needed, as a fixed strategy is used.