Whittle

Released: April 6, 2024

Tool Links: GitHub Repository

Whittle is a command-line tool designed to manipulate and optimize wordlists for security testing. It was born out of a frustration with existing wordlists and wordlist generation tools that either did not contain the right features, or were too complex to use.

Motivation

When conducting security assessments, especially in internal infrastructure penetration testing, having the right wordlist can make a significant difference to efficiency and success rates when password cracking. Frequently, I found myself:

Combining multiple wordlists
Creating wordlists from scratch that were way too large, or not specific enough
Wasting time using wordlists comprised of passwords that would never be accepted by default password policies

While there are tools available that can perform some of these functions, I never found one that considers Microsoft's password complexity requirements.

Features

Whittle focuses on being a versatile, yet simple tool for refining wordlists:

Microsoft's "Must meet password complexity requirements" - By default, active directory will not accept passwords that don't meet the complexity requirements. Whittle can be used to remove candidate entries that don't meet these requirements. It also considers account full names and usernames!
Filter - Remove entries based on minimum and maximum length, to deal with custom password policies.
Statistics - Verbose statistics are provided, including the number of rejections and processing times.

Implementation & Usage

Whittle is written in Python, making it cross-platform and easy to extend. It uses efficient data structures to handle large wordlists with minimal memory consumption, and supports hyperthreading for parallel processing.

# Enforce Microsoft's Password Complexity Requirements and output to text file
$ python whittle.py -c -w /path/to/wordlist.txt -o /path/to/output.txt
                
# Only allow passwords with a minimum length of 8 and a maximum length of 12
$ python whittle.py -m 8 -M 12 -w /path/to/wordlist.txt -o /path/to/output.txt
                
# Enforce Microsoft's Password Complexity Requirements, alongside the samAccountName and displayName of a target
$ python whittle.py -c --sam-account jdoe --display-name "John Doe" -w /path/to/wordlist.txt -o /path/to/output.txt
                
# Process with verbose statistics
$ python whittle.py -v -w /path/to/wordlist.txt -o /path/to/output.txt
                
# With no output file, contents will be output to stdout. Verbose statistics use stderr so output can be piped / redirected.
$ python whittle.py -c -v -w /path/to/wordlist.txt | some-other-command

Case Study: Improving Success Rates

During a particularly successful penetration testing engagement for a financial services client, I found myself with an NTDS.dit file that was 1.7GB in size (that's a lot of password material). Rather than just using rockyou.txt as-is, I used whittle to refine it to be 100% compliant with the password policy.

$ python whittle.py -w rockyou.txt -m 6 -M 12 -c -v -o filtered.txt
Using character encoding: utf-8
Using 32 threads with chunk size of 100000

Passwords processed: 14344391/14344391 (100.0%) | Memory: 35.5MB (Peak: 43.0MB)
Total Passwords Processed: 14344391
Passwords Accepted: 780469
Passwords Rejected: 13563922
Processing Time: 6.38 seconds
Peak Memory Usage: 43.0MB
Output File: filtered.txt
New File Size: 7.31MB

So to summarise, that's just shy of 800k passwords that were accepted, and over 13.5 million that were rejected. Those 13.5 million rejected passwords would never have hit (due to the password policy) and so constituted completely wasted compute time. This also only took 6.38 seconds to process - compare that to the extra time it would have taken for hashcat to use those 13.5 million rejected passwords, and you're in for a winner.

Note:

This is only applicable where extra processing is not done - for example with hashcat rules. Since getting the NTDS.dit file took most of the engagement time, and I had a pretty low performance GPU, I didn't have the time to use beefy rules like dive or oneruletorulethemall. If this was purely a password audit assessment, this would be a different story.

Future Development

I'm continuously improving Whittle based on feedback and real-world usage. Some planned features include:

Support for custom regex patterns, to allow for more complex filtering.
The ability to provide a password policy export, which will be parsed for complexity requirements.

← Back to Home