Add Bluecoat XSRF fixes by @froyo75 and CHANGELOG

CHANGELOG

@@ -0,0 +1,77 @@
+# CHANGELOG
+
+## Unreleased
+
++ Bluecoat XSRF fixes by @froyo75 adopted from PR [#37](https://github.com/threatexpress/domainhunter/pull/37) (Huge thanks!)
++ Fix for ExpiredDomains login by @davidlebr1
+
+## 07 January 2021
+
++ Fix Symantec Site Review (Bluecoat) reputation checking to bypass XSRF and additional POST parameter checks
++ Temporary fix for broken malware domains link. This service is no longer offered in the form used by DomainHunter.
++ Add internal code comments for readability
++ Add check for ExpiredDomains username before asking for a password
++ Disable Google Safe Browsing/PhishTank reputation from MxToolbox as this service has changed
+
+## 21 February 2020
+
++ updated Pillow version to support Python3.7+
++ Add instructions to install using pipenv
+
+## 13 August 2019
+
++ Added authentication support for ExpiredDomains.net thanks to acole76!
+
+## 5 October 2018
+
++ Fixed logic for filtering domains with desirable categorizations. Previously, some error conditions weren't filtered and would result in domains without a valid categorization making it into the final list.
+
+## 4 October 2018
+
++ Tweaked parsing logic
++ Fixed changes parsed columns indexes
+
+## 17 September 2018
+
++ Fixed Symantec WebPulse Site Review parsing errors caused by service updates
+
+## 18 May 2018
+
++ Add --alexa switch to control Alexa ranked site filtering
+
+## 16 May 2018
+
++ Update queries to increase probability of quickly finding a domain available for instant purchase. Previously, many reported domains had an "In Auction" or "Make an Offer" status. New criteria: .com|.net|.org + Alexa Ranked + Available for Purchase
++ Improved logic to filter out uncategorized and some potentially undesirable domain categorizations in the final text table and HTML output
++ Removed unnecessary columns from HTML report
+
+## 6 May 2018
+
++ Fixed expired domains parsing when performing a keyword search
++ Minor HTML and text table output updates
++ Filtered reputation checks to only execute for .COM, .ORG, and .NET domains and removed check for Archive.org records when performing a default or keyword search. Credit to @christruncer for the original PR and idea.
+
+## 11 April 2018
+
++ Added OCR support for CAPTCHA solving with tesseract. Thanks to t94j0 for the idea in [AIRMASTER](https://github.com/t94j0/AIRMASTER)  
++ Added support for input file list of potential domains (-f/--filename)
++ Changed -q/--query switch to -k/--keyword to better match its purpose
++ Added additional error checking for ExpiredDomains.net parsing
+
+## 9 April 2018
+
++ Added -t switch for timing control. -t <1-5>
++ Added Google SafeBrowsing and PhishTank reputation checks
++ Fixed bug in IBMXForce response parsing
+
+## 7 April 2018
+
++ Fixed support for Symantec WebPulse Site Review (formerly Blue Coat WebFilter)
++ Added Cisco Talos Domain Reputation check
++ Added feature to perform a reputation check against a single non-expired domain. This is useful when monitoring reputation for domains used in ongoing campaigns and engagements.
+
+## 6 June 2017
+
++ Added python 3 support
++ Code cleanup and bug fixes
++ Added Status column (Available, Make Offer, Price, Backorder, etc)

README.md

@@ -6,66 +6,7 @@ Domain name selection is an important aspect of preparation for penetration test
 
 This Python based tool was written to quickly query the Expireddomains.net search engine for expired/available domains with a previous history of use. It then optionally queries for domain reputation against services like Symantec Site Review (BlueCoat), IBM X-Force, and Cisco Talos. The primary tool output is a timestamped HTML table style report.
 
-## Changelog
-
-- 07 January 2021
-   + Fix Symantec Site Review (Bluecoat) reputation checking to bypass XSRF and additional POST parameter checks
-   + Temporary fix for broken malware domains link. This service is no longer offered in the form used by DomainHunter.
-   + Add internal code comments for readability
-   + Add check for ExpiredDomains username before asking for a password
-   + Disable Google Safe Browsing/PhishTank reputation from MxToolbox as this service has changed
-
-- 21 February 2020
-   + updated Pillow version to support Python3.7+
-   + Add instructions to install using pipenv
-
-- 13 August 2019
-   + Added authentication support for ExpiredDomains.net thanks to acole76!
-   
-- 5 October 2018
-   + Fixed logic for filtering domains with desirable categorizations. Previously, some error conditions weren't filtered and would result in domains without a valid categorization making it into the final list.
-
-- 4 October 2018
-   + Tweaked parsing logic
-   + Fixed changes parsed columns indexes
-
-- 17 September 2018
-    + Fixed Symantec WebPulse Site Review parsing errors caused by service updates
-
-- 18 May 2018
-    + Add --alexa switch to control Alexa ranked site filtering
-
-- 16 May 2018
-    + Update queries to increase probability of quickly finding a domain available for instant purchase. Previously, many reported domains had an "In Auction" or "Make an Offer" status. New criteria: .com|.net|.org + Alexa Ranked + Available for Purchase
-    + Improved logic to filter out uncategorized and some potentially undesirable domain categorizations in the final text table and HTML output
-    + Removed unnecessary columns from HTML report
-
-- 6 May 2018
-    + Fixed expired domains parsing when performing a keyword search
-    + Minor HTML and text table output updates
-    + Filtered reputation checks to only execute for .COM, .ORG, and .NET domains and removed check for Archive.org records when performing a default or keyword search. Credit to @christruncer for the original PR and idea.
-
-- 11 April 2018
-    + Added OCR support for CAPTCHA solving with tesseract. Thanks to t94j0 for the idea in [AIRMASTER](https://github.com/t94j0/AIRMASTER)  
-    + Added support for input file list of potential domains (-f/--filename)
-    + Changed -q/--query switch to -k/--keyword to better match its purpose
-    + Added additional error checking for ExpiredDomains.net parsing
-
-- 9 April 2018
-    + Added -t switch for timing control. -t <1-5>
-    + Added Google SafeBrowsing and PhishTank reputation checks
-    + Fixed bug in IBMXForce response parsing
-
-- 7 April 2018
-    + Fixed support for Symantec WebPulse Site Review (formerly Blue Coat WebFilter)
-    + Added Cisco Talos Domain Reputation check
-    + Added feature to perform a reputation check against a single non-expired domain. This is useful when monitoring reputation for domains used in ongoing campaigns and engagements.
-
-- 6 June 2017
-    + Added python 3 support
-    + Code cleanup and bug fixes
-    + Added Status column (Available, Make Offer, Price, Backorder, etc)
-
+See [CHANGELOG](./CHANGELOG) for history of updates and release notes!
 ## Features
 
 - Retrieve specified number of recently expired and deleted domains (.com, .net, .org) from ExpiredDomains.net

domainhunter.py

@@ -18,8 +18,10 @@ import os
 import sys
 from urllib.parse import urlparse
 import getpass
+# Bluecoat XSRF
+from hashlib import sha256
 
-__version__ = "20210107"
+__version__ = "20221024"
 
 ## Functions
 
@@ -93,11 +95,22 @@ def checkBluecoat(domain):
             'U2NyaXB0aW5nIGFnYWluc3QgU2l0ZSBSZXZpZXcgaXMgYWdhaW5zdCB0aGUgU2l0ZSBSZXZpZXcgVGVybXMgb2YgU2VydmljZQ=='
         ]
         
+        # New Bluecoat XSRF Code added May 2022 thanks to @froyo75
+        xsrf_token_parts = token.split('-')
+        xsrf_random_part = random.choice(xsrf_token_parts)
+        key_data = xsrf_random_part + ': ' + token
+        # Key used as part of POST data
+        key = sha256(key_data.encode('utf-8')).hexdigest()
+        random_phrase = base64.b64decode(random.choice(phrases)).decode('utf-8')
+        phrase_data = xsrf_random_part + ': ' + random_phrase
+        # Phrase used as part of POST data
+        phrase = sha256(phrase_data.encode('utf-8')).hexdigest()
+        
         postData = {
             'url':domain,
             'captcha':'',
-            'key':'%032x' % random.getrandbits(256), # Generate a random 256bit "hash-like" string
-            'phrase':random.choice(phrases), # Pick a random base64 phrase from the list
+            'key':key,
+            'phrase':phrase, # Pick a random base64 phrase from the list
             'source':'new-lookup'}
 
         headers = {'User-Agent':useragent,