ODIN – Automating Penetration Testing Tasks

ODIN [ Observe, Detect, and Investigate Networks ] is a Python tool for automating intelligence gathering, testing and reporting.

ODIN is still in active development. ODIN is designed to be run on Linux. About 90% of it will absolutely work on Windows or MacOS with Python 3 and a copy of urlcrazy, but `extract`, used for pulling metadata from non-PDF files, is exclusive to Linux. You’ll be fine using an OS without access to `extract`, but you’ll see some warnings and get less information.

ODIN aims to automate many of the common recon tasks carried out by penetration testers. Such as:

• Harvesting email addresses and employee names for a company.
• Linking employees and companies to social media profiles.
• Checking to see if discovered accounts have been a part of any public security breaches or appeared in any pastes.
• Collecting data on domains and IP addresses from Shodan, Censys, DNS records, and whois/RDAP.
• Discovering subdomains, their related IP addresses, and looking for CDNs that might allow for domain fronting.
• Hunting Office files and PDFs under a domain, downloading them, and extracting metadata.
• Linking key words, like a company name or domain, to AWS via S3 buckets and account aliases.
• More to come in the future…

Installing ODIN

ODIN requires Python 3. Using pipenv for managing the required libraries is the best option to avoid Python installations getting mixed-up.

1. Run pip3 install --user pipenv or python3 -m pip install --user pipenv.
2. Run git clone https://github.com/chrismaddalena/ODIN.git.
3. Run cd ODIN && pipenv install.
4. Run pipenv shell to get started using ODIN.

Note: On MacOS you may get an error about pew not being in your PATH after installing pipenv and attempting to install ODIN. To fix it, follow these steps in order:

• Uninstall virtualenv, pipenv, and pew.
• Install virtualenv
• Install pew
• Install pipenv

Setup API Keys

1. Review the keys.config.sample file to fill-in your API keys and create a keys.config file.
2. cd into the /setup directory and run setup_check.py to make sure your keys.config file is in order.
3. Install awscli and run aws configure.

The APIs and Services

ODIN uses several APIs to gather information. Some of these require an API key, but most of the APIs are free. That is to say, you can get a free key and then pay for more requests/day. Shodan is a good example of this. You may prefer to not use APIs at all for one reason or another. You can still use ODIN, but a few of the APIs are just really fantastic and you should consider using them, specifically Censys and Shodan.

Whois and RDAP

Both of these services are used to collect data on domains and IP addresses. This includes attaching domains to IP addresses, identifying the network CIDRs for these addresses, and pulling information about the owners.

No API key is needed.

Robtex

The Robtex free REST API is used to collect domain names tied to IP addresses. This information is displayed alongside the RDAP information for IP addresses, so you can see what else is hosted at that IP address.

No API key is needed.

Shodan

Shodan is used to search for domains and lookup hosts (IP addresses). This pulls in information like open ports, banners, hostnames, and location data. Shodan also flags hosts for well known vulnerabilities like Heartbleed. This data is recorded as well, if it exists, but does tend to be outdated (or just wrong) a lot of the time.

Sign-up for an account to get your API key: shodan.io

Censys

Censys is very much like Shodan, except less information about open ports/services is available. However, Censys provides a way to search for certificates tied to a domain. This can be a lot of data, but you may find new hosts, like those tied to an employee’s email address and used for a VPS in the cloud.

Sign-up for an account to get your API key: censys.io

URLVoid

URLVoid offers reputation data for domains, including Alexa and Google rankings, domain age, and location data. It also keeps track of domains that have been flagged for malicious activity by various entities (e.g. Fortinet, Avira).

This may be the most “skippable” of the APIs, but some of the data can be useful and worthwhile. It’s included for those occasions.

Sign-up for an account to get your API key: urlvoid.com/api

Twitter

If you setup a Twitter app for ODIN, the tokens can be used with Tweepy to collect account data (e.g. real name, location, follower count, and user description) from Twitter profiles ODIN has linked to the target organization.

In the future, this may be used to collect analytics from Twitter to help you find very active users or get a profile of them.

Become a Twitter developer by going to dev.twitter.com and then create an app on apps.twitter.com.

Cymon

eSentire’s Cymon is used to check domains and IP addresses to see if the target appears in any of Cymon’s collected threat intelligence feeds. This is used for reputation checks, but also used in combination with urlcrazy to check similar, registered domains to see if the domain or the domain’s A-record IP addresses have been reported.

Note that appearing in a threat feed doesn’t mean something is wrong or that Cymon has bad data. A domain may have been used for phishing, been detected and seized, and is now dormant with the old malicious A records. Then you have things like cloud service IPs that change hands often. Events like that can lead to a domain or IP being used for malicious activities one day and safe the next. Always investigate these findings before crying wolf to your client.

Sign-up for an account to get your API key: cymon.io

HaveIBeenPwned

Email addresses are checked against HIBP to determine if any email addresses for the organization have been mentioned in any pastes or been involved in any security breaches.

No API key is needed.

DNS Dumpster

DNS Dumpster is a cool project you can find at dnsdumpster.com. Subdomain information is collected from DNS Dumpster, including a neat domain map image!

No API key is needed.

NetCraft

ODIN will check NetCraft for domain history and known subdomains. This does require a web driver for Selenium. If you download a driver and provide the path to it in your keys.config file (Yes, this isn’t really a key, but so be it), NetCraft searches will be kicked off automatically when you perform domain OSINT.

The Chrome web driver is recommended, but the Firefox/Gecko driver should work just fine, too.

Chrome Web Driver Gecko Driver

EmailHunter

Meant for marketing folks to find leads and contacts at a company, this service offers free API keys for harvesting their contact information organizaed by company/domain. Hunter will return names, email addresses, phone numbers, Twitter handles, LinkedIn profile links, and job titles.

Sign-up for an account to get your API key: hunter.io

Full Contact

Full Contact support is implemented only for their Company API at the moment, but support for the People API may come in the future. For now, this is used to build a company profile based on a target domain, such as the client’s primary domain used for email and their website. Full Contact catalogues everything from website info and company logo(s) to website blurbs and social media profiles.

It’s likely Full Contact will get some things wrong, such as number of employees. In my experience, the data is usually not too far off the mark, but the profile is only meant to act as a snapshot to get you started.

Sign-up for an account to get your API key: app.fullcontact.com

AWS

Yes, Amazon Web Services. ODIN will perform recon against AWS to find things like S3 buckets and accounts names and aliases. Account names are strings of numbers, so you will need some idea of what you’re looking for there. Aliases, however, can be anything, like a company name, and those can be validated as existing or not.

By default, ODIN uses the client (-c) name and domain (-d) for searches. ODIN will search for the name with spaces stripped out, the domain with the TLD, and the domain without the TLD. Then ODIN will add some common suffices and prefixes, like “downloads-” or “-apps” to these keywords.

Optional wordlists can be provided for additional keywords and ‘fixes. Keywords can be anything, really. Consider assembling a list of related words, alternate client names, etc.

An Amazon and awscli are required.

Digital Ocean

ODIN will search for Digital Ocean Spaces just like it searches for S3 buckets. Spaces follows the same standards as S3, so it is simple to verify existing Spaces.

No API key is needed.