Browsing tag

crawling

ACHE – A Web Crawler For Domain-Specific Search

ACHE is a focused web crawler. It collects web pages that satisfy some specific criteria, e.g., pages that belong to a given domain or that contain a user-specified pattern. ACHE differs from generic crawlers in sense that it uses page classifiers to distinguish between relevant and irrelevant pages in a given domain. A page classifier […]

contact.sh – An OSINT tool to find contacts in order to report security vulnerabilities

An OSINT tool to find contacts in order to report security vulnerabilities. Installation Linux Make sure you have installed the whois and jq packages. $ git clone https://github.com/EdOverflow/contact.sh.git $ cd contact.sh/ $ chmod u+x contact.sh $ ./contact.sh -d google.com -c google OSX $ brew install gnu-sed –with-default-names $ brew install jq $ git clone https://github.com/EdOverflow/contact.sh.git […]

RED HAWK v2 – All In One Tool for Information Gathering and Vulnerability Scanning – Kali Linux 2017.3

All in one tool for Information Gathering and Vulnerability Scanning RED HAWK : https://github.com/Tuhinshubhra/RED_HAWK Scans That You Can Perform Using RED HAWK : Basic Scan Site Title NEW IP Address Web Server Detection IMPROVED CMS Detection Cloudflare Detection robots.txt Scanner Whois Lookup IMPROVED Geo-IP Lookup Grab Banners IMPROVED DNS Lookup Subnet Calculator Nmap Port Scan […]

SEO Tip: To Slash Or Not To Slash At The End Of A URL

Short Bytes: Traditionally the URL ending with a slash denoted a directory presence at the path, and a non-slashed version implied a file. Today, it’s a good practice that both the versions of a URL produces same content. But returning the same content for both versions can hurt SEO. Smart and proper redirection from one version […]

How Search Engine Works and Makes Your Life Easier?

Short Bytes: Search Engine is a software that allows the display of relevant webpage results based on the search query input by the use of Web Crawling and Web Indexing, some fat formulae and intelligent algorithms in order to gather the appropriate data. A few thousand searches were made in the time this webpage got […]