All about DNS

Content

.nl stats and data - SIDN LabsAPNIC Labs StatsAkamai DNS Trends and TrafficCZ.NIC StatisticsCensysCheck Propagation of DNS RecordsCollection of "bad" packets in PCAPsCompacted-DNS (C-DNS): A Format for DNS Packet CaptureCurated Lists of DNS Server SoftwareCyber Threat Intelligence FeedsDMAP Domain Mapper by SIDN LabsDNS Authoritative Server BenchmarksDNS Census 2013DNS CoffeeDNS Privacy ProjectDNS Quality/Overview ToolsDNS Queries to Authoritative DNS Server at SURFnet by Google's Public DNS ResolverDNS Replay Tool (drool)DNS ToysDNSCAPDNSDBDNSMONDNSSEC Deployment ReportsDNSSEC Early Warning SystemDNSTOPDomain Crawling ListsDomain Name System (DNS) ParametersFlamethrowerForward DNS Rapid7Get your public IP using DNSHello DNSICANN Managed Root Servers StatisticsIP Flow Information Export (IPFIX) EntitiesInternet Maps (RIPE NCC)List of DNS related RFCsLists of DNS BlacklistsMeasurement Factory: DNS SurveyNextDNSOWASP AmassOpen INTELPacketQ: Query PCAPs using SQLPassive DNS (CIRCL)Passive DNS – Common Output FormatPi-holePublic DNS Server ListPublic Suffix ListRIPE AtlasRIPEstat: Providing open data and insights for Internet resourcesRSSAC002 DNS Root Server DataRelationship between DNS RFCsResolver TestbedRoot ServersRouting Information Service (RIS)SIDN Labs DNS WorkbenchShadowserver Scanning ProjectShodanWAND Active Measurement ProjectWildcard DNS for IP AddressesZMap Projectdnsdumpsterdnskv: DNS-based Key-Value Storagednsperf and resperfdnsstream (Twitter)dnsteal DNS Exfiltration Tooldnsthoughtiodine DNS Tunnelk-v.io: DNS-based Key-Value Storemess with dnsrespdiff

.nl stats and data - SIDN Labs

 https://stats.sidnlabs.nl/en/

DNS | DNSSEC | Dataset | IP | Network

Historic datasets (from 2014 onwards) for the .nl TLD. Datasets are available in JSON format.

Datasets cover information about:

  • DNS
    • Domain Names
    • Query Type
    • Resonse Codes
    • IPv6 Support
  • Resolvers
    • Location
    • Number of IP addresses
    • Validating Resolvers
    • Popular Networks
    • Port Randomness
  • DNSSEC
    • Validating Queries
    • DANE
    • Used Algorithms
  • Mail
    • Mail RRs
    • SPF Information


APNIC Labs Stats

 https://stats.labs.apnic.net/

Autonomous System | BGP | DNS | DNSSEC | Dataset | IP

APNIC gathers many statistics and offers them on their website. However, they provide way more data than it might initially look like, since many of the datasets are not linked from their main page.



CZ.NIC Statistics

 https://stats.adam.nic.cz/dashboard/en/index.html

DNS | Dataset

The website contains information about the cz. TLD operated by CZ.NIC. It contains information about the query volume, query type, round-trip time (RTT) and geographic location of the traffic sources. It also has information about the registry functions, such as registrar information, domain transfers or whois requests. Lastly, information about the mojeID accounts, a login provider operated by CZ.NIC are also available.


Censys

 https://censys.io/

Certificate | DNS | Dataset | IP | Network

Censys performs regular scans for common protocols (e.g., DNS, HTTP(S), SSH). Provides a search for TLS certificates.

Access is free, but requires registration. The website no longer provides free bulk access. Bulk access requires a commercial or a research license. The free access is limited to 1000 API calls per day.

@InProceedings{censys15,
    author = {Zakir Durumeric and David Adrian and Ariana Mirian and Michael Bailey and J. Alex Halderman},
    title = {A Search Engine Backed by {I}nternet-Wide Scanning},
    booktitle = {Proceedings of the 22nd {ACM} Conference on Computer and Communications Security},
    month = oct,
    year = 2015
}










DNS Coffee

 https://dns.coffee/

DNS | Dataset | IP | Network | Search

DNS Coffee collects and archives stats from DNS Zone files in order to provide insights into the growth and changes in DNS over time.

The website includes information such as the size of different zones. It track over 1200 zone files.

It provides searching through the zones files based on domain names, name servers, or IP addresses. It can also visualize the relationship between a domain, the parent zones and the name server in what they call a "Trust Tree".



DNS Quality/Overview Tools

DNS | DNSSEC | Dataset | Network | Tool

Check My DNS

Browser-based DNS resolver quality measurement tool. Uses the browser to generate many resolver queries and tests for features they should have, such as EDNS support, IPv6, QNAME Minimisation, etc.

This test is also available as a CLI tool: https://github.com/DNS-OARC/cmdns-cli

DNSSEC Debugger

Analyze DNSSEC deployment for a zone and show errors in the configuration.

DNSViz

Gives an overview over DNSSEC delegations, response sizes, and name servers.

GitHub: https://github.com/dnsviz/dnsviz

DNS X-Ray

The website has an online test, which performs DNS lookups. These DNS lookups test if certain resource records are overwritten in the cache. The tool can then determine what DNS software is used, where the server is located, how many caches there are, etc.

EDNS Compliance Tester

Test name server of zones for correct EDNS support.

The Transitive Trust and DNS Dependency Graph Portal

Shows the trust dependencies in DNS. Given a domain name it can show how zones delegate to each other and why. The delegation is done between IP addresses and zones.

Root Canary Project

The project used to monitor the first root KSK key rollover. Now it contains the paper "Roll, Roll, Roll your Root: A Comprehensive Analysis of the FirstEver DNSSEC Root KSK Rollover" describing the experiences of the first root KSK rollover

Additionally, it includes a tester for DNSSEC algorithm support, which shows the algorithms supported by the currently used recursive resolver. It provides statistics about support for DNSSEC algorithms. It has a web based test to test your own resolver and provides a live monitoring using the RIPA Atlas.

DNSSEC algorithms resolver test


DNS Queries to Authoritative DNS Server at SURFnet by Google's Public DNS Resolver

 https://data.4tu.nl/articles/dataset/DNS_Queries_to_Authoritative_DNS_Server_at_SURFnet_by_Google_s_Public_DNS_Resolver/12682040

DNS | Dataset | Network

This dataset covers approximately 3.5 billion DNS queries that were received at one of SURFnet's authoritative DNS servers from Google's Public DNS Resolver. The queries were collected during 2.5 years. The dataset contains only those queries that contained an EDNS Client Subnet.

The dataset covers data from 2015-06 through 2018-01.

DOI Identifier



DNS Toys

 https://www.dns.toys/

DNS | Tool

DNS Toys is an authoritative DNS server offering different unit conversion and lookup tools.

Some examples from the website include:

# Lookup time by city name
dig newyork.time @dns.toys

# or weather
dig newyork.weather @dns.toys

# Return the client IP address
dig ip @dns.toys

# Number conversion from decimal to hex
dig 100dec-hex.base @dns.toys



DNSDB

 https://scout.dnsdb.info/

DNS | Dataset | Network

Historical DNS database. Contains information recorded at recursive resolver about domain names, first/last seen, current bailiwick. Allows to see the lifetime of resource records and can be used as a large database.






Domain Crawling Lists

DNS | Dataset

Domain popularity lists provide a starting point for crawling domains with the most users. The most commonly used list for security research is the Alexa list.

  • Alexa
    The list is updated daily and contains one million websites. The ranking is based on page views, but very volatile.
  • CISCO Umbrella
    The list is updated daily and contains one million websites. The ranking is based on traffic seen on the OpenDNS resolvers.
  • Majestic
    The list is updated daily and contains one million websites. The ranking is based on backlinks from other websites.
  • Tranco
    A Research-Oriented Top Sites Ranking Hardened Against Manipulation
    The Tranco list aims to provide a better list for security research. The authors explain on their website and their paper what the flaws in the existing lists
  • Quantcast
    The list is updated daily and contains around 500,000 websites. It is based on users visiting the site within the previous month and highly US focussed.




Forward DNS Rapid7

 https://opendata.rapid7.com/sonar.fdns_v2/

DNS | Dataset | IP | Network

This dataset contains the responses to DNS requests for all forward DNS names known by Rapid7's Project Sonar. Until early November 2017, all of these were for the 'ANY' record with a fallback A and AAAA request if neccessary. After that, the ANY study represents only the responses to ANY requests, and dedicated studies were created for the A, AAAA, CNAME and TXT record lookups with appropriately named files.

The data is updated every month. Historic data can be downloaded after creating a free account.


Get your public IP using DNS

 https://docs.google.com/spreadsheets/d/1lkNJ0uQwbeC1ZTRrxdtuPLCIl7mlUreoKfSIgajnSyY/htmlview

DNS | IP

Some Open DNS operators provide a way to return the IP address of the request packet.

Google DNS:

dig o-o.myaddr.l.google.com txt @ns1.google.com +short

OpenDNS:

dig myip.opendns.com @resolver1.opendns.com +short

Akamai:

$ dig +short TXT whoami.ds.akahelp.net
"ns" "2001:db8::abcd"
"ecs" "203.0.113.0/24/0"
"ip" "203.0.113.132" 

The ds stands for dual stack. It is also reachable under whoami.ipv6.akahelp.net to force IPv6 or whoami.ipv4.akahelp.net to force IPv4.

ns stands for the unicast IP address of the recursive resolver. ecs contains the EDNS client subnet information, if the option is used by the resolver. In this case ip contains a representative IP within the ECS, not necessarily the IP of the client.

The new version is introduced in this blog post https://www.akamai.com/blog/developers/introducing-new-whoami-tool-dns-resolver-information.

The old version might be decomissioned:

dig whoami.akamai.net. @ns1-1.akamaitech.net. +short

Source







Lists of DNS Blacklists

DNS | Dataset | IP | Network | Spam | Tool

These projects either operate DNS based Real-time Blackhole Lists (RBL) or allow checking if an IP is contained. The Multi-RBL websites are helpful in finding a large quantity of RBLs.



NextDNS

 https://nextdns.io/

DNS | Tool

A free and configurable DNS resolver. It provides customizable blocking, such as for ads, trackers, or malicious websites. Additionally, statistics can be shown, such as for most blocked website.

A similar self-hosted variant is Pi-hole.


OWASP Amass

 https://github.com/OWASP/Amass

CTF | DNS | Tool

The OWASP Amass tool suite obtains subdomain names by scraping data sources, recursive brute forcing, crawling web archives, permuting/altering names and reverse DNS sweeping. Additionally, Amass uses the IP addresses obtained during resolution to discover associated netblocks and ASNs. All the information is then used to build maps of the target networks.


Open INTEL

 https://www.openintel.nl/

DNS | Dataset | IP | Network

Open INTEL is an active DNS database. It gathers information from public zone files, domain lists (Alexa, Umbrella), and reverse DNS entries. Once every 24 hours data is collected about a bunch of DNS RRsets (SOA, NS, A, AAAA, MX, TXT, DNSKEY, DS, NSEC3, CAA, CDS, CDNSKEY). The data is openly avaible as AVRO files and dates back until 2016.

The data can be freely downloaded. There is documentation on the layout of the AVRO files.

The project is similar to Active DNS but seems to be larger in scope.





Pi-hole

 https://pi-hole.net/

DNS | Tool

A free and configurable DNS stub-resolver. It provides customizable blocking, such as for ads, trackers, or malicious websites. Additionally, statistics can be shown, such as for most blocked website.

It can also function as a DHCP server for clients on the same network.

A similar service is NextDNS.





RIPEstat: Providing open data and insights for Internet resources

 https://stat.ripe.net/

Autonomous System | BGP | DNS | Dataset | Network | Tool

RIPEstat is a network statistics platform by RIPE. The platform shows data for IP addresses, networks, ASNs, and DNS names. This includes information such as the registration information, abuse contacts, blocklist status, BGP information, geolocation lookups, or reverse DNS names. Additionally, the website links to many other useful tools, such as an address space hierarchy viewer, historical whois information, and routing consistency checks.




Resolver Testbed

 https://github.com/icann/resolver-testbed

DNS | Tool

This repo describes a testbed to test various DNS resolvers. The purpose of the testbed is to allow researchers to set up many resolvers and run tests on each. For example, a test might see what the resolver emits when it is priming, or when it is responding to a particular query while using DNSSEC validation.


Root Servers

 https://root-servers.org

DNS | Dataset | Tool

Overview page for the DNS root servers. It contains links to general news and all the supporting organizations.

The website features a map with all geographic locations. It contains information about locations, IPv4/IPv6 reachability and IP addresses.

Each root server has its own subdomain in the form of https://a.root-servers.org. It contains access to historical performance data like:

  • Size and time of zone updates
  • RCODE volume
  • query and response sizes for UDP and TCP
  • traffic volume (packets per time)
  • Unique sources



SIDN Labs DNS Workbench

 https://workbench.sidnlabs.nl/

DNS | DNSSEC

The DNS workbench is a testbed which allows experimentation how different authoritative DNS servers answer to queries.

It covers five open source authoritative servers, namely Bind9, Knot, NSD4, PowerDNS, and Yadifa. The workbench contains zones to test the support for many different resource record (RR) types, DNSSEC validation and how invalid zones are managed, delegations, zone transfers, and potentially more.

Find the project on GitHub.


Shadowserver Scanning Project

 https://scan.shadowserver.org/

DNS | Dataset | Malware | Network

The Shadowserver Scanning projects performs regular Internet wide scans for many protocols. They scan for four main types of protocols:

  1. Amplification protocols, e.g., DNS or NTP
  2. Botnet protocols, e.g., Gameover Zeus or Sality
  3. Protocols that should not be exposed, e.g., Elastic Search, LDAP, or RDP
  4. Vulnerable Protocols, e.g., SSLv3

The website is a great resource to get general statistics about the protocols, like the number of hosts speaking the protocol, their geographic distribution, associated ASNs, and the historic information.



WAND Active Measurement Project

 https://amp.wand.net.nz/

Autonomous System | DNS | Dataset | Network | Tool | Traceroute

AMP is a system designed to continuously perform active network measurements between a mesh of specialist monitor machines, as well as to other targets of interest. These measurements are used to provide both a view of long-term network performance as well as to detect notable network events when they happen.

The project is run with a custom client and server software. The measurement results can be viewed on the website. It includes traceroutes, latencies (DNS, HTTP, ICMP, TCP), HTTP page sizes, and packet loss. The software is available as open source.


Wildcard DNS for IP Addresses

DNS | IP | Network | Tool

These services allow you to create a domain name for any IP address. The IP address is encoded into the domain name. An overview over different services can be found here.

Online Services

  • https://nip.io/ provides IPv4 only

    • Supports both . and - separators.
    • 10.0.0.1.nip.io resolves to 10.0.0.1
    • 192-168-1-250.nip.io resolves to 192.168.1.250
    • customer1.app.10.0.0.1.nip.io resolves to 10.0.0.1
    • magic-127-0-0-1.nip.io resolves to 127.0.0.1
  • https://sslip.io/ provides IPv4 and IPv6

    • Supports both . and - separators.
    • Provides the ability to use the service with your own branding.
    • 192.168.0.1.sslip.io resolves to 192.168.0.1
    • 192-168-1-250.sslip.io resolves to 192.168.1.250
    • www.192-168-0-1.sslip.io resolves to 192.168.0.1
    • –1.sslip.io resolves to ::1
    • 2a01-4f8-c17-b8f--2.sslip.io resolves to 2a01:4f8:c17:b8f::2

Self-hosted Options

  • hipio is a Haskell service for IPv4.


ZMap Project

 https://zmap.io/

DNS | IP | Network | Tool

Different utilities for network scanning. Most imporantly the zmap component, which is a packet scanner for different protocols. It also contains other tools like ways to iterate over the IPv4 address space and blacklist/whitelist management.



dnskv: DNS-based Key-Value Storage

 https://dnskv.com/

DNS | Tool

This is a custom DNS server which allows setting and retrieving text based data. New values can be written as subdomains and retrieved via a normal TXT lookup.

  • To set a key: dig my-value.my-key.dnskv.com txt +short
  • To get a key: dig my-key.dnskv.com txt +short

The service provides many extra options like setting an expiry time or the TTL.







k-v.io: DNS-based Key-Value Store

 https://k-v.io/

DNS | Tool

This is a custom DNS server which allows setting and retrieving text based data. New values can be written as subdomains and retrieved via a normal TXT lookup.

  • To set a key: dig @ns.sslip.io put.my-value.my-key.k-v.io txt +short
  • To get a key: dig @ns.sslip.io my-key.k-v.io txt +short
  • To delete a key: dig @ns.sslip.io delete.my-key.k-v.io txt +short


mess with dns

 https://messwithdns.net/

DNS | Tool

"mess with dns" is a tool which allows you to experiment with DNS. The website allows creation of resource records of many types. They are all within a custom 3rd level domain. The website also shows the DNS requests the authoritative DNS server received.