Historic datasets (from 2014 onwards) for the .nl TLD. Datasets are available in JSON format.

Datasets cover information about:

• DNS
  • Domain Names
  • Query Type
  • Response Codes
  • IPv6 Support
• Resolvers
  • Location
  • Number of IP addresses
  • Validating Resolvers
  • Popular Networks
  • Port Randomness
• DNSSEC
  • Validating Queries
  • DANE
  • Used Algorithms
• Mail
  • Mail Resource Records (RRs)
  • SPF Information

APNIC gathers many statistics and offers them on their website. However, they provide way more data than it might initially look like, since many of the datasets are not linked from their main page.

• BGP announcements by region
• Customers by ASN
• DNSSEC validation rate by region
• IPv4 vs. IPv6 RTT Comparison by region
• IPv6 Customers by ASN
• IPv6 usage by region
• Statistics about their ad-based measurement system
• Use of open DNS resolver by region

The website contains information about the cz. TLD operated by CZ.NIC. It contains information about the query volume, query type, round-trip time (RTT) and geographic location of the traffic sources. It also has information about the registry functions, such as registrar information, domain transfers or whois requests. Lastly, information about the mojeID accounts, a login provider operated by CZ.NIC is also available.

The website allows executing DNS queries from various locations worldwide to check the returned values.

Cloudflare Radar is Cloudflare's reporting website about internet trends and general traffic statistics. The website shows information about observed attacks and attack types and links to the DDoS report. General traffic statistics are reported, such as the used browser, fraction of human traffic, IP, HTTP, and TLS version.

The website also provides more detailed information on domains and IP addresses. Domains have information about age, popularity, and visitors. IP addresses have ASN and geolocation information.

• https://radar.cloudflare.com/ip/134.96.0.0

More information about Cloudflare Radar is available in the introduction blog post.

The Radar data is also available via API, for example the attack data: https://developers.cloudflare.com/api/resources/radar/subresources/attacks/subresources/layer3/subresources/summary/

Collection of "bad" packets in PCAPs that can be used for testing software.

C-DNS is a space efficient file format for storing DNS traffic. It is based on CBOR with a fixed schema as defined in RFC 8618.

Some tools exist to work with C-DNS files. The dns-stats compactor can convert PCAP-to-CDNS and vice versa. It can be configured flexibly to only record those fields of the DNS message which are of interest, thus saving space in the compacted format.

https://www.statdns.com/resources/

Contains lists for DNS servers, libraries, tools, and other resources.

https://linuxmafia.com/kb/Network_Other/dns-servers.html

List of maintained and unmaintained DNS servers, including descriptions for each of them.

https://dnsinstitute.com/implementations/

Short overview of open-source projects for authoritative and recursive servers and development libraries.

Provides an outdated list of different Cyber Thread Intelligence Feeds of other organizations.

DMAP is a scalable web scanning suit which supports DNS, HTTPS, TLS, and SMTP. It works based on domain names and crawls the domain for all supported protocols. The advantage over other tools is the unified SQL data model with 166 features and the easy scalability over many crawling machines.

The website is an ongoing project by Knot DNS to measure the performance of various DNS servers. Four open-source servers are tested, namely BIND, Knot DNS, NSD, and PowerDNS. The benchmark includes different zone configurations matching to root zones, TLD zones, or hosting zones as well as different DNSSEC configurations.

The DNS Census 2013 consists of about 2.5 billion DNS records collected in 2012/2013. The data is gathered from some available zone files and passive or active DNS collecting. The DNS records are written into CSV files containing one DNS record per line.

DNS Coffee collects and archives stats from DNS Zone files in order to provide insights into the growth and changes in DNS over time.

The website includes information such as the size of different zones. It tracks over 1200 zone files.

It provides searching through the zones files based on domain names, name servers, or IP addresses. It can also visualize the relationship between a domain, the parent zones and the name server in what they call a "Trust Tree".

The DNS Core Census is an ICANN project to gather information about top-level-domains (TLDs). This covers ccTLDs, gTLDs, effective TLDs (like co.uk), and entries in arpa. The census contains information about the zone, like metadata and contractual information, about the name servers, about addresses of the name servers, and the route origins. The data is kept for a 35-day rolling window.

Further information about the project can be found in this presentation and OCTO-019 from ICANN's Chief Technology Officer

The DNS Privacy Project aims to improve privacy for users on the Internet.

The project is split into different groups working on DNS privacy:

• IETF DPRIVE Working Group
• NDSS DNS Privacy Workshops

The project focuses mostly on DNS over TLS. They provide overviews for the implementation status, configuration for test servers, and ongoing server monitoring, which features they provide.

Check My DNS

Browser-based DNS resolver quality measurement tool. Uses the browser to generate many resolver queries and tests for features they should have, such as EDNS support, IPv6, QNAME Minimization, etc.

This test is also available as a CLI tool: https://github.com/DNS-OARC/cmdns-cli

DNSSEC Debugger

Analyze DNSSEC deployment for a zone and show errors in the configuration.

DNSViz

Gives an overview of DNSSEC delegations, response sizes, and name servers.

GitHub: https://github.com/dnsviz/dnsviz

DNS X-Ray

The website has an online test, which performs DNS lookups. These DNS lookups test if certain resource records are overwritten in the cache. The tool can then determine what DNS software is used, where the server is located, how many caches there are, etc.

EDNS Compliance Tester

Test name server of zones for correct EDNS support.

The Transitive Trust and DNS Dependency Graph Portal

Shows the trust dependencies in DNS. Given a domain name, it can show how zones delegate to each other and why. The delegation is done between IP addresses and zones.

Root Canary Project

The project used to monitor the first root KSK key rollover. Now it contains the paper "Roll, Roll, Roll your Root: A Comprehensive Analysis of the First Ever DNSSEC Root KSK Rollover" describing the experiences of the first root KSK rollover

Additionally, it includes a tester for DNSSEC algorithm support, which shows the algorithms supported by the currently used recursive resolver. It provides statistics about support for DNSSEC algorithms. It has a web-based test to test your own resolver and provides a live monitoring using the RIPE Atlas.

DNSSEC algorithms resolver test

This dataset covers approximately 3.5 billion DNS queries that were received at one of SURFnet's authoritative DNS servers from Google's Public DNS Resolver. The queries were collected during 2.5 years. The dataset contains only those queries that contained an EDNS Client Subnet.

The dataset covers data from 2015-06 through 2018-01.

DOI Identifier

Tool to replay DNS queries captured in a PCAP file with accurate timing between queries. Allows modifying the replay, like changing IP addresses, speeding up or slowing down the queries.

The weberdns.de zone hosts many weird DNS records which are helpful in testing and configurations and see how they handle these entries.

Some examples of hosted entries:

• ttl-0s.weberdns.de with a TTL of 0 seconds
• loop.weberdns.de a CNAME loop of length 2
• 64a.weberdns.de with 64 A records
• many-rrs.weberdns.de with a wide mix of record types for ANY queries
• abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz1234567890.weberdns.de for long domains and labels
• áàăâåäãąąāæćĉčċçďđéèĕêěëėęēğĝġģ.weberdns.de to test umlauts
• Domains for SRV, HINFO, CAA, LOC, TLSA, SSHFP, and OPENPGPKEY records

DNS Toys is an authoritative DNS server offering different unit conversion and lookup tools.

Some examples from the website include:

# Lookup time by city name
dig newyork.time @dns.toys

# or weather
dig newyork.weather @dns.toys

# Return the client IP address
dig ip @dns.toys

# Number conversion from decimal to hex
dig 100dec-hex.base @dns.toys

The website hosts the zone data for a couple of DNS zones, mainly some ccTLDs. This provides a good starting point for zone file analysis together with other sources.

DNS network capture utility. Similar in concept to tcpdump, but with specialized options for DNS.

Historical DNS database. Contains information recorded at recursive resolver about domain names, first/last seen, current bailiwick. Allows to see the lifetime of resource records and can be used as a large database.

Historical information about the reachability of root and some TLD name servers.

The Internet Society published maps showing the distribution of IPv6 support worldwide. The maps are available also with historic data, but are only updated sporadically. More current maps and CSV files are available on the mailing list.

Regularly updated reports about the current DNSSEC deployment. Contains information per TLD and global distribution.

• https://rick.eng.br/dnssecstat/
• https://stats.dnssec-tools.org/

The website keeps track of all DNSSEC keys in the top-level domains (TLDs) and informs when the signatures are about to expire. The time before some RRSIGs expire is color coded. It also shows error which happened during validation.

Top-like utility showing information about captured DNS requests. It shows information about the domains queries, the types, and responses.

DNSThought lists many statistics about the resolvers visible to the .nl-authoritative name servers. The data is gathered from the RIPE Atlas probes. There is a dashboard which only works partially.

Raw data access is also available.

The website provides a simple interface for dig. You can specify the various CLI arguments of dig with a web-based form.

The website provides a simple interface for dig. You can specify the various CLI arguments of dig with a web-based form.

Domain popularity lists provide a starting point for crawling domains with the most users. The most commonly used list for security research is the Alexa list.

• CISCO Umbrella
  The list is updated daily and contains one million websites. The ranking is based on traffic seen on the OpenDNS resolvers.
• Majestic
  The list is updated daily and contains one million websites. The ranking is based on backlinks from other websites.
• Tranco
  A Research-Oriented Top Sites Ranking Hardened Against Manipulation
  The Tranco list aims to provide a better list for security research. The authors explain on their website and their paper what the flaws in the existing lists.
• Cloudflare Radar Cloudflare uses their 1.1.1.1 DNS resolver to create a top 1 million list. The lists are also available on a per country level, e.g., https://radar.cloudflare.com/domains/de. More details are available in their announcement blog post.
• CrUX Chrome Google Chrome collects the top 1 million visited website and published them as part of the Chrome UX Report. The repository captures the monthly data and provides access to older versions. In an Internet Measurement Conference (IMC) paper this list was shown to best correlate with the HTTP requests as seen by Cloudflare.

Internet Assigned Numbers Authority (IANA) tracks the assignment of all DNS related constants. This includes available classes, resource records types, OpCodes, RCODEs, and various other values. Each assigned number is linked to the corresponding RFC describing its function. This is a great overview to see which values are available.

Flamethrower is a small, fast, configurable tool for functional testing, benchmarking, and stress testing DNS servers and networks. It supports IPv4, IPv6, UDP, TCP, DoT, and DoH and has a modular system for generating queries used in the tests.

This dataset contains the responses to DNS requests for all forward DNS names known by Rapid7's Project Sonar. Until early November 2017, all of these were for the 'ANY' record with a fallback A and AAAA request if necessary. After that, the ANY study represents only the responses to ANY requests, and dedicated studies were created for the A, AAAA, CNAME and TXT record lookups with appropriately named files.

The data is updated every month. Historic data can be downloaded after creating a free account.

Some Open DNS operators provide a way to return the IP address of the request packet.

Google DNS:

dig o-o.myaddr.l.google.com txt @ns1.google.com +short

OpenDNS:

dig myip.opendns.com @resolver1.opendns.com +short

Akamai:

$ dig +short TXT whoami.ds.akahelp.net
"ns" "2001:db8::abcd"
"ecs" "203.0.113.0/24/0"
"ip" "203.0.113.132" 

The ds stands for dual stack. It is also reachable under whoami.ipv6.akahelp.net to force IPv6 or whoami.ipv4.akahelp.net to force IPv4.

ns stands for the unicast IP address of the recursive resolver. ecs contains the EDNS client subnet information if the resolver uses the option. In this case, ip contains a representative IP within the ECS, not necessarily the IP of the client.

The new version is introduced in this blog post https://www.akamai.com/blog/developers/introducing-new-whoami-tool-dns-resolver-information.

The old version might be decommissioned:

dig whoami.akamai.net. @ns1-1.akamaitech.net. +short

addr.tools:

$ curl myip.addr.tools
2a02:810b:4540:9e84::c1a4

$ curl myipv4.addr.tools
95.91.221.165

$ curl myipv6.addr.tools
2a02:810b:4540:9e84::c1a4

$ curl myip.addr.tools/json
{"ip":"2a02:810b:4540:9e84::c1a4"}

Source

Hello DNS is a project to write an easy to read/understand summary of the DNS specification. It provides an entry point to understand DNS given that the full DNS specification is easily 2000 pages in size.

ICANN tracks the general health of the DNS ecosystem and related ecosystems. The data is updated irregularly, but historic data is available. The collected data covers eight major topics:

1. M1: inaccuracy of Whois Data
2. M2: Domain Name Abuse
3. M3: DNS Root Traffic Analysis
4. M4: DNS Recursive Server Analysis
5. M5: Recursive Resolver Integrity
6. M6: IANA registries for DNS parameters
7. M7: DNSSEC Deployment.
8. M8: DNS Authoritative Servers Analysis

Each topic has too many sub categories to list here.

The Grafana dashboard shows live statistics about query volume, query type, and geographic locations. The data is collected for ICANN Managed Root Servers (IMRS) which are the L-root servers.

Internet Assigned Numbers Authority (IANA) tracks the assignment of all NetFlow/IPFIX related constants. The website lists all available fields and describes their meaning. Each assigned number is linked to the corresponding RFC describing its function. This is a great overview to see which values are available.

Maps of measurements done with the RIPE Atlas.

Contains information about the state of the RFCs and what kind of information they contain.

These projects either operate DNS based Real-time Blackhole Lists (RBL) or allow checking if an IP is contained. The Multi-RBL websites are helpful in finding a large quantity of RBLs.

• https://multirbl.valli.org/
  • Contains hundreds of RBLs, including descriptions of them and their return values.
• http://www.anti-abuse.org/multi-rbl-check/
• https://dnsbllookup.com
• https://www.dnsbl.info
• https://www.spamhaus.org/
  • Operates different RBLs such as for spam, exploits, domains with bad reputation, do-no-route ("drop-all-traffic"), and known spam operators.

The website collects major outages caused by broken DNSSEC deployments in the wild. It tracks root and TLD errors and some selected websites. Each outage is timestamped and has some description about the problem. The entries contain information from the VeriSign DNSSEC debugger and DNSViz.

The Measurement Factory performed a study of open DNS resolvers between 2006 and 2017. The website has an archive of daily reports, which each list the number of open resolver per ASN.

A free and configurable DNS resolver. It provides customizable blocking, such as for ads, trackers, or malicious websites. Additionally, statistics can be shown, such as for most blocked websites.

A similar self-hosted variant is Pi-hole.

The OWASP Amass tool suite obtains subdomain names by scraping data sources, recursive brute forcing, crawling web archives, permuting/altering names and reverse DNS sweeping. Additionally, Amass uses the IP addresses obtained during resolution to discover associated net blocks and ASNs. All the information is then used to build maps of the target networks.

Open INTEL is an active DNS database. It gathers information from public zone files, domain lists (Alexa, Umbrella), and reverse DNS entries. Once every 24 hours, data is collected about a bunch of DNS RRsets (SOA, NS, A, AAAA, MX, TXT, DNSKEY, DS, NSEC3, CAA, CDS, CDNSKEY). The data is openly available as AVRO files and dates back until 2016.

The data can be freely downloaded. There is documentation on the layout of the AVRO files.

The project is similar to Active DNS but seems to be larger in scope.

packetq is a command line tool to run SQL queries directly on PCAP files, the results can be outputted as JSON (default), formatted/compact CSV and XML. It also contains a very simplistic web-server to inspect PCAP files remotely,

Passive DNS dataset from circl.lu.

The Passive DNS Common Output Format describes a format used for querying passive DNS interfaces. The format is currently an IETF RFC draft. The format is used by CERT.at, Farsight, and CIRCL, as well as other projects.

A free and configurable DNS stub-resolver. It provides customizable blocking, such as for ads, trackers, or malicious websites. Additionally, statistics can be shown, such as for the most blocked website.

It can also function as a DHCP server for clients on the same network.

A similar service is NextDNS.

The website provides a curated list of various public DNS resolver operators and the IP addresses of the DNS servers.

The public suffix list gives a way to easily determine the effective second level domain, i.e., the domain which a domain owner registered and which can be under different owners.

RIPE operates a set of probes, which can be used to send pings or similar measurements. The probes are mainly placed in Europe, but some are also in other continents.

All the collected measurements can be found in the RIPE Atlas Daily Archives. The blog post gives some more details.

RIPEstat is a network statistics platform by RIPE. The platform shows data for IP addresses, networks, ASNs, and DNS names. This includes information such as the registration information, abuse contacts, blocklist status, BGP information, geolocation lookups, or reverse DNS names. Additionally, the website links to many other useful tools, such as an address space hierarchy viewer, historical whois information, and routing consistency checks.

RSSAC002 describes measurements for DNS root servers. It collects data, such as the load time, rcode volumes, traffic volume, and unique sources. The data is collected daily and goes back to 2013.

The data is also available in a git repository, which is not always up-to-date. https://github.com/rssac-caucus/RSSAC002-data

The site contains two PDFs showing the relationship between the DNS RFCs. There is a simplified overview and a full overview. The graphs are regularly updated. The RFCs are organized by time, by topic area (e.g., DNSSEC, RR), and the RFC status (e.g., Standard, Best Current Practice).

This repo describes a testbed to test various DNS resolvers. The purpose of the testbed is to allow researchers to set up many resolvers and run tests on each. For example, a test might see what the resolver emits when it is priming, or when it is responding to a particular query while using DNSSEC validation.

Overview page for the DNS root servers. It contains links to general news and all the supporting organizations.

The website features a map with all geographic locations. It contains information about locations, IPv4/IPv6 reachability and IP addresses.

Each root server has its own subdomain in the form of https://a.root-servers.org. It contains access to historical performance data like:

• Size and time of zone updates
• RCODE volume
• query and response sizes for UDP and TCP
• traffic volume (packets per time)
• Unique sources

Different information regarding reachability and connectivity of ASes.

The DNS workbench is a testbed which allows experimentation how different authoritative DNS servers answer to queries.

It covers five open-source authoritative servers, namely Bind9, Knot, NSD4, PowerDNS, and Yadifa. The workbench contains zones to test the support for many different resource record (RR) types, DNSSEC validation and how invalid zones are managed, delegations, zone transfers, and potentially more.

Find the project on GitHub.

The Shadowserver Scanning projects performs regular Internet wide scans for many protocols. The dashboard shows the gathered data about botnet sinkholes, Internet scans, honeypots, DDoS, and IoT data. This includes information about the size of botnets, the number of IP addresses with open ports like MySQL, the botnets as seen by honeypots, or the used protocols for DDoS attacks.

The blog post provides an introduction to the new dashboard.

The Shadowserver Scanning projects performs regular Internet wide scans for many protocols. They scan for four main types of protocols:

1. Amplification protocols, e.g., DNS or NTP
2. Botnet protocols, e.g., Gameover Zeus or Sality
3. Protocols that should not be exposed, e.g., Elasticsearch, LDAP, or RDP
4. Vulnerable Protocols, e.g., SSLv3

The website is a great resource to get general statistics about the protocols, like the number of hosts speaking the protocol, their geographic distribution, associated ASNs, and the historic information.

Shodan performs regular scan on common ports.

Access is free, but requires registration. More results can be gained with a paid account.

The blog post about Subdomain Enumeration in the APNIC blog provides a great overview of the techniques, defenses, and tools for it. Subdomain enumeration is the act of learning available subdomains in a zone using DNSSEC. This is with NSEC records and somewhat harder with NSEC3, due to hashing of names. The blog goes explains how online signing can combat subdomain enumeration, using the white lies or the black lies strategies. Lastly, it links to tools for performing these attacks.

The TLD Apex History is an ICANN project to gather DNSSEC related records for all TLDs.

Further information about the project can be found in this presentation.

Validin offers a free passive DNS search, but it requires an account. Free accounts accounts get access to 50 queries per month and four years worth of DNS history.

These services allow you to create a domain name for any IP address. The IP address is encoded into the domain name. An overview of different services can be found here.

Online Services

• https://nip.io/ provides IPv4 only

  • Supports both . and - separators.
  • 10.0.0.1.nip.io resolves to 10.0.0.1
  • 192-168-1-250.nip.io resolves to 192.168.1.250
  • customer1.app.10.0.0.1.nip.io resolves to 10.0.0.1
  • magic-127-0-0-1.nip.io resolves to 127.0.0.1

• https://sslip.io/ provides IPv4 and IPv6

  • Supports both . and - separators.
  • Provides the ability to use the service with your own branding.
  • 192.168.0.1.sslip.io resolves to 192.168.0.1
  • 192-168-1-250.sslip.io resolves to 192.168.1.250
  • www.192-168-0-1.sslip.io resolves to 192.168.0.1
  • –1.sslip.io resolves to ::1
  • 2a01-4f8-c17-b8f--2.sslip.io resolves to 2a01:4f8:c17:b8f::2

• https://ip.addr.tools/ provides IPv4 and IPv6

  • Supports both . and - separators.
  • 192.168.0.1.ip.addr.tools resolves to 192.168.0.1
  • 192-168-1-250.ip.addr.tools resolves to 192.168.1.250
  • www.192-168-0-1.ip.addr.tools resolves to 192.168.0.1
  • 2a01-4f8-c17-b8f--2.ip.addr.tools resolves to 2a01:4f8:c17:b8f::2

Self-hosted Options

• hipio is a Haskell service for IPv4.

Different utilities for network scanning. Most importantly, the zmap component, which is a packet scanner for different protocols. It also contains other tools like ways to iterate over the IPv4 address space and denylist/allowlist management.

The website provides download access to domains in many TLDs. Most lists are updated daily. However, not all the lists seem complete. For example, DENIC reports that they manage over 17 million domains, whereas zonefiles.io only reports over 6 million domains.

The zonemaster website runs various checks on the name servers for a zone. It covers various basics like IP reachability, consistency, diversity in the operators, various correctness tests like QNAME case insensitivity, and checking the quality of the EDNS setup. It is similar to Zonemaster.se.

The zonemaster website runs various checks on the name servers for a zone. It covers various basics like IP reachability, consistency, diversity in the operators, various correctness tests like QNAME case insensitivity, and checking the quality of the EDNS setup. It is similar to Zonemaster by cz.nic.

• https://challenges.addr.tools This is a helper tool for performing dns-01 ACME challenges. It serves the DNS validation and acts as a CNAME target.

• https://dyn.addr.tools Simple dynamic DNS server that can be used without an account.

• https://header-echo.addr.tools See the HTTP request headers. With URL parameters, the returned response can be customized.

• https://info.addr.tools Get information about IP addresses or domains.

  https://info.addr.tools/example.com
  https://info.addr.tools/1.1.1.1
  https://info.addr.tools/2606:4700:4700::1111
  https://info.addr.tools/me

• https://ip.addr.tools Return any IP address, e.g., 192-0-2-1.ip.addr.tools. Dynamic DNS updates for _acme-challenge are possible.

• https://myip.addr.tools Query your own IP address.

• https://www.dnscheck.tools/ Prints information about the currently used recursive DNS resolver and the DNSSEC implementation support.

• https://myaddr.tools Free dynamic DNS service.

dnsdumpster.com fetches a lot of DNS information belonging to one domain. It checks the authoritative name servers, which records exist, and where the servers are located.

This is a custom DNS server which allows setting and retrieving text-based data. New values can be written as subdomains and retrieved via a normal TXT lookup.

• To set a key: dig my-value.my-key.dnskv.com txt +short
• To get a key: dig my-key.dnskv.com txt +short

The service provides many extra options like setting an expiry time or the TTL.

DNS performance measurement tools.

dnsteal provides a fake DNS server and encodes a file into a series of DNS requests. The fake DNS server then reassembles the file. This can be used to hide the file exfiltration as DNS traffic, however, since it doesn't use the default DNS server it is quite noisy.

iodine allows to tunnel IPv4 traffic through a DNS server. This can be used if network access is restricted, but DNS is unfiltered, for example in when a captive portal is deployed.

This is a custom DNS server which allows setting and retrieving text-based data. New values can be written as subdomains and retrieved via a normal TXT lookup.

• To set a key: dig @ns.sslip.io put.my-value.my-key.k-v.io txt +short
• To get a key: dig @ns.sslip.io my-key.k-v.io txt +short
• To delete a key: dig @ns.sslip.io delete.my-key.k-v.io txt +short

"mess with dns" is a tool which allows you to experiment with DNS. The website allows creation of resource records of many types. They are all within a custom 3rd level domain. The website also shows the DNS requests the authoritative DNS server received.

DNS responses gathering and differences analysis toolchain.