Rapid7 FDNS ANY Dataset

Stream data with DDA:

import os, git
from dagshub.streaming import DagsHubFilesystem
url = 'https://<username>:<token>@dagshub.com/DagsHub-Datasets/rapid7-fdns-any-dataset.git'
git.Git("./").clone(url)
fs = DagsHubFilesystem("rapid7-fdns-any-dataset")
print(list(fs.scandir("rapid7-fdns-any-dataset")))
print(list(fs.listdir("rapid7-fdns-any-dataset/s3://rapid7-opendata/fdns/any/v1/")))

Description:

This dataset has been deprecated. Please see this Rapid7 blog post for details.

Contact:

This dataset has been deprecated. Please see this Rapid7 blog post for details.

Update Frequency:

Monthly

Managed By:

Not currently managed

Resources:

1. resource:

   • Description: FDNS ANY Gzip-compressed parquet files following a HIVE partitioning model, with a 90-day retention period. This dataset represents the data returned for a name when an ANY query is issued. Data is partitioned by date=YYYYMM which is the year-month that the FDNS ANY study was launched.

   • ARN: arn:aws:s3:::rapid7-opendata/fdns/any/v1/

   • Region: us-east-1

   • Type: S3 Bucket

2. resource:

   • Description: New FDNS parquet dataset announcements

   • ARN: arn:aws:sns:us-east-1:740874647437:s3-event-notification-new-aws-pds-dataset

   • Region: us-east-1

   • Type: SNS Topic

Tags:

aws-pds, computer security, cyber security, analytics, internet

Tutorials:

1. tutorial:

   • Title: How to Conduct DNS Reconnaissance for $.02 Using Rapid7 Open Data and AWS
   • URL: https://blog.rapid7.com/2018/10/16/how-to-conduct-dns-reconnaissance-for-02-using-rapid7-open-data-and-aws/
   • AuthorName: Shan Sikdar at Rapid7
   • AuthorURL: https://blog.rapid7.com/author/shan/

2. tutorial:

   • Title: Creating a Project Sonar FDNS API with AWS
   • URL: http://securityriskadvisors.com/blog/creating-a-project-sonar-fdns-api-with-aws/
   • AuthorName: Evan Perotti at SecurityRiskAdvisors
   • AuthorURL: https://twitter.com/2xxeformyshirt