Collector Plugins

Collectors are plugins responsible for gathering resource information from remote API’s, such as the AWS or GCP APIs.

Probator ships with AWS collectors built-in, but its relatively trivial to implement your own collectors as well. To write your own collector, you’ll need to implement the BaseCollector class, which is documented in more details below.

Base Collector

The base collector will require you to implement a number of attributes and methods on your class.

Attributes and methods

• name - string attribute
• ns - string attribute
• type - CollectorType attribute
• interval - integer attribute
• options - ConfigOption list of attribute
• run - method

name

This is the human-readable name of the collector, for example MyResourceCollector.

ns

The configuration namespace where the settings for the plugin is created, if applicable. The namespace should use a valid python variable identifier, so it should not contain any spaces or non-ASCII characters and you should use snake_case for the namespace.

It’s also considered best practices to prefix the namespace with the plugin type. If our collector is called MyResourceCollector we would make the namespace collector_my_resource.

type

The type of collector you are creating, which must be one of the probator.constants:CollectorType values. If you use any other type, the collector will be loaded but never executed.

The currently supported collector types are:

• CollectorType.GLOBAL

  • A collector plugin that is not connected to any account resources. The DNS Collector is an example of a global collector

• CollectorType.AWS_REGION

  • A collector for gathering AWS Region specific resources, such as EC2 Instances

• CollectorType.AWS_ACCOUNT

  • A collector for gathering AWS Account specific resources, such as IAM Users

interval

This setting controls how frequently the collector is executed, in minutes

options

If supplied, contains a list of all the configuration options for the plugin. The core framework automatically adds any missing configuration items on startup. The options variable must contain a list of probator.constants:ConfigOption objects.

run

The run method is the entry-point for the collector, however the method signature varies slightly based on the type of collector you are writing.

CollectorType	Signature	Arguments
GLOBAL	run()	No arguments
AWS_ACCOUNT	run(account)	account - An instance of AWSAccount object
AWS_REGION	run(account, region)	account - An instance of AWSAccount<br> region - Name of the region to collect from

AWS Collectors

The AWS Collector is the backbone of the core framework and it is shipped as part of the core framework and not a standalone plugin. All collectors are subclasses of the Base Collector

This is actually a collection of many individual collector plugins, one for each type of resource that is being collected. There are two base types of collectors for AWS

• AWS Account Collectors
• AWS Region Collectors

AWS Account Collectors

The AWS Account Collectors are responsible for collecting resources that are not specific to a given AWS region. Below is a list of the currently implemented account wide collectors:

Collector Class	Resources collected
AWSBaseAccountCollector	The base collector that all account collectors must be a subclass of
AWSRoute53Collector	Route53 zones and records
AWSS3Collector	S3 Buckets
AWSCloudFrontCollector	CloudFront distributions
AWSIAMUserCollector	IAM Users and Access Keys

AWS Region Collectors

The AWS Region collectors collect resources only within the provided region. The list below shows what collectors are shipped as part of the core framework:

Collector Class	Resources collected
AWSEC2InstanceCollector	EC2 Instances
AWSEBSVolumeCollector	EBS Volumes
AWSEBSSnapshotCollector	EBS Snapshots
AWSAMICollector	Amazon Machine Images (AMIs)
AWSBeanStalkCollector	Elastic BeanStalks (web stalks only)
AWSVPCCollector	VPCs
AWSClassicLoadBalancerCollector	Classic Load Balancers (ELB)
AWSLoadBalancerCollector	Application Load Balancers (ALB and NLB)
AWSENICollector	Elastic Network Interfaces (ENIs)
AWSRDSCollector	RDS Database Instances
AWSEKSCollector	EKS Clusters (control-plane)