Helper Methods

Sections

• Filtering
• Date & Time
• Information
• Other

Filtering

filter_by_timestamp

static curator.filter_by_timestamp(object_list=[], timestring=None, time_unit='days', older_than=999999, prefix='logstash-', suffix='', snapshot_prefix='curator-', utc_now=None, **kwargs)

Pass in a list of indices or snapshots. Return a list of objects older than n time_units matching prefix, timestring, and suffix.

Parameters:

• object_list – A list of indices or snapshots
• timestring – An strftime string to match the datestamp in an index name.
• time_unit – One of hours, days, weeks, months. Default is days
• older_than – Indices older than the indicated number of whole time_units will be operated on.
• prefix – A string that comes before the datestamp in an index name. Can be empty. Wildcards acceptable. Default is logstash-.
• suffix – A string that comes after the datestamp of an index name. Can be empty. Wildcards acceptable. Default is empty, ''.
• snapshot_prefix – Override the default with this value. Defaults to curator-
• utc_now – Used for testing. Overrides current time with specified time.

Return type:

generator object (list of strings)

filter_by_space

static curator.filter_by_space(client, disk_space=2097152.0, prefix='logstash-', suffix='', exclude_pattern=None, **kwargs)

Yield a list of indices to delete based on space consumed, starting with the oldest.

Parameters:

• client – The Elasticsearch client connection
• disk_space – Delete indices over n gigabytes, starting from the oldest indices.
• prefix – A string that comes before the datestamp in an index name. Can be empty. Wildcards acceptable. Default is logstash-.
• suffix – A string that comes after the datestamp of an index name. Can be empty. Wildcards acceptable. Default is empty, ''.
• exclude_pattern – Exclude indices matching the provided regular expression.

Return type:

generator object (list of strings)

Date & Time

get_date_regex

static curator.get_date_regex(timestring)

Return a regex string based on a provided strftime timestring.

Parameters:timestring – An strftime pattern Return type:Regex as string

get_index_time

static curator.get_index_time(index_timestamp, timestring)

Return the datetime extracted from the index name, which is the index creation time.

Parameters:

• index_timestamp – The timestamp extracted from an index name
• timestring – An strftime pattern

Return type:

Datetime object

get_target_month

static curator.get_target_month(month_count, utc_now=None)

Return datetime object for # of full months older than month_count from now, or utc_now, if provided.

Parameters:

• month_count – Number of full months
• utc_now – Used for testing. Overrides current time with specified time.

Return type:

Datetime object

get_cutoff

static curator.get_cutoff(older_than=999999, time_unit='days', utc_now=None)

Find the cutoff time based on older_than and time_unit.

Parameters:

• older_than – time_unit multiplier
• time_unit – One of hours, days, weeks, months. Default is days
• utc_now – Used for testing. Overrides current time with specified time.

Return type:

Datetime object

Information

get_alias

static curator.get_alias(client, alias)

Return information about the specified alias.

Parameters:

• client – The Elasticsearch client connection
• alias – Alias name to operate on.

Return type:

list of strings

get_indices

static curator.get_indices(client, prefix='logstash-', suffix='', exclude_pattern=None)

Return a sorted list of indices matching prefix and suffix and optionally filtered by exclude_pattern.

Parameters:

• client – The Elasticsearch client connection
• prefix – A string that comes before the datestamp in an index name. Can be empty. Wildcards acceptable. Default is logstash-.
• suffix – A string that comes after the datestamp of an index name. Can be empty. Wildcards acceptable. Default is empty, ''.
• exclude_pattern – Exclude indices matching the provided regular expression.

Return type:

list of strings

get_object_list

static curator.get_object_list(client, data_type='index', prefix='logstash-', suffix='', repository=None, snapshot_prefix='curator-', exclude_pattern=None, **kwargs)

Return a list of indices matching prefix and suffix or snapshots matching snapshot_prefix.

Can optionally exclude by regular expression pattern with exclude_pattern.

Parameters:

• client – The Elasticsearch client connection
• data_type – Either index or snapshot
• prefix – A string that comes before the datestamp in an index name. Can be empty. Wildcards acceptable. Default is logstash-.
• suffix – A string that comes after the datestamp of an index name. Can be empty. Wildcards acceptable. Default is empty, ''.
• repository – The Elasticsearch snapshot repository to use (only with snapshots)
• snapshot_prefix – Override the default with this value. Defaults to curator-
• exclude_pattern – Exclude indices matching the provided regular expression.

Return type:

list of strings

get_repository

static curator.get_repository(client, repository='')

Return configuration information for the indicated repository.

Parameters:

• client – The Elasticsearch client connection
• repository – The Elasticsearch snapshot repository to use

Return type:

dict

get_segmentcount

static curator.get_segmentcount(client, index_name)

Return a tuple of (shardcount, segmentcount) from the provided index_name.

Parameters:

• client – The Elasticsearch client connection
• index_name – The index name

Return type:

tuple

get_snaplist

static curator.get_snaplist(client, repository='', snapshot_prefix='curator-')

Get _all snapshots containing snapshot_prefix from repository and return a list.

Parameters:

• client – The Elasticsearch client connection
• repository – The Elasticsearch snapshot repository to use
• snapshot_prefix – Override the default with this value. Defaults to curator-

Return type:

list of strings

get_snapshot

static curator.get_snapshot(client, repository='', snapshot='')

Return information about a snapshot (or a comma-separated list of snapshots)

Parameters:

• client – The Elasticsearch client connection
• repository – The Elasticsearch snapshot repository to use
• snapshot – The snapshot name, or a comma-separated list of snapshots

Return type:

dict

get_version

static curator.get_version(client)

Return the ES version number as a tuple. Omits trailing tags like -dev, or Beta

Parameters:client – The Elasticsearch client connection Return type:tuple

index_closed

static curator.index_closed(client, index_name)

Return True if the indicated index is closed.

Parameters:

• client – The Elasticsearch client connection
• index_name – The index name

Return type:

bool

is_master_node

static curator.is_master_node(client)

Return True if the connected client node is the elected master node in the Elasticsearch cluster, otherwise return False.

Parameters:client – The Elasticsearch client connection Return type:bool

Other

_op_loop

static curator._op_loop(client, object_list, op=None, dry_run=False, **kwargs)

Perform the op on indices or snapshots in the object_list

Note

All kwargs are passed on to the specified op.

Any arg the op needs must be passed in kwargs.

Parameters:

• client – The Elasticsearch client connection
• object_list – The list of indices or snapshots
• op – The operation to perform on each object in object_list
• dry_run – If true, simulate, but do not perform the operation
• delay – Can be specified to pause after each iteration in the loop. Useful to allow cluster to quiesce after heavy I/O optimizations.