The package allows you query time-series data and statistics from Axibase Time-Series Database (ATSD). List of package functions:
Execute library(atsd) to start working with atsd package.
The connection parameters are loaded from the package configuration file,
atsd/connection.config,
which is located in atsd package folder.
The command
installed.packages()["atsd", "LibPath"]
shows you where atsd package folder is. Open a text editor and modify the configuration file. It should look as follows:
# the url of ATSD including port number
url=http://host_name:port_number
# the user name
user=atsd_user_name
# the user's password
password=atsd_user_password
# validate ATSD SSL certificate: yes, no
verify=no
# cryptographic protocol used by ATSD https server:
# default, ssl2, ssl3, tls1
encryption=ssl3
Reload the modified connection parameters from the configuration file:
set_connection()
Check that parameters are correct:
show_connection()
Refer to Chapter 8 for more options on managing ATSD connection parameters.
Description: The function retrieves historical time-series data or forecasts from ATSD.
Returns object: data frame
Arguments:
metric (required, string)
The name of the metric you want to get data for,
for example, “disk_used_percent”.
To obtain a list of metrics collected by ATSD use the
get_metrics() function.
selection_interval (required, string)
This is the time interval for which the data will be selected.
Specify it as “n-unit”, where
unit is a Second, Minute, Hour, Day, Week, Month, Quarter, or Year
and n is the number of units,
for example, “3-Week” or “12-Hour”.
entity (optional, string)
The name of the entity you want to get data for.
If not provided, then data for all entities will be fetched
for the specified metric.
Obtain the list of entities with the
get_entities() function.
entity_group (optional, string)
The name of entity group, for example, “HP Servers”.
Extracts data for all entities belonging to this group.
tags (optional, string vector)
List of user-defined series tags to filter the fetched time-series data,
for example, c(“disk_name=sda1”, “mount_point=/”) .
end_time (optional, string)
The end time of the selection interval, for example, 'date(2014-12-27)'.
If not provided, the current time will be used.
Specify the date and time, or use one of the supported expressions:
end time syntax.
For example, 'current_day' would set the end of selection interval
to 00:00:00 of the current day.
aggregate_interval (optional, string)
The length of the aggregation interval.
The period of produced time-series will be equal to the
aggregate_interval.
The value for each period is computed by the
aggregate_statistics
function applied to all samples of the original time-series within the period.
The format of the
aggregate_interval
is the same as for the
selection_interval
argument, for example, “1-Minute”.
aggregate_statistics (optional, string vector)
The statistic functions used for aggregation.
Multiple values are supported, for example, c(“Min”, “Avg”, “StDev”).
The default value is “Avg”.
interpolation (optional, string)
If aggregation is enabled, then the values for the periods without data
will be computed by one of the following interpolation functions:
“None”, “Linear”, “Step”. The default value is “None”.
export_type (optional, string)
Supported options: “History” or “Forecast”. The default value is “History”.
verbose (optional, string)
If verbose = FALSE, then all console output will be suppressed.
By default, verbose = TRUE.
Examples:
# get historic data for the given entity, metric, and selection_interval
dfr <- query(entity = "nurswgvml007", metric = "cpu_busy", selection_interval = "1-Hour")
# end_time usage example
query(entity = "host-383", metric = "cpu_usage", selection_interval = "1-Day",
end_time = "date('2015-02-10 10:15:03')")
# get forecasts
query(metric = "cpu_busy", selection_interval = "30-Minute", export_type = "Forecast", verbose = FALSE)
# use aggregation
query(metric = "disk_used_percent", entity_group = "Linux", tags = c("mount_point=/boot",
"file_system=/dev/sda1"), selection_interval = "1-Week", aggregate_interval = "1-Minute",
aggregate_statistics = c("Avg", "Min", "Max"), interpolation = "Linear", export_type = "Forecast")
Description: The function constructs a zoo object based on the 'Timestamp' and 'Value' columns of the given data frame. Information from other columns will be ignored.
Returns object: zoo object
Arguments:
Examples:
# query ATSD for data and transform it to zoo object
dfr <- query(entity = "nurswgvml007", metric = "cpu_busy", selection_interval = "1-Hour")
z <- to_zoo(dfr)
# show head of the zoo object
head(z, 3)
#> 2015-03-16 13:41:34 2015-03-16 13:41:50 2015-03-16 13:42:06
#> 11.22 5.05 6.06
Description: This function fetches a list of metrics and their tags from ATSD, and converts it to a data frame.
Returns object: data frame
Each row of the data frame corresponds to a metric and its tags:
name
Metric name (unique)
counter
Counters are metrics with continuously incrementing value
lastInsertTime
Last time value was received by ATSD for this metric
tags
User-defined tags (as requested by the “tags” argument)
Arguments:
expression (optional, string)
Select metrics matching particular name pattern and/or user-defined metric tags.
For examples refer to “Expression syntax” chapter.
active (optional,
one of strings: “true” or “false”)
Filter metrics by lastInsertTime attribute.
If active = “true”,
only metrics with positive lastInsertTime are included in the response.
tags (optional, string vector)
User-defined metric tags to be included in the response.
By default, all the tags will be included.
limit (optional, integer)
If limit > 0, the response shows the top-N metrics ordered by name.
verbose (optional, string)
If verbose = FALSE, then all console output will be suppressed.
Examples:
# get all metrics and include all their tags in the data frame
metrics <- get_metrics()
colnames(metrics)
#> [1] "name" "counter" "label" "lastInsertTime"
#> [5] "tags.table" "tags.source"
metrics[1, ]
#> name counter label lastInsertTime tags.table tags.source
#> 1 %_privileged_time FALSE <NA> <NA> <NA> <NA>
# get the first 100 active metrics which have the tag, "table",
# include this tag into response and exclude oter user-defined metric tags
metrics <- get_metrics(expression = "tags.table != ''", active = "true", tags = "table", limit = 100)
tail(metrics$name)
#> [1] "nmon.logical_partition.sharedcpu"
#> [2] "nmon.logical_partition.usedallcpu%"
#> [3] "nmon.logical_partition.usedpoolcpu%"
#> [4] "nmon.logical_partition.virtualcpus"
#> [5] "nmon.logical_partition.vp_idle%"
#> [6] "nmon.logical_partition.vp_sys%"
Description: This function fetches a list of entities and their tags from ATSD, and converts it to a data frame.
Returns object: data frame
Each row of the data frame corresponds to an entity and its tags:
name
Entity name (unique)
enabled
Enabled status, incoming data is discarded for disabled entities
lastInsertTime
Last time value was received by ATSD for this entity
tags
User-defined tags (as requested by the “tags” argument)
Arguments:
expression (optional, string)
Select entities matching particular name pattern and/or user-defined entity tags.
For examples refer to “Expression syntax” chapter.
active (optional,
one of strings: “true” or “false”)
Filter entities by lastInsertTime attribute.
If active = “true”,
only entities with positive lastInsertTime are included in the response.
tags (optional, string vector)
User-defined entity tags to be included in the response.
By default, all the tags will be included.
limit (optional, integer)
If limit > 0, the response shows the top-N entities ordered by name.
verbose (optional, string)
If verbose = FALSE, then all console output will be suppressed.
Examples:
# get all entities
entities <- get_entities()
names(entities)
#> [1] "name" "enabled" "tags.test" "lastInsertTime"
#> [5] "tags.app" "tags.ip" "tags.os" "tags.loc_area"
#> [9] "tags.loc_code" "tags.uuu.tag" "tags.uuu.tag.1"
nrow(entities)
#> [1] 195
# select entities by name and user-defined tag "app"
entities <- get_entities(expression = "name like 'nur*' and lower(tags.app) like '*hbase*'")
entities$name
#> [1] "nurswgvml006" "nurswgvml203" "nurswgvml204" "nurswgvml205"
#> [5] "nurswgvml206" "nurswgvml207" "nurswgvml208"
In this section, we explain the syntax of the
expression
argument of the functions get_metrics() and get_entities().
The expression
is used to filter result for which
expression evaluates to TRUE .
The variable name is used to select metrics/entities by names:
# get metric with name 'actions_per_minute'
get_metrics(expression = "name = 'actions_per_minute'", verbose = FALSE)
Metrics and entities have user-defined tags.
Each of these tags is a pair (“tag_name” : “tag_value”).
The variable tags.tag_name in an expression refers to the tag_value
for given metric/entity.
If a metric/entity does not have this tag, the tag_value will be an empty string.
# get metrics without 'source' tag, and include all tags of fetched metrics in output
get_metrics(expression = "tags.source != ''", tags = "*")
To get metrics with a user-defined tag 'table' equal to 'System':
# get metrics whose tag 'table' is equal to 'System'
metrics <- get_metrics(expression = "tags.table = 'System'", tags = "*")
#> Your request was successfully processed by server. Start parsing and filtering.
#> Parsing and filtereng done. Start converting to data frame.
#> Converting to data frame done.
To build more complex expressions, use brackets (, ), and
and, or, not logical operators as well as && , ||, !.
entities <- get_entities(expression = "tags.app != '' and (tags.os != '' or tags.ip != '')")
#> Your request was successfully processed by server. Start parsing and filtering.
#> Parsing and filtereng done. Start converting to data frame.
#> Converting to data frame done.
To test if a string is in a collections, use in operator:
get_entities(expression = "name in ('derby-test', 'atom.axibase.com')")
Use like operator to match values with expressions containing wildcards:
expression = "name like 'disk*'" .
The wildcard * mean zero or more characters.
The wildcard . means any one character.
metrics <- get_metrics(expression = "name like '*cpu*' and tags.table = 'System'")
# get metrics with names consisting of 3 letters
metrics <- get_metrics(expression = "name like '...'")
# print names of fetched metrics
print(metrics$name)
#> [1] "ask" "bid" "jmx"
There are additional functions you can use in an expression:
list(string, delimeter))
Splits the string by delimeter. The default delimiter is a comma.
upper(string)
Converts the string argument to upper case.
lower(string)
Converts the string argument to lower case.
collection(name)
Refers to a named collection of strings created in ATSD.
likeAll(string, collection of patterns)
Returns true if every element in the collection of patterns matches the given string.
likeAny(string, collection of patterns)
Returns true if at least one element in the collection of patterns matches the given string.
\code{name in list('actions_per_minute,alerts_total,cpu_usage')}
get_metrics(expression = "likeAll(lower(name), list('cpu*,*use*'))")
get_metrics(expression = "likeAny(lower(name), list('cpu*,*use*'))")
get_metrics(expression = "name in collection('fs_ignore')")
The atsd package uses connection parameters to connect with ATSD. These parameters are:
url - the url of ATSD including port number
user - the user name
password - the user's password
verify - should ATSD SSL certificate be validated
encryption - cryptographic protocol used by ATSD https server
The configuration parameters are loaded from the package configuration file when you load the atsd package into R. (See Section 2.)
The functions show_connection(), set_connection(),
and save_connection(),
show configuration parameters, change them,
and store them in the configuration file.
Function name: show_connection()
Returns object: NULL
Description: The function prints current values of the connection parameters. (They may be different from the values in the configuration file.)
Arguments: no
Examples:
show_connection()
#> url = NA
#> user = NA
#> password = NA
#> verify = no
#> encryption = ssl3
Function name: set_connection()
Returns object: NULL
Description: The function overrides the connection parameters for the duration of the current R session without changing the configuration file. If called without arguments the function sets the connection parameters from the configuration file. If the file argument is provided the function use it. In both cases the current values of the parameters became the same as in the file. In case the file argument is not provided, but some of other arguments are specified, the only specified parameters will be changed.
Arguments:
url (optional, string)
The url of ATSD including port number.
user (optional, string)
The user name.
password (optional, string)
The user's password.
verify (optional, string)
String - “yes” or “no”,
verify = "yes" ensures validation of ATSD SSL certificate
and verify = "no" suppresses the validation
(applicable in the case of 'https' protocol).
encryption (optional, string)
Cryptographic protocol used by ATSD https server.
Possible values are: “default”, “ssl2”, “ssl3”, and “tls1”
(In most cases, use “ssl3” or “tls1”.)
file (optional, string)
The absolute path to the file from which
the connection parameters could be read.
The file should be formatted as the package configuration file,
see Section 2.
Examples:
# Modify the user
set_connection(user = "user001")
# Modify the cryptographic protocol
set_connection(encryption = "tls1")
show_connection()
#> url = NA
#> user = user001
#> password = NA
#> verify = no
#> encryption = tls1
# Set the parameters of the https connection: url, user name, password
# should the certificate of the server be verifyed
# which cryptographic protocol is used for communication
set_connection(url = "https://my.company.com:8443", user = "user001", password = "123456",
verify = "no", encryption = "ssl3")
show_connection()
#> url = https://my.company.com:8443
#> user = user001
#> password = 123456
#> verify = no
#> encryption = ssl3
# Set up the connection parameters from the file:
set_connection(file = "/home/user001/atsd_https_connection.txt")
Function name: save_connection()
Returns object: NULL
Description: The function writes the connection parameters into the configuration file. If called without arguments the functions use current values of the connection parameters (including NAs). Otherwise only the provided arguments will be written to the configuration file. If configuration file is absent it will be created in the atsd package folder. Arguments:
url (optional, string)
The url of ATSD including port number.
user (optional, string)
The user name.
password (optional, string)
The user's password.
verify (optional, string)
String - “yes” or “no”,
verify = "yes" ensures validation of ATSD SSL certificate
and verify = "no" suppresses the validation
(applicable in the case of 'https' protocol).
encryption (optional, string)
Cryptographic protocol used by ATSD https server.
Possible values are: “default”, “ssl2”, “ssl3”, and “tls1”
(In most cases, use “ssl3” or “tls1”.)
Examples:
# Write the current values of the connection parameters to the configuration file.
save_connection()
# Write the user name and password in the configuration file.
save_connection(user = "user00", password = "123456")
# Write all parameters nedeed for the https connection to the configuration file.
save_connection(url = "https://my.company.com:8443", user = "user001", password = "123456",
verify = "no", encryption = "ssl3")