阿里云Elasticsearch團隊自研的TimeStream時序引擎增強插件,支持使用PromQL查詢Elasticsearch數據,無縫對接Prometheus+Grafana。本文介紹如何基于Elasticsearch TimeStream使用Prometheus集成接口。
背景信息
TimeStream是阿里云Elasticsearch團隊自研,并結合Elastic社區時序類產品特性共建的時序引擎。阿里云Elasticsearch提供的TimeStream時序增強功能插件,優化了Elasticsearch在存儲指標數據方面的DSL(Domain-Specific Language)查詢復雜且慢以及存儲成本過高等問題。詳細信息請參見TimeStream時序增強引擎介紹。
前提條件
已創建阿里云Elasticsearch實例,且實例版本為7.10(內核版本為1.8.0及以上)或7.16及以上(內核版本為1.7.0及以上)。具體操作請參見創建阿里云Elasticsearch實例。
Prometheus API說明
Prometheus API除了可以訪問通過Prometheus Remote Write方式寫入的數據,也可以訪問任意TimeStream索引。
Prometheus API支持去掉返回的指標和維度的前后綴關鍵字。如果使用默認的數據模型,指標前綴metric.會被去掉,維度前綴label.會被去掉。
{
"labels": {
"instance": "127.0.0.1:9114",
"job": "elasticsearch"
},
"metrics": {
"up": 0.0,
"scrape_samples_post_metric_relabeling": 0.0,
"scrape_samples_scraped": 0.0,
"scrape_duration_seconds": 4.45999E-4,
"scrape_series_added": 0.0
},
"@timestamp": "1655717638795"
}
{
"status" : "success",
"data" : [
"__name__",
"instance",
"job"
]
}
{
"status" : "success",
"data" : {
"scrape_samples_post_metric_relabeling" : [
{
"type" : "gauge",
"help" : "",
"unit" : ""
}
],
"scrape_samples_scraped" : [
{
"type" : "gauge",
"help" : "",
"unit" : ""
}
],
"scrape_duration_seconds" : [
{
"type" : "gauge",
"help" : "",
"unit" : ""
}
],
"scrape_series_added" : [
{
"type" : "gauge",
"help" : "",
"unit" : ""
}
],
"up" : [
{
"type" : "gauge",
"help" : "",
"unit" : ""
}
]
}
}
PUT _time_stream/{name}
{
"time_stream": {
"labels_fields": "@labels.*_l",
"metrics_fields": "@metrics.*_m",
"label_prefix": "@labels.",
"label_suffix": "_l",
"metric_prefix": "@metrics.",
"metric_suffix": "_m"
}
}
RemoteWrite接口
API
POST /_time_stream/prom_write/{index}
---PB data---
使用說明
remote_write:
- url: "http://127.0.0.1:9200/_time_stream/prom_write/prom_index"
basic_auth:
username: elastic
password: xxxx
Prometheus通過Remote Write寫入數據到TimeStream索引,只支持默認的時序數據模型,即維度前綴是label.,指標前綴是metric.。
Instant Query接口
API
GET /_time_stream/prom/{index}/query
POST /_time_stream/prom/{index}/query
使用說明
Prometheus Instant Query接口,詳細信息請參見Prometheus instant queries。
Range Query接口
API
GET /_time_stream/prom/{index}/query_range
POST /_time_stream/prom/{index}/query_range
使用說明
Prometheus Range Query接口,詳細信息請參見Prometheus range queries。
Series Metadata接口
API
GET /_time_stream/prom/{index}/series
POST /_time_stream/prom/{index}/series
使用說明
Prometheus查詢時間線的接口,詳細信息請參見Prometheus finding series by label matchers。
Labels Metadata接口
API
GET /_time_stream/prom/{index}/labels
POST /_time_stream/prom/{index}/labels
使用說明
Prometheus查詢維度(label)列表的接口,詳細信息請參見Prometheus getting label names。
Label Values Metadata接口
API
GET /_time_stream/prom/{index}/label/<label_name>/values
使用說明
Prometheus查詢維度對應values列表的接口,詳細信息請參見Prometheus querying label values。
Metric Metadata接口
API
GET /_time_stream/prom/{index}/metadata
使用說明
Prometheus查詢指標(Metrics)列表的接口,詳細信息請參見Prometheus querying metric metadata。
TimeStream PromQL支持情況
表達式支持情況
表達式 | 類型 | 是否支持 | 說明 |
powOp | binaryOperator | 是 | 冪乘 |
unaryOp | binaryOperator | 是 | 正、負 |
multOp | binaryOperator | 是 | 乘、除、取模 |
addOp | binaryOperator | 是 | 加、減 |
compareOp | binaryOperator | 是 | 比較(==,>=, >, <=, <, !=) |
andUnlessOp | binaryOperator | 是 | 集合運算(and、or、unless) |
orOp | binaryOperator | 是 | 集合運算(or) |
function | vector | 是 | 函數 |
aggregation | vector | 是 | 聚合運算 |
instantSelector | vector | 是 | 瞬時選擇器 |
matrixSelector | vector | 是 | 向量選擇器 |
offset | vector | 是 | 偏移量 |
literal | vector | 是 | 常量 |
labelMatcher | vector | 是 | 標簽匹配 |
grouping | group | 否 | group聚合,包括on、ignoring、group_left、group_right |
Aggregation operator支持情況
Aggregation operator | 是否支持 |
sum(calculate sum over dimensions) | 是 |
min(select minimum over dimensions) | 是 |
max(select maximum over dimensions) | 是 |
avg(calculate the average over dimensions) | 是 |
group(all values in the resulting vector are 1) | 否 |
stddev(calculate population standard deviation over dimensions) | 否 |
stdvar(calculate population standard variance over dimensions) | 否 |
count(count number of elements in the vector) | 是 |
count_values(count number of elements with the same value) | 是 |
bottomk(smallest k elements by sample value) | 是 |
topk(largest k elements by sample value) | 是 |
quantile(calculate φ-quantile (0 ≤ φ ≤ 1) over dimensions) | 是 |
Function支持情況
Function | 是否支持 |
abs(v instant-vector) | 是 |
absent(v instant-vector) | 否 |
absent_over_time(v range-vector) | 否 |
ceil(v instant-vector) | 是 |
changes(v range-vector) | 否 |
clamp(v instant-vector, min scalar, max scalar) | 是 |
clamp_max(v instant-vector, max scalar) | 是 |
clamp_min(v instant-vector, min scalar) | 是 |
day_of_month(v=vector(time()) instant-vector) | 是 |
day_of_week(v=vector(time()) instant-vector) | 是 |
days_in_month(v=vector(time()) instant-vector) | 否 |
delta(v range-vector) | 是 |
deriv(v range-vector) | 否 |
exp(v instant-vector) | 是 |
floor(v instant-vector) | 是 |
histogram_quantile(φ scalar, b instant-vector) | 否 |
holt_winters(v range-vector, sf scalar, tf scalar) | 否 |
hour(v=vector(time()) instant-vector) | 是 |
idelta(v range-vector) | 是 |
increase(v range-vector) | 是 |
irate(v range-vector) | 是 |
label_join(v instant-vector, dst_label string, separator string, src_label_1 string, src_label_2 string, ...) | 否 |
label_replace(v instant-vector, dst_label string, replacement string, src_label string, regex string) | 否 |
ln(v instant-vector) | 是 |
log2(v instant-vector) | 否 |
log10(v instant-vector) | 是 |
minute(v=vector(time()) instant-vector) | 是 |
month(v=vector(time()) instant-vector) | 是 |
predict_linear(v range-vector, t scalar) | 否 |
rate(v range-vector) | 是 |
resets(v range-vector) | 否 |
round(v instant-vector, to_nearest=1 scalar) | 否 |
scalar(v instant-vector) | 否 |
sgn(v instant-vector) | 是 |
sort(v instant-vector) | 否 |
sort_desc() | 否 |
sqrt(v instant-vector) | 是 |
time() | 否 |
timestamp(v instant-vector) | 是 |
vector(s scalar) | 否 |
year(v=vector(time()) instant-vector) | 是 |
avg_over_time(range-vector) | 是 |
min_over_time(range-vector) | 是 |
max_over_time(range-vector) | 是 |
sum_over_time(range-vector) | 是 |
count_over_time(range-vector) | 是 |
quantile_over_time(scalar, range-vector) | 是 |
stddev_over_time(range-vector) | 否 |
stdvar_over_time(range-vector) | 否 |
last_over_time(range-vector) | 是 |
present_over_time(range-vector) | 否 |
acos(v instant-vector) | 是 |
acosh(v instant-vector) | 是 |
asin(v instant-vector) | 是 |
asinh(v instant-vector) | 否 |
atan(v instant-vector) | 是 |
atanh(v instant-vector) | 否 |
cos(v instant-vector) | 是 |
cosh(v instant-vector) | 是 |
sin(v instant-vector) | 是 |
sinh(v instant-vector) | 是 |
tan(v instant-vector) | 是 |
tanh(v instant-vector) | 是 |
deg(v instant-vector) | 是 |
pi() | 是 |
rad(v instant-vector) | 是 |