Skip to content

Commit 593b009

Browse files
authored
Support for scraping multiple mysqld hosts (prometheus#651)
Add multi-target exporter scrape support. Breaking change: `DATA_SOURCE_NAME` is now removed. Local configuration is now supplied by `MYSQLD_EXPORTER_PASSWORD` and command line arguments. Signed-off-by: Mattias Ängehov <[email protected]>
1 parent c7ab579 commit 593b009

13 files changed

+640
-234
lines changed

Makefile

+3-2
Original file line numberDiff line numberDiff line change
@@ -22,8 +22,9 @@ STATICCHECK_IGNORE =
2222

2323
DOCKER_IMAGE_NAME ?= mysqld-exporter
2424

25-
test-docker:
26-
@echo ">> testing docker image"
25+
.PHONY: test-docker-single-exporter
26+
test-docker-single-exporter:
27+
@echo ">> testing docker image for single exporter"
2728
./test_image.sh "$(DOCKER_IMAGE_NAME):$(DOCKER_IMAGE_TAG)" 9104
2829

2930
.PHONY: test-docker

README.md

+51-14
Original file line numberDiff line numberDiff line change
@@ -30,15 +30,50 @@ NOTE: It is recommended to set a max connection limit for the user to avoid over
3030

3131
### Running
3232

33-
Running using an environment variable:
34-
35-
export DATA_SOURCE_NAME='user:password@(hostname:3306)/'
36-
./mysqld_exporter <flags>
33+
##### Single exporter mode
3734

3835
Running using ~/.my.cnf:
3936

4037
./mysqld_exporter <flags>
4138

39+
##### Multi-target support
40+
41+
This exporter supports the multi-target pattern. This allows running a single instance of this exporter for multiple MySQL targets.
42+
43+
To use the multi-target functionality, send an http request to the endpoint /probe?target=foo:5432 where target is set to the DSN of the MySQL instance to scrape metrics from.
44+
45+
To avoid putting sensitive information like username and password in the URL, you can have multiple configurations in `config.my-cnf` file and match it by adding `&auth_module=<section>` to the request.
46+
47+
Sample config file for multiple configurations
48+
49+
[client]
50+
user = foo
51+
password = foo123
52+
[client.servers]
53+
user = bar
54+
password = bar123
55+
56+
On the prometheus side you can set a scrape config as follows
57+
58+
- job_name: mysql # To get metrics about the mysql exporter’s targets
59+
params:
60+
# Not required. Will match value to child in config file. Default value is `client`.
61+
auth_module: client.servers
62+
static_configs:
63+
- targets:
64+
# All mysql hostnames to monitor.
65+
- server1:3306
66+
- server2:3306
67+
relabel_configs:
68+
- source_labels: [__address__]
69+
target_label: __param_target
70+
- source_labels: [__param_target]
71+
target_label: instance
72+
- target_label: __address__
73+
# The mysqld_exporter host:port
74+
replacement: localhost:9104
75+
76+
##### Flag format
4277
Example format for flags for version > 0.10.0:
4378

4479
--collect.auto_increment.columns
@@ -102,6 +137,8 @@ collect.heartbeat.utc | 5.1 | U
102137
### General Flags
103138
Name | Description
104139
-------------------------------------------|--------------------------------------------------------------------------------------------------
140+
mysqld.address | Hostname and port used for connecting to MySQL server, format: `host:port`. (default: `locahost:3306`)
141+
mysqld.username | Username to be used for connecting to MySQL Server
105142
config.my-cnf | Path to .my.cnf file to read MySQL credentials from. (default: `~/.my.cnf`)
106143
log.level | Logging verbosity (default: info)
107144
exporter.lock_wait_timeout | Set a lock_wait_timeout (in seconds) on the connection to avoid long metadata locking. (default: 2)
@@ -112,6 +149,15 @@ web.listen-address | Address to listen on for web interf
112149
web.telemetry-path | Path under which to expose metrics.
113150
version | Print the version information.
114151

152+
### Environment Variables
153+
Name | Description
154+
-------------------------------------------|--------------------------------------------------------------------------------------------------
155+
MYSQLD_EXPORTER_PASSWORD | Password to be used for connecting to MySQL Server
156+
157+
### Configuration precedence
158+
159+
If you have configured cli with both `mysqld` flags and a valid configuration file, the options in the configuration file will override the flags for `client` section.
160+
115161
## TLS and basic authentication
116162

117163
The MySQLd Exporter supports TLS and basic authentication.
@@ -120,12 +166,6 @@ To use TLS and/or basic authentication, you need to pass a configuration file
120166
using the `--web.config.file` parameter. The format of the file is described
121167
[in the exporter-toolkit repository](https://github.com/prometheus/exporter-toolkit/blob/master/docs/web-configuration.md).
122168

123-
### Setting the MySQL server's data source name
124-
125-
The MySQL server's [data source name](http://en.wikipedia.org/wiki/Data_source_name)
126-
must be set via the `DATA_SOURCE_NAME` environment variable.
127-
The format of this variable is described at https://github.com/go-sql-driver/mysql#dsn-data-source-name.
128-
129169
## Customizing Configuration for a SSL Connection
130170

131171
If The MySQL server supports SSL, you may need to specify a CA truststore to verify the server's chain-of-trust. You may also need to specify a SSL keypair for the client side of the SSL connection. To configure the mysqld exporter to use a custom CA certificate, add the following to the mysql cnf file:
@@ -141,8 +181,6 @@ ssl-key=/path/to/ssl/client/key
141181
ssl-cert=/path/to/ssl/client/cert
142182
```
143183

144-
Customizing the SSL configuration is only supported in the mysql cnf file and is not supported if you set the mysql server's data source name in the environment variable DATA_SOURCE_NAME.
145-
146184

147185
## Using Docker
148186

@@ -157,9 +195,8 @@ docker pull prom/mysqld-exporter
157195
docker run -d \
158196
-p 9104:9104 \
159197
--network my-mysql-network \
160-
-e DATA_SOURCE_NAME="user:password@(hostname:3306)/" \
161198
prom/mysqld-exporter
162-
```
199+
--config.my-cnf=<path_to_cnf>
163200

164201
## heartbeat
165202

config/config.go

+229
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,229 @@
1+
// Copyright 2022 The Prometheus Authors
2+
// Licensed under the Apache License, Version 2.0 (the "License");
3+
// you may not use this file except in compliance with the License.
4+
// You may obtain a copy of the License at
5+
//
6+
// http://www.apache.org/licenses/LICENSE-2.0
7+
//
8+
// Unless required by applicable law or agreed to in writing, software
9+
// distributed under the License is distributed on an "AS IS" BASIS,
10+
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
11+
// See the License for the specific language governing permissions and
12+
// limitations under the License.
13+
14+
package config
15+
16+
import (
17+
"crypto/tls"
18+
"crypto/x509"
19+
"fmt"
20+
"net"
21+
"os"
22+
"sync"
23+
24+
"github.com/go-kit/log"
25+
"github.com/go-kit/log/level"
26+
27+
"github.com/go-sql-driver/mysql"
28+
"github.com/prometheus/client_golang/prometheus"
29+
30+
"gopkg.in/ini.v1"
31+
)
32+
33+
var (
34+
configReloadSuccess = prometheus.NewGauge(prometheus.GaugeOpts{
35+
Namespace: "mysqld_exporter",
36+
Name: "config_last_reload_successful",
37+
Help: "Mysqld exporter config loaded successfully.",
38+
})
39+
40+
configReloadSeconds = prometheus.NewGauge(prometheus.GaugeOpts{
41+
Namespace: "mysqld_exporter",
42+
Name: "config_last_reload_success_timestamp_seconds",
43+
Help: "Timestamp of the last successful configuration reload.",
44+
})
45+
46+
cfg *ini.File
47+
48+
opts = ini.LoadOptions{
49+
// Do not error on nonexistent file to allow empty string as filename input
50+
Loose: true,
51+
// MySQL ini file can have boolean keys.
52+
AllowBooleanKeys: true,
53+
}
54+
55+
err error
56+
)
57+
58+
type Config struct {
59+
Sections map[string]MySqlConfig
60+
}
61+
62+
type MySqlConfig struct {
63+
User string `ini:"user"`
64+
Password string `ini:"password"`
65+
Host string `ini:"host"`
66+
Port int `ini:"port"`
67+
Socket string `ini:"socket"`
68+
SslCa string `ini:"ssl-ca"`
69+
SslCert string `ini:"ssl-cert"`
70+
SslKey string `ini:"ssl-key"`
71+
TlsInsecureSkipVerify bool `ini:"ssl-skip-verfication"`
72+
}
73+
74+
type MySqlConfigHandler struct {
75+
sync.RWMutex
76+
TlsInsecureSkipVerify bool
77+
Config *Config
78+
}
79+
80+
func (ch *MySqlConfigHandler) GetConfig() *Config {
81+
ch.RLock()
82+
defer ch.RUnlock()
83+
return ch.Config
84+
}
85+
86+
func (ch *MySqlConfigHandler) ReloadConfig(filename string, mysqldAddress string, mysqldUser string, tlsInsecureSkipVerify bool, logger log.Logger) error {
87+
var host, port string
88+
defer func() {
89+
if err != nil {
90+
configReloadSuccess.Set(0)
91+
} else {
92+
configReloadSuccess.Set(1)
93+
configReloadSeconds.SetToCurrentTime()
94+
}
95+
}()
96+
97+
if cfg, err = ini.LoadSources(
98+
opts,
99+
[]byte("[client]\npassword = ${MYSQLD_EXPORTER_PASSWORD}\n"),
100+
filename,
101+
); err != nil {
102+
return fmt.Errorf("failed to load %s: %w", filename, err)
103+
}
104+
105+
if host, port, err = net.SplitHostPort(mysqldAddress); err != nil {
106+
return fmt.Errorf("failed to parse address: %w", err)
107+
}
108+
109+
if clientSection := cfg.Section("client"); clientSection != nil {
110+
if cfgHost := clientSection.Key("host"); cfgHost.String() == "" {
111+
cfgHost.SetValue(host)
112+
}
113+
if cfgPort := clientSection.Key("port"); cfgPort.String() == "" {
114+
cfgPort.SetValue(port)
115+
}
116+
if cfgUser := clientSection.Key("user"); cfgUser.String() == "" {
117+
cfgUser.SetValue(mysqldUser)
118+
}
119+
}
120+
121+
cfg.ValueMapper = os.ExpandEnv
122+
config := &Config{}
123+
m := make(map[string]MySqlConfig)
124+
for _, sec := range cfg.Sections() {
125+
sectionName := sec.Name()
126+
127+
if sectionName == "DEFAULT" {
128+
continue
129+
}
130+
131+
mysqlcfg := &MySqlConfig{
132+
TlsInsecureSkipVerify: tlsInsecureSkipVerify,
133+
}
134+
if err != nil {
135+
level.Error(logger).Log("msg", "failed to load config", "section", sectionName, "err", err)
136+
continue
137+
}
138+
139+
err = sec.StrictMapTo(mysqlcfg)
140+
if err != nil {
141+
level.Error(logger).Log("msg", "failed to parse config", "section", sectionName, "err", err)
142+
continue
143+
}
144+
if err := mysqlcfg.validateConfig(); err != nil {
145+
level.Error(logger).Log("msg", "failed to validate config", "section", sectionName, "err", err)
146+
continue
147+
}
148+
149+
m[sectionName] = *mysqlcfg
150+
}
151+
config.Sections = m
152+
if len(config.Sections) == 0 {
153+
return fmt.Errorf("no configuration found")
154+
}
155+
ch.Lock()
156+
ch.Config = config
157+
ch.Unlock()
158+
return nil
159+
}
160+
161+
func (m MySqlConfig) validateConfig() error {
162+
if m.User == "" {
163+
return fmt.Errorf("no user specified in section or parent")
164+
}
165+
if m.Password == "" {
166+
return fmt.Errorf("no password specified in section or parent")
167+
}
168+
169+
return nil
170+
}
171+
172+
func (m MySqlConfig) FormDSN(target string) (string, error) {
173+
var dsn, host, port string
174+
175+
user := m.User
176+
password := m.Password
177+
if target == "" {
178+
host := m.Host
179+
port := m.Port
180+
socket := m.Socket
181+
if socket != "" {
182+
dsn = fmt.Sprintf("%s:%s@unix(%s)/", user, password, socket)
183+
} else {
184+
dsn = fmt.Sprintf("%s:%s@tcp(%s:%d)/", user, password, host, port)
185+
}
186+
} else {
187+
if host, port, err = net.SplitHostPort(target); err != nil {
188+
return dsn, fmt.Errorf("failed to parse target: %s", err)
189+
}
190+
dsn = fmt.Sprintf("%s:%s@tcp(%s:%s)/", user, password, host, port)
191+
}
192+
193+
if m.SslCa != "" {
194+
if err := m.CustomizeTLS(); err != nil {
195+
err = fmt.Errorf("failed to register a custom TLS configuration for mysql dsn: %w", err)
196+
return dsn, err
197+
}
198+
dsn = fmt.Sprintf("%s?tls=custom", dsn)
199+
}
200+
201+
return dsn, nil
202+
}
203+
204+
func (m MySqlConfig) CustomizeTLS() error {
205+
var tlsCfg tls.Config
206+
caBundle := x509.NewCertPool()
207+
pemCA, err := os.ReadFile(m.SslCa)
208+
if err != nil {
209+
return err
210+
}
211+
if ok := caBundle.AppendCertsFromPEM(pemCA); ok {
212+
tlsCfg.RootCAs = caBundle
213+
} else {
214+
return fmt.Errorf("failed parse pem-encoded CA certificates from %s", m.SslCa)
215+
}
216+
if m.SslCert != "" && m.SslKey != "" {
217+
certPairs := make([]tls.Certificate, 0, 1)
218+
keypair, err := tls.LoadX509KeyPair(m.SslCert, m.SslKey)
219+
if err != nil {
220+
return fmt.Errorf("failed to parse pem-encoded SSL cert %s or SSL key %s: %w",
221+
m.SslCert, m.SslKey, err)
222+
}
223+
certPairs = append(certPairs, keypair)
224+
tlsCfg.Certificates = certPairs
225+
}
226+
tlsCfg.InsecureSkipVerify = m.TlsInsecureSkipVerify
227+
mysql.RegisterTLSConfig("custom", &tlsCfg)
228+
return nil
229+
}

0 commit comments

Comments
 (0)