test/apisix
1
0
Fork 0
mirror of https://gitee.com/iresty/apisix.git synced 2024-12-03 12:37:36 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity

feature(limit-count): supported global limit count with redis server. (#624)

Browse Source
This commit is contained in:
YuanSheng Wang 2019-10-08 13:15:48 +08:00 committed by GitHub
parent b4d279d315
commit 6b875cec33
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
8 changed files with 437 additions and 25 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
.travis.yml
View File

@ -31,6 +31,9 @@ addons:
homebrew:
update: true
services:
- redis-server
cache:
directories:
- build-cache

6
Makefile
View File

@ -58,7 +58,8 @@ check:
lua/apisix/core/*.lua \
lua/apisix/http/*.lua \
lua/apisix/plugins/*.lua \
lua/apisix/plugins/grpc-transcode/*.lua > \
lua/apisix/plugins/grpc-transcode/*.lua \
lua/apisix/plugins/limit-count/*.lua > \
/tmp/check.log 2>&1 || (cat /tmp/check.log && exit 1)
@ -132,6 +133,9 @@ install:
$(INSTALL) -d $(INST_LUADIR)/apisix/lua/apisix/plugins/grpc-transcode/
$(INSTALL) lua/apisix/plugins/grpc-transcode/*.lua $(INST_LUADIR)/apisix/lua/apisix/plugins/grpc-transcode/
$(INSTALL) -d $(INST_LUADIR)/apisix/lua/apisix/plugins/limit-count/
$(INSTALL) lua/apisix/plugins/limit-count/*.lua $(INST_LUADIR)/apisix/lua/apisix/plugins/limit-count/
$(INSTALL) -d $(INST_LUADIR)/apisix/lua/apisix/plugins
$(INSTALL) lua/apisix/plugins/*.lua $(INST_LUADIR)/apisix/lua/apisix/plugins/

54
doc/plugins/limit-count-cn.md
View File

@ -5,15 +5,22 @@
在指定的时间范围内，限制总的请求个数。并且在 HTTP 响应头中返回剩余可以请求的个数。
### 参数
* `count`：指定时间窗口内的请求数量阈值
* `time_window`：时间窗口的大小（以秒为单位），超过这个时间就会重置
* `rejected_code`：当请求超过阈值被拒绝时，返回的 HTTP 状态码，默认是 503
* `key`：是用来做请求计数的依据，当前接受的 key 有："remote_addr"(客户端IP地址), "server_addr"(服务端 IP 地址), 请求头中的"X-Forwarded-For" 或 "X-Real-IP"。
|名称 |可选项 |说明|
|--------- |--------|-----------|
|count |必选 |指定时间窗口内的请求数量阈值|
|time_window |必选 |时间窗口的大小（以秒为单位），超过这个时间就会重置|
|key |必选 |是用来做请求计数的依据，当前接受的 key 有: "remote_addr", "server_addr", "http_x_real_ip", "http_x_forwarded_for"。|
|rejected_code |可选 |T当请求超过阈值被拒绝时，返回的 HTTP 状态码，默认是 503|
|policy |可选 |用于检索和增加限制的速率限制策略。可选的值有：`local`(计数器被以内存方式保存在节点本地，默认选项) 和 `redis`(计数器保存在 Redis 服务节点上，从而可以跨节点共享结果，通常用它来完成全局限速).|
|redis.host |可选 |当使用 `redis` 限速策略时，该属性是 Redis 服务节点的地址。|
|redis.port |可选 |当使用 `redis` 限速策略时，该属性是 Redis 服务节点的端口，默认端口 6379。|
|redis.timeout |可选 |当使用 `redis` 限速策略时，该属性是 Redis 服务节点以毫秒为单位的超时时间，默认是 1000 ms（1 秒）。|
### 示例
#### 开启插件
下面是一个示例，在指定的 route 上开启了 limit count 插件:
下面是一个示例，在指定的 `route` 上开启了 `limit count` 插件:
```shell
curl -i http://127.0.0.1:9080/apisix/admin/routes/1 -X PUT -d '
@ -42,6 +49,37 @@ curl -i http://127.0.0.1:9080/apisix/admin/routes/1 -X PUT -d '
然后在 route 页面中添加 limit-count 插件：
![](../images/plugin/limit-count-2.png)
如果你需要一个集群级别的流量控制，我们可以借助 redis server 来完成。不同的 APISIX 节点之间将共享流量限速结果，实现集群流量限速。
请看下面例子：
```shell
curl -i http://127.0.0.1:9080/apisix/admin/routes/1 -X PUT -d '
{
"uri": "/index.html",
"plugins": {
"limit-count": {
"count": 2,
"time_window": 60,
"rejected_code": 503,
"key": "remote_addr",
"policy": "redis",
"redis": {
"host": "127.0.0.1",
"port": 6379,
"timeout": 1001
}
}
},
"upstream": {
"type": "roundrobin",
"nodes": {
"39.97.63.215:80": 1
}
}
}'
```
#### 测试插件
上述配置限制了 60 秒内只能访问 2 次，前两次访问都会正常访问：
```shell
@ -76,10 +114,10 @@ Server: APISIX web server
</html>
```
这就表示 limit count 插件生效了。
这就表示 `limit count` 插件生效了。
#### 移除插件
当你想去掉 limit count 插件的时候，很简单，在插件的配置中把对应的 json 配置删除即可，无须重启服务，即刻生效：
当你想去掉 `limit count` 插件的时候，很简单，在插件的配置中把对应的 json 配置删除即可，无须重启服务，即刻生效：
```shell
curl http://127.0.0.1:9080/apisix/admin/routes/1 -X PUT -d '
@ -95,4 +133,4 @@ curl http://127.0.0.1:9080/apisix/admin/routes/1 -X PUT -d '
}'
```
现在就已经移除了 limit count 插件了。其他插件的开启和移除也是同样的方法。
现在就已经移除了 `limit count` 插件了。其他插件的开启和移除也是同样的方法。

56
doc/plugins/limit-count.md
View File

@ -2,15 +2,24 @@
# limit-count
### Parameters
* `count`: is the specified number of requests threshold.
* `time_window`: is the time window in seconds before the request count is reset.
* `rejected_code`: The HTTP status code returned when the request exceeds the threshold is rejected. The default is 503.
* `key`: is the user specified key to limit the rate, now accept those as key: "remote_addr"(client's IP), "server_addr"(server's IP), "X-Forwarded-For/X-Real-IP" in request header.
|name |option |description|
|--------- |--------|-----------|
|count |required|the specified number of requests threshold.|
|time_window |required|the time window in seconds before the request count is reset.|
|key |required|the user specified key to limit the rate. Here is fully key list: "remote_addr", "server_addr", "http_x_real_ip", "http_x_forwarded_for".|
|rejected_code |optional|The HTTP status code returned when the request exceeds the threshold is rejected. The default is 503.|
|policy |optional|The rate-limiting policies to use for retrieving and incrementing the limits. Available values are `local`(the counters will be stored locally in-memory on the node, default value) and `redis`(counters are stored on a Redis server and will be shared across the nodes, usually used it to do the global speed limit).|
|redis.host |optional|When using the `redis` policy, this property specifies the address of the Redis server.|
|redis.port |optional|When using the `redis` policy, this property specifies the port of the Redis server. The default port is 6379.|
|redis.timeout |optional|When using the `redis` policy, this property specifies the timeout in milliseconds of any command submitted to the Redis server. The default timeout is 1000 ms(1 second).|
### example
#### enable plugin
Here's an example, enable the limit count plugin on the specified route:
Here's an example, enable the `limit count` plugin on the specified route:
```shell
curl -i http://127.0.0.1:9080/apisix/admin/routes/1 -X PUT -d '
@ -39,6 +48,37 @@ You can open dashboard with a browser: `http://127.0.0.1:9080/apisix/dashboard/`
Then add limit-count plugin:
![](../images/plugin/limit-count-2.png)
If you need a cluster-level precision traffic limit, then we can do it with the redis server. The rate limit of the traffic will be shared between different APISIX nodes to limit the rate of cluster traffic.
Here is the example:
```shell
curl -i http://127.0.0.1:9080/apisix/admin/routes/1 -X PUT -d '
{
"uri": "/index.html",
"plugins": {
"limit-count": {
"count": 2,
"time_window": 60,
"rejected_code": 503,
"key": "remote_addr",
"policy": "redis",
"redis": {
"host": "127.0.0.1",
"port": 6379,
"timeout": 1001
}
}
},
"upstream": {
"type": "roundrobin",
"nodes": {
"39.97.63.215:80": 1
}
}
}'
```
#### test plugin
The above configuration limits access to only 2 times in 60 seconds. The first two visits will be normally:
```shell
@ -74,10 +114,10 @@ Server: APISIX web server
</html>
```
This means that the limit count plugin is in effect.
This means that the `limit count` plugin is in effect.
#### disable plugin
When you want to disable the limit count plugin, it is very simple,
When you want to disable the `limit count` plugin, it is very simple,
you can delete the corresponding json configuration in the plugin configuration,
no need to restart the service, it will take effect immediately:
```shell
@ -94,4 +134,4 @@ curl http://127.0.0.1:9080/apisix/admin/routes/1 -X PUT -d '
}'
```
The limit count plugin has been disabled now. It works for other plugins.
The `limit count` plugin has been disabled now. It works for other plugins.

66
lua/apisix/plugins/limit-count.lua
View File

@ -1,6 +1,11 @@
local limit_count_new = require("resty.limit.count").new
local limit_local_new = require("resty.limit.count").new
local core = require("apisix.core")
local plugin_name = "limit-count"
local limit_redis_new
do
local redis_src = "apisix.plugins.limit-count.limit-count-redis"
limit_redis_new = require(redis_src).new
end
local schema = {
@ -8,11 +13,31 @@ local schema = {
properties = {
count = {type = "integer", minimum = 0},
time_window = {type = "integer", minimum = 0},
key = {type = "string",
key = {
type = "string",
enum = {"remote_addr", "server_addr", "http_x_real_ip",
"http_x_forwarded_for"},
},
rejected_code = {type = "integer", minimum = 200, maximum = 600},
policy = {
type = "string",
enum = {"local", "redis"},
},
redis = {
type = "object",
properties = {
host = {
type = "string", minLength = 2
},
port = {
type = "integer", minimum = 1
},
timeout = {
type = "integer", minimum = 1
},
},
required = {"host"},
},
},
additionalProperties = false,
required = {"count", "time_window", "key", "rejected_code"},
@ -20,21 +45,50 @@ local schema = {
local _M = {
version = 0.1,
priority = 1002, -- TODO: add a type field, may be a good idea
version = 0.2,
priority = 1002,
name = plugin_name,
schema = schema,
}
function _M.check_schema(conf)
return core.schema.check(schema, conf)
local ok, err = core.schema.check(schema, conf)
if not ok then
return false, err
end
if not conf.policy then
conf.policy = "local"
end
if conf.policy == "redis" then
if not conf.redis then
return false, "missing valid redis options"
end
conf.redis.port = conf.redis.port or 6379
conf.redis.timeout = conf.redis.timeout or 1000
end
return true
end
local function create_limit_obj(conf)
core.log.info("create new limit-count plugin instance")
return limit_count_new("plugin-limit-count", conf.count, conf.time_window)
if not conf.policy or conf.policy == "local" then
return limit_local_new("plugin-" .. plugin_name, conf.count,
conf.time_window)
end
if conf.policy == "redis" then
return limit_redis_new("plugin-" .. plugin_name,
conf.count, conf.time_window, conf.redis)
end
return nil
end

72
lua/apisix/plugins/limit-count/limit-count-redis.lua Normal file
View File

@ -0,0 +1,72 @@
local redis_new = require("resty.redis").new
local core = require("apisix.core")
local assert = assert
local setmetatable = setmetatable
local tostring = tostring
local _M = {version = 0.1}
local mt = {
__index = _M
}
function _M.new(plugin_name, limit, window, redis_conf)
assert(limit > 0 and window > 0)
local self = {limit = limit, window = window, redis = redis_conf,
plugin_name = plugin_name}
return setmetatable(self, mt)
end
function _M.incoming(self, key)
local red = redis_new()
local conf = self.redis
local timeout = conf.timeout or 1000 -- 1sec
core.log.info("ttl key: ", key, " timeout: ", timeout)
red:set_timeouts(timeout, timeout, timeout)
local ok, err = red:connect(conf.host, conf.port or 6379)
if not ok then
return false, err
end
local limit = self.limit
local window = self.window
local remaining
key = self.plugin_name .. tostring(key)
local ret, err = red:ttl(key)
core.log.info("ttl key: ", key, " ret: ", ret, " err: ", err)
if ret < 0 then
ret, err = red:set(key, limit -1, "EX", window, "NX")
if not ret then
return nil, err
end
return 0, limit -1
end
remaining, err = red:incrby(key, -1)
if not remaining then
return nil, err
end
local ok, err = red:set_keepalive(10000, 100)
if not ok then
return nil, err
end
if remaining < 0 then
return nil, "rejected"
end
return 0, remaining
end
return _M

4
t/node/merge-route.t
View File

@ -215,5 +215,5 @@ location /t {
--- request
GET /t
--- error_log eval
[qr/merge_service_route.*"time_window":60,"rejected_code":503/,
qr/merge_service_route.*"time_window":60,"rejected_code":503/]
[qr/merge_service_route.*"time_window":60,/,
qr/merge_service_route.*"time_window":60,/]

201
t/plugin/limit-count-redis.t Normal file
View File

@ -0,0 +1,201 @@
BEGIN {
if ($ENV{TEST_NGINX_CHECK_LEAK}) {
$SkipReason = "unavailable for the hup tests";
} else {
$ENV{TEST_NGINX_USE_HUP} = 1;
undef $ENV{TEST_NGINX_USE_STAP};
}
}
use t::APISIX 'no_plan';
repeat_each(1);
no_long_string();
no_shuffle();
no_root_location();
run_tests;
__DATA__
=== TEST 1: set route, missing redis host
--- config
location /t {
content_by_lua_block {
local t = require("lib.test_admin").test
local code, body = t('/apisix/admin/routes/1',
ngx.HTTP_PUT,
[[{
"plugins": {
"limit-count": {
"count": 2,
"time_window": 60,
"rejected_code": 503,
"key": "remote_addr",
"policy": "redis"
}
},
"upstream": {
"nodes": {
"127.0.0.1:1980": 1
},
"type": "roundrobin"
},
"uri": "/hello"
}]]
)
if code >= 300 then
ngx.status = code
end
ngx.print(body)
}
}
--- request
GET /t
--- error_code: 400
--- response_body
{"error_msg":"failed to check the configuration of plugin limit-count err: missing valid redis options"}
--- no_error_log
[error]
=== TEST 2: set route, with redis host and port
--- config
location /t {
content_by_lua_block {
local t = require("lib.test_admin").test
local code, body = t('/apisix/admin/routes/1',
ngx.HTTP_PUT,
[[{
"uri": "/hello",
"plugins": {
"limit-count": {
"count": 2,
"time_window": 60,
"rejected_code": 503,
"key": "remote_addr",
"policy": "redis",
"redis": {
"host": "127.0.0.1",
"port": 6379,
"timeout": 1001
}
}
},
"upstream": {
"nodes": {
"127.0.0.1:1980": 1
},
"type": "roundrobin"
}
}]]
)
if code >= 300 then
ngx.status = code
end
ngx.say(body)
}
}
--- request
GET /t
--- response_body
passed
--- no_error_log
[error]
=== TEST 3: set route(default value: port and timeout)
--- config
location /t {
content_by_lua_block {
local t = require("lib.test_admin").test
local code, body = t('/apisix/admin/routes/1',
ngx.HTTP_PUT,
[[{
"uri": "/hello",
"plugins": {
"limit-count": {
"count": 2,
"time_window": 60,
"rejected_code": 503,
"key": "remote_addr",
"policy": "redis",
"redis": {
"host": "127.0.0.1"
}
}
},
"upstream": {
"nodes": {
"127.0.0.1:1980": 1
},
"type": "roundrobin"
}
}]],
[[{
"node": {
"value": {
"plugins": {
"limit-count": {
"count": 2,
"time_window": 60,
"rejected_code": 503,
"key": "remote_addr",
"policy": "redis",
"redis": {
"host": "127.0.0.1",
"port": 6379,
"timeout": 1000
}
}
},
"upstream": {
"nodes": {
"127.0.0.1:1980": 1
},
"type": "roundrobin"
},
"uri": "/hello"
},
"key": "/apisix/routes/1"
},
"action": "set"
}]]
)
if code >= 300 then
ngx.status = code
end
ngx.say(body)
}
}
--- request
GET /t
--- response_body
passed
--- no_error_log
[error]
=== TEST 4: up the limit
--- pipelined_requests eval
["GET /hello", "GET /hello", "GET /hello", "GET /hello"]
--- error_code eval
[200, 200, 503, 503]
--- no_error_log
[error]
=== TEST 5: up the limit
--- pipelined_requests eval
["GET /hello1", "GET /hello", "GET /hello2", "GET /hello", "GET /hello"]
--- error_code eval
[404, 503, 404, 503, 503]
--- no_error_log
[error]