init source

This commit is contained in:
Le Viet
2022-03-07 22:07:57 +07:00
parent e4376f3777
commit 8aba590a8d
11240 changed files with 1012977 additions and 0 deletions
+50
View File
@@ -0,0 +1,50 @@
(The MIT License)
Copyright (c) 2018-present Sushant <sushantdhiman@outlook.com>
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
'Software'), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
--------------------------------
(Original Fork License)
[generic-pool@2.5]
Copyright (c) 2010-2016 James Cooper <james@bitmechanic.com>
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
'Software'), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+130
View File
@@ -0,0 +1,130 @@
# sequelize-pool
[![npm](https://img.shields.io/npm/v/sequelize-pool.svg?style=flat-square)](https://www.npmjs.com/package/sequelize-pool)
[![Travis (.org)](https://img.shields.io/travis/sushantdhiman/sequelize-pool.svg?style=flat-square)](https://travis-ci.org/sushantdhiman/sequelize-pool)
Resource pool. Can be used to reuse or throttle expensive resources such as
database connections.
This is a fork from [generic-pool@v2.5](https://github.com/coopernurse/node-pool/tree/v2.5).
## Installation
```bash
$ npm install --save sequelize-pool
$ yarn add sequelize-pool
```
## Example
### Step 1 - Create pool using a factory object
```js
// Create a MySQL connection pool
var Pool = require('sequelize-pool').Pool;
var mysql2 = require('mysql2/promise');
var pool = new Pool({
name : 'mysql',
create : function() {
// return Promise
return mysql2.createConnection({
user: 'scott',
password: 'tiger',
database:'mydb'
});
},
destroy : function(client) { client.end(); },
max : 10,
// optional. if you set this, make sure to drain() (see step 3)
min : 2,
// Delay in milliseconds after which available resources in the pool will be destroyed.
idleTimeoutMillis : 30000,
// Delay in milliseconds after which pending acquire request in the pool will be rejected.
acquireTimeoutMillis: 30000,
// Function, defaults to console.log
log : true
});
```
### Step 2 - Use pool in your code to acquire/release resources
```js
// acquire connection
pool.acquire().then(connection => {
client.query("select * from foo", [], function() {
// return object back to pool
pool.release(client);
});
});
```
### Step 3 - Drain pool during shutdown (optional)
If you are shutting down a long-lived process, you may notice
that node fails to exit for 30 seconds or so. This is a side
effect of the idleTimeoutMillis behaviour -- the pool has a
setTimeout() call registered that is in the event loop queue, so
node won't terminate until all resources have timed out, and the pool
stops trying to manage them.
This behaviour will be more problematic when you set factory.min > 0,
as the pool will never become empty, and the setTimeout calls will
never end.
In these cases, use the pool.drain() function. This sets the pool
into a "draining" state which will gracefully wait until all
idle resources have timed out. For example, you can call:
```js
// Only call this once in your application -- at the point you want
// to shutdown and stop using this pool.
pool.drain().then(() => pool.destroyAllNow());
```
If you do this, your node process will exit gracefully.
## Draining
If you know would like to terminate all the resources in your pool before
their timeouts have been reached, you can use `destroyAllNow()` in conjunction
with `drain()`:
```js
pool.drain().then(() => pool.destroyAllNow());
```
One side-effect of calling `drain()` is that subsequent calls to `acquire()`
will throw an Error.
## Pool info
The following functions will let you get information about the pool:
```js
// returns factory.name for this pool
pool.name
// returns number of resources in the pool regardless of
// whether they are free or in use
pool.size
// returns number of unused resources in the pool
pool.available
// returns number of callers waiting to acquire a resource
pool.waiting
// returns number of maxixmum number of resources allowed by ppol
pool.maxSize
// returns number of minimum number of resources allowed by ppol
pool.minSize
```
## Run Tests
$ npm install
$ npm test
+45
View File
@@ -0,0 +1,45 @@
"use strict";
const { TimeoutError } = require("./TimeoutError");
class Deferred {
constructor() {
this._timeout = null;
this._promise = new Promise((resolve, reject) => {
this._reject = reject;
this._resolve = resolve;
});
}
registerTimeout(timeoutInMillis, callback) {
if (this._timeout) return;
this._timeout = setTimeout(() => {
callback();
this.reject(new TimeoutError("Operation timeout"));
}, timeoutInMillis);
}
_clearTimeout() {
if (!this._timeout) return;
clearTimeout(this._timeout);
this._timeout = null;
}
resolve(value) {
this._clearTimeout();
this._resolve(value);
}
reject(error) {
this._clearTimeout();
this._reject(error);
}
promise() {
return this._promise;
}
}
module.exports = Deferred;
+487
View File
@@ -0,0 +1,487 @@
"use strict";
const Deferred = require("./Deferred");
/**
* Generate an Object pool with a specified `factory`.
*
* @class
* @param {Object} factory
* Factory to be used for generating and destroying the items.
* @param {String} [factory.name]
* Name of the factory. Serves only logging purposes.
* @param {Function} factory.create
* Should create the item to be acquired,
* and call it's first callback argument with the generated item as it's argument.
* @param {Function} factory.destroy
* Should gently close any resources that the item is using.
* Called before the items is destroyed.
* @param {Function} factory.validate
* Should return true if connection is still valid and false
* If it should be removed from pool. Called before item is
* acquired from pool.
* @param {Number} factory.max
* Maximum number of items that can exist at the same time.
* Any further acquire requests will be pushed to the waiting list.
* @param {Number} factory.min
* Minimum number of items in pool (including in-use).
* When the pool is created, or a resource destroyed, this minimum will
* be checked. If the pool resource count is below the minimum, a new
* resource will be created and added to the pool.
* @param {Number} [factory.idleTimeoutMillis=30000]
* Delay in milliseconds after which available resources in the pool will be destroyed.
* This does not affects pending acquire requests.
* @param {Number} [factory.acquireTimeoutMillis=30000]
* Delay in milliseconds after which pending acquire request in the pool will be rejected.
* Pending acquires are acquire calls which are yet to receive an response from factory.create
* @param {Number} [factory.reapIntervalMillis=1000]
* Clean up is scheduled in every `factory.reapIntervalMillis` milliseconds.
* @param {Boolean|Function} [factory.log=false]
* Whether the pool should log activity. If function is specified,
* that will be used instead. The function expects the arguments msg, loglevel
*/
class Pool {
constructor(factory) {
if (!factory.create) {
throw new Error("create function is required");
}
if (!factory.destroy) {
throw new Error("destroy function is required");
}
if (!factory.validate) {
throw new Error("validate function is required");
}
if (
typeof factory.min !== "number" ||
factory.min < 0 ||
factory.min !== Math.round(factory.min)
) {
throw new Error("min must be an integer >= 0");
}
if (
typeof factory.max !== "number" ||
factory.max <= 0 ||
factory.max !== Math.round(factory.max)
) {
throw new Error("max must be an integer > 0");
}
if (factory.min > factory.max) {
throw new Error("max is smaller than min");
}
// defaults
factory.idleTimeoutMillis = factory.idleTimeoutMillis || 30000;
factory.acquireTimeoutMillis = factory.acquireTimeoutMillis || 30000;
factory.reapInterval = factory.reapIntervalMillis || 1000;
factory.max = parseInt(factory.max, 10);
factory.min = parseInt(factory.min, 10);
factory.log = factory.log || false;
this._factory = factory;
this._count = 0;
this._draining = false;
// queues
this._pendingAcquires = [];
this._inUseObjects = [];
this._availableObjects = [];
// timing controls
this._removeIdleTimer = null;
this._removeIdleScheduled = false;
}
get size() {
return this._count;
}
get name() {
return this._factory.name;
}
get available() {
return this._availableObjects.length;
}
get using() {
return this._inUseObjects.length;
}
get waiting() {
return this._pendingAcquires.length;
}
get maxSize() {
return this._factory.max;
}
get minSize() {
return this._factory.min;
}
/**
* logs to console or user defined log function
* @private
* @param {string} message
* @param {string} level
*/
_log(message, level) {
if (typeof this._factory.log === "function") {
this._factory.log(message, level);
} else if (this._factory.log) {
console.log(`${level.toUpperCase()} pool ${this.name} - ${message}`);
}
}
/**
* Checks and removes the available (idle) clients that have timed out.
* @private
*/
_removeIdle() {
const toRemove = [];
const now = Date.now();
let i;
let available = this._availableObjects.length;
const maxRemovable = this._count - this._factory.min;
let timeout;
this._removeIdleScheduled = false;
// Go through the available (idle) items,
// check if they have timed out
for (i = 0; i < available && maxRemovable > toRemove.length; i++) {
timeout = this._availableObjects[i].timeout;
if (now >= timeout) {
// Client timed out, so destroy it.
this._log(
"removeIdle() destroying obj - now:" + now + " timeout:" + timeout,
"verbose"
);
toRemove.push(this._availableObjects[i].resource);
}
}
toRemove.forEach(this.destroy, this);
// NOTE: we are re-calculating this value because it may have changed
// after destroying items above
// Replace the available items with the ones to keep.
available = this._availableObjects.length;
if (available > 0) {
this._log("this._availableObjects.length=" + available, "verbose");
this._scheduleRemoveIdle();
} else {
this._log("removeIdle() all objects removed", "verbose");
}
}
/**
* Schedule removal of idle items in the pool.
*
* More schedules cannot run concurrently.
*/
_scheduleRemoveIdle() {
if (!this._removeIdleScheduled) {
this._removeIdleScheduled = true;
this._removeIdleTimer = setTimeout(() => {
this._removeIdle();
}, this._factory.reapInterval);
}
}
/**
* Try to get a new client to work, and clean up pool unused (idle) items.
*
* - If there are available clients waiting, pop the first one out (LIFO),
* and call its callback.
* - If there are no waiting clients, try to create one if it won't exceed
* the maximum number of clients.
* - If creating a new client would exceed the maximum, add the client to
* the wait list.
* @private
*/
_dispense() {
let resourceWithTimeout = null;
const waitingCount = this._pendingAcquires.length;
this._log(
`dispense() clients=${waitingCount} available=${this._availableObjects.length}`,
"info"
);
if (waitingCount < 1) {
return;
}
while (this._availableObjects.length > 0) {
this._log("dispense() - reusing obj", "verbose");
resourceWithTimeout = this._availableObjects[
this._availableObjects.length - 1
];
if (!this._factory.validate(resourceWithTimeout.resource)) {
this.destroy(resourceWithTimeout.resource);
continue;
}
this._availableObjects.pop();
this._inUseObjects.push(resourceWithTimeout.resource);
const deferred = this._pendingAcquires.shift();
return deferred.resolve(resourceWithTimeout.resource);
}
if (this._count < this._factory.max) {
this._createResource();
}
}
/**
* @private
*/
_createResource() {
this._count += 1;
this._log(
`createResource() - creating obj - count=${this._count} min=${this._factory.min} max=${this._factory.max}`,
"verbose"
);
this._factory
.create()
.then(resource => {
const deferred = this._pendingAcquires.shift();
this._inUseObjects.push(resource);
if (deferred) {
deferred.resolve(resource);
} else {
this._addResourceToAvailableObjects(resource);
}
})
.catch(error => {
const deferred = this._pendingAcquires.shift();
this._count -= 1;
if (this._count < 0) this._count = 0;
if (deferred) {
deferred.reject(error);
}
process.nextTick(() => {
this._dispense();
});
});
}
_addResourceToAvailableObjects(resource) {
const resourceWithTimeout = {
resource: resource,
timeout: Date.now() + this._factory.idleTimeoutMillis
};
this._availableObjects.push(resourceWithTimeout);
this._dispense();
this._scheduleRemoveIdle();
}
/**
* @private
*/
_ensureMinimum() {
let i, diff;
if (!this._draining && this._count < this._factory.min) {
diff = this._factory.min - this._count;
for (i = 0; i < diff; i++) {
this._createResource();
}
}
}
/**
* Requests a new resource. This will call factory.create to request new resource.
*
* It will be rejected with timeout error if `factory.create` didn't respond
* back within specified `acquireTimeoutMillis`
*
* @returns {Promise<Object>}
*/
acquire() {
if (this._draining) {
return Promise.reject(
new Error("pool is draining and cannot accept work")
);
}
const deferred = new Deferred();
deferred.registerTimeout(this._factory.acquireTimeoutMillis, () => {
// timeout triggered, promise will be rejected
// remove this object from pending list
this._pendingAcquires = this._pendingAcquires.filter(
pending => pending !== deferred
);
});
this._pendingAcquires.push(deferred);
this._dispense();
return deferred.promise();
}
/**
* Return the resource to the pool, in case it is no longer required.
*
* @param {Object} resource The acquired object to be put back to the pool.
*
* @returns {void}
*/
release(resource) {
// check to see if this object has already been released
// (i.e., is back in the pool of this._availableObjects)
if (
this._availableObjects.some(
resourceWithTimeout => resourceWithTimeout.resource === resource
)
) {
this._log(
"release called twice for the same resource: " + new Error().stack,
"error"
);
return;
}
// check to see if this object exists in the `in use` list and remove it
const index = this._inUseObjects.indexOf(resource);
if (index < 0) {
this._log(
"attempt to release an invalid resource: " + new Error().stack,
"error"
);
return;
}
this._inUseObjects.splice(index, 1);
this._addResourceToAvailableObjects(resource);
}
/**
* Request the client to be destroyed. The factory's destroy handler
* will also be called.
*
* This should be called within an acquire() block as an alternative to release().
*
* @param {Object} resource The acquired item to be destroyed.
*
* @returns {void}
*/
destroy(resource) {
const available = this._availableObjects.length;
const using = this._inUseObjects.length;
this._availableObjects = this._availableObjects.filter(
object => object.resource !== resource
);
this._inUseObjects = this._inUseObjects.filter(
object => object !== resource
);
// resource was not removed, then no need to decrement _count
if (
available === this._availableObjects.length &&
using === this._inUseObjects.length
) {
this._ensureMinimum();
return;
}
this._count -= 1;
if (this._count < 0) this._count = 0;
this._factory.destroy(resource);
this._ensureMinimum();
}
/**
* Disallow any new requests and let the request backlog dissipate.
*
* @returns {Promise}
*/
drain() {
this._log("draining", "info");
// disable the ability to put more work on the queue.
this._draining = true;
const check = callback => {
// wait until all client requests have been satisfied.
if (this._pendingAcquires.length > 0) {
// pool is draining so we wont accept new acquires but
// we need to clear pending acquires
this._dispense();
return setTimeout(() => {
check(callback);
}, 100);
}
// wait until in use object have been released.
if (this._availableObjects.length !== this._count) {
return setTimeout(() => {
check(callback);
}, 100);
}
callback();
};
// No error handling needed here.
return new Promise(resolve => check(resolve));
}
/**
* Forcibly destroys all clients regardless of timeout. Intended to be
* invoked as part of a drain. Does not prevent the creation of new
* clients as a result of subsequent calls to acquire.
*
* Note that if factory.min > 0, the pool will destroy all idle resources
* in the pool, but replace them with newly created resources up to the
* specified factory.min value. If this is not desired, set factory.min
* to zero before calling destroyAllNow()
*
* @returns {Promise}
*/
destroyAllNow() {
this._log("force destroying all objects", "info");
const willDie = this._availableObjects.slice();
const todo = willDie.length;
this._removeIdleScheduled = false;
clearTimeout(this._removeIdleTimer);
return new Promise(resolve => {
if (todo === 0) {
return resolve();
}
let resource;
let done = 0;
while ((resource = willDie.shift())) {
this.destroy(resource.resource);
++done;
if (done === todo && resolve) {
return resolve();
}
}
});
}
}
exports.Pool = Pool;
exports.default = Pool;
exports.TimeoutError = require("./TimeoutError").TimeoutError;
+5
View File
@@ -0,0 +1,5 @@
"use strict";
class TimeoutError extends Error {}
exports.TimeoutError = TimeoutError;
+73
View File
@@ -0,0 +1,73 @@
{
"_args": [
[
"sequelize-pool@2.3.0",
"/home/app"
]
],
"_from": "sequelize-pool@2.3.0",
"_id": "sequelize-pool@2.3.0",
"_inBundle": false,
"_integrity": "sha512-Ibz08vnXvkZ8LJTiUOxRcj1Ckdn7qafNZ2t59jYHMX1VIebTAOYefWdRYFt6z6+hy52WGthAHAoLc9hvk3onqA==",
"_location": "/sequelize-pool",
"_phantomChildren": {},
"_requested": {
"type": "version",
"registry": true,
"raw": "sequelize-pool@2.3.0",
"name": "sequelize-pool",
"escapedName": "sequelize-pool",
"rawSpec": "2.3.0",
"saveSpec": null,
"fetchSpec": "2.3.0"
},
"_requiredBy": [
"/sequelize"
],
"_resolved": "https://registry.npmjs.org/sequelize-pool/-/sequelize-pool-2.3.0.tgz",
"_spec": "2.3.0",
"_where": "/home/app",
"author": {
"name": "Sushant",
"email": "sushantdhiman@outlook.com"
},
"bugs": {
"url": "https://github.com/sushantdhiman/sequelize-pool/issues"
},
"dependencies": {},
"description": "Resource pooling for Node.JS",
"devDependencies": {
"eslint": "^5.16.0",
"eslint-config-prettier": "^4.3.0",
"eslint-plugin-prettier": "^3.1.0",
"prettier": "1.18.2",
"tap": "^12.7.0"
},
"engines": {
"node": ">= 6.0.0"
},
"files": [
"lib"
],
"homepage": "https://github.com/sushantdhiman/sequelize-pool#readme",
"keywords": [
"pool",
"pooling",
"throttle",
"sequelize"
],
"license": "MIT",
"main": "lib/Pool.js",
"name": "sequelize-pool",
"repository": {
"type": "git",
"url": "git+ssh://git@github.com/sushantdhiman/sequelize-pool.git"
},
"scripts": {
"lint": "eslint lib test",
"pretty": "prettier lib/**/*.js test/**/*.js --write",
"test": "npm run lint && npm run test:raw",
"test:raw": "tap test/**/*-test.js"
},
"version": "2.3.0"
}